I am somewhat convinced by my own testing, that for non-coding, the 9B at UD-Q8_K-XL variant is better than the 27B Q4_K_XL & Q5_K_XL. To me, it felt like going to the highest quant really showed itself with good quality results and faster. Not only that, I am able to pair Qwen3-TTS with it and use a custom voice (I am using Scarlett Johansson's voice). Once the first prompt is loaded and voice is called, it is really fast. I was testing with the same context size for 27 and 9B.
This is mostly about how the quality of the higher end 9B 8-bit quant felt better for general purpose stuff, compared to the 4 or 5 bit quants of 27B. It makes me want to get another GPU to add to my 3090 so that i can run the 27B at 8 bit.
Has anyone seen anything similar.
[link] [comments]




