Updated Qwen3.5-9B Quantization Comparison

Reddit r/LocalLLaMA / 4/14/2026

💬 OpinionDeveloper Stack & InfrastructureTools & Practical UsageModels & Research

Key Points

  • The post provides a data-driven KLD (KL divergence) evaluation comparing community GGUF quantizations of Qwen3.5-9B against a BF16 baseline.
  • It argues that KLD is a more reliable indicator of “faithfulness” than PPL because PPL can be affected by dataset noise and lucky runs, while KLD measures distribution drift from the baseline.
  • The selection guidance is straightforward: choose the quantization with the lowest KLD score to minimize information loss.
  • Early results highlight the Q8_0 variants (e.g., eaddario/Qwen3.5-9B-Q8_0) as among the lowest-KLD options, while still reporting comparable sizes around ~8.9–12.1 GiB depending on the quantization.
  • The article emphasizes that the goal is to help users pick the right file/quantization rather than relying on availability or assumptions.
Updated Qwen3.5-9B Quantization Comparison

This is a KLD eval across community GGUF quants of Qwen3.5-9B, comparing mean KLD to the BF16 baseline.

The goal is to give people a data-driven basis for picking a file rather than just grabbing whatever is available.

KLD (KL Divergence): "Faithfulness." It shows how much the quantized model's probability distribution drifts from a baseline (the probability distribution of the original weights). Lower = closer.

Since we are trying to see how much information we've lost and since PPL is noisy as it can get a better score by pure luck, KLD is better as it is not relying on the dataset but on the baseline.

If you need the most faithfull quant, pick the one with the lowest KLD.

This is a dense plot, sorry about that.

 KLD RANKINGS 

bolded KLD Score <0.01 - lower is better

Quantization Size_GiB PPL_Score KLD_Score
eaddario/Qwen3.5-9B-Q8_0 8.873 19.177240 0.001198
unsloth/Qwen3.5-9B-UD-Q8_K_XL 12.083 19.183966 0.001243
bartowski/Qwen_Qwen3.5-9B-Q8_0 8.89 19.184374 0.001405
lmstudio-community/Qwen3.5-9B-Q8_0 8.873 19.184470 0.001410
ZeroWw/Qwen3.5-9B.q8_p 8.873 19.189372 0.001412
unsloth/Qwen3.5-9B-Q8_0 8.873 19.175181 0.001433
AaryanK/Qwen3.5-9B.q8_0 8.873 19.177790 0.001445
DevQuasar/Qwen.Qwen3.5-9B.Q8_0 8.873 19.186216 0.001464
ZeroWw/Qwen3.5-9B.q8_0 10.649 19.188892 0.001679
unsloth/Qwen3.5-9B-UD-Q6_K_XL 8.156 19.193957 0.001910
bartowski/Qwen_Qwen3.5-9B-Q6_K_L 7.592 19.202837 0.002371
bartowski/Qwen_Qwen3.5-9B-Q6_K 7.134 19.213584 0.002813
unsloth/Qwen3.5-9B-Q6_K 6.946 19.200108 0.003080
Mungert/Qwen3.5-9B-q6_k_m 6.872 19.235596 0.003609
mradermacher/Qwen3.5-9B.i1-Q6_K 6.854 19.234343 0.003735
ZeroWw/Qwen3.5-9B.q6_k 9.089 19.259351 0.004625
AaryanK/Qwen3.5-9B.q6_k 6.854 19.258445 0.004779
DevQuasar/Qwen.Qwen3.5-9B.Q6_K 6.854 19.272393 0.004801
lmstudio-community/Qwen3.5-9B-Q6_K 6.854 19.263994 0.004905
bartowski/Qwen_Qwen3.5-9B-Q5_K_L 6.976 19.268033 0.006068
unsloth/Qwen3.5-9B-UD-Q5_K_XL 6.281 19.260486 0.006419
bartowski/Qwen_Qwen3.5-9B-Q5_K_M 6.392 19.274078 0.006604
Mungert/Qwen3.5-9B-q5_k_m 6.336 19.263969 0.006714
unsloth/Qwen3.5-9B-Q5_K_M 6.126 19.298573 0.007290
bartowski/Qwen_Qwen3.5-9B-Q5_K_S 6.078 19.271394 0.008110
unsloth/Qwen3.5-9B-Q5_K_S 5.924 19.330239 0.009137
bartowski/Qwen_Qwen3.5-9B-Q4_K_L 6.188 19.377795 0.015064
unsloth/Qwen3.5-9B-UD-Q4_K_XL 5.556 19.355771 0.015238
bartowski/Qwen_Qwen3.5-9B-Q4_K_M 5.485 19.409285 0.016754
AaryanK/Qwen3.5-9B.q5_0 5.872 19.516510 0.019535
bartowski/Qwen_Qwen3.5-9B-Q4_K_S 5.197 19.426160 0.020576
eaddario/Qwen3.5-9B-Q6_K 6.854 19.648966 0.021010
bartowski/Qwen_Qwen3.5-9B-Q4_1 5.512 19.467238 0.023208
byteshape/Qwen3.5-9B-Q5_K_S-5.10bpw 5.329 19.532163 0.023510
byteshape/Qwen3.5-9B-IQ4_XS-4.98bpw 5.198 19.558089 0.024250
bartowski/Qwen_Qwen3.5-9B-IQ4_NL 5.07 19.498178 0.024696
mradermacher/Qwen3.5-9B.i1-Q5_K_M 6.074 19.706723 0.025498
bartowski/Qwen_Qwen3.5-9B-IQ4_XS 4.846 19.514750 0.025705
eaddario/Qwen3.5-9B-Q5_K 6.024 19.714336 0.026344
Mungert/Qwen3.5-9B-iq4_nl 4.972 19.562374 0.026716
mradermacher/Qwen3.5-9B.i1-Q5_K_S 5.872 19.725820 0.027342
Mungert/Qwen3.5-9B-iq4_xs 4.743 19.594639 0.027766
mradermacher/Qwen3.5-9B.i1-IQ4_NL 4.952 19.591508 0.027867
mradermacher/Qwen3.5-9B.i1-IQ4_XS 4.722 19.621767 0.028870
ZeroWw/Qwen3.5-9B.q5_k 8.435 19.830399 0.031931
byteshape/Qwen3.5-9B-Q5_K_S-4.75bpw 4.958 19.681021 0.032144
AaryanK/Qwen3.5-9B.q5_k_m 6.074 19.846397 0.032233
DevQuasar/Qwen.Qwen3.5-9B.Q5_K_M 6.074 19.852639 0.032304
eaddario/Qwen3.5-9B-Q4_K-B 5.485 19.858831 0.033141
AaryanK/Qwen3.5-9B.q5_1 6.334 19.748779 0.034313
Mungert/Qwen3.5-9B-q4_k_m 5.564 19.841286 0.034431
AaryanK/Qwen3.5-9B.q5_k_s 5.872 19.864724 0.034770
DevQuasar/Qwen.Qwen3.5-9B.Q5_K_S 5.872 19.882870 0.034819
eaddario/Qwen3.5-9B-Q4_K-U 5.29 19.912657 0.036301
llmware/Qwen3.5-9B-Q4_K_M 5.29 19.854865 0.036925
unsloth/Qwen3.5-9B-Q4_K_M 5.29 19.859386 0.037104
eaddario/Qwen3.5-9B-Q4_K 5.243 19.959778 0.037505
eaddario/Qwen3.5-9B-Q4_K_M-naive 5.243 19.898625 0.038486
byteshape/Qwen3.5-9B-Q5_K_S-4.60bpw 4.802 19.790823 0.038704
mradermacher/Qwen3.5-9B.i1-Q4_K_M 5.241 19.908672 0.039594
unsloth/Qwen3.5-9B-Q4_K_S 5.024 19.908924 0.040750
byteshape/Qwen3.5-9B-IQ4_XS-4.43bpw 4.626 19.800843 0.041636
unsloth/Qwen3.5-9B-Q4_1 5.436 19.903143 0.042209
unsloth/Qwen3.5-9B-IQ4_NL 5.002 19.937468 0.042506
mradermacher/Qwen3.5-9B.i1-Q4_K_S 4.974 19.977873 0.043795
unsloth/Qwen3.5-9B-IQ4_XS 4.814 19.952831 0.043811
bartowski/Qwen_Qwen3.5-9B-Q4_0 5.074 19.864063 0.044698
mradermacher/Qwen3.5-9B.i1-Q4_1 5.41 19.993730 0.044785
unsloth/Qwen3.5-9B-UD-Q3_K_XL 4.707 19.833348 0.046158
steampunque/Qwen3.5-9B.Q4_K_H 5.663 19.988807 0.047851
byteshape/Qwen3.5-9B-IQ4_XS-4.20bpw 4.384 19.994381 0.051704
mradermacher/Qwen3.5-9B.i1-Q4_0 4.96 20.031403 0.052661
bartowski/Qwen_Qwen3.5-9B-Q3_K_XL 5.556 20.092393 0.058763
Mungert/Qwen3.5-9B-iq3_s 4.418 20.059272 0.059535
Mungert/Qwen3.5-9B-iq3_m 4.418 20.072130 0.059772
ZeroWw/Qwen3.5-9B.q8q4 5.944 20.261738 0.060661
DevQuasar/Qwen.Qwen3.5-9B.Q4_K_M 5.241 20.299136 0.062447
AaryanK/Qwen3.5-9B.q4_k_m 5.241 20.273619 0.062641
bartowski/Qwen_Qwen3.5-9B-Q3_K_L 4.727 20.110764 0.062688
lmstudio-community/Qwen3.5-9B-Q4_K_M 5.241 20.284701 0.063009
unsloth/Qwen3.5-9B-Q4_0 5.01 20.336317 0.064799
bartowski/Qwen_Qwen3.5-9B-Q3_K_M 4.533 20.152567 0.067070
AaryanK/Qwen3.5-9B.q4_0 4.948 20.244066 0.067778
AaryanK/Qwen3.5-9B.q4_k_s 4.974 20.421610 0.071165
DevQuasar/Qwen.Qwen3.5-9B.Q4_K_S 4.974 20.425910 0.071280
Mungert/Qwen3.5-9B-q3_k_m 4.861 20.419780 0.073549
eaddario/Qwen3.5-9B-Q3_K 4.306 20.544374 0.075912
bartowski/Qwen_Qwen3.5-9B-IQ3_M 4.349 20.411438 0.076311
Mungert/Qwen3.5-9B-iq3_xs 4.289 20.262784 0.076315
keyuan01/qwen3.5-9b-mix 4.508 20.462178 0.082440
mradermacher/Qwen3.5-9B.i1-Q3_K_L 4.493 20.475629 0.082614
AaryanK/Qwen3.5-9B.q4_1 5.41 20.693102 0.084915
mradermacher/Qwen3.5-9B.i1-Q3_K_M 4.299 20.565871 0.087404
bartowski/Qwen_Qwen3.5-9B-IQ3_XS 4.197 20.598822 0.087739
mradermacher/Qwen3.5-9B.i1-IQ3_M 4.112 20.568608 0.087748
unsloth/Qwen3.5-9B-Q3_K_M 4.353 20.668516 0.088135
Mungert/Qwen3.5-9B-iq3_xxs 3.982 20.749878 0.094229
mradermacher/Qwen3.5-9B.i1-IQ3_S 3.971 20.694098 0.094688
byteshape/Qwen3.5-9B-Q4_K_S-3.92bpw 4.095 20.856006 0.100597
bartowski/Qwen_Qwen3.5-9B-Q3_K_S 4.3 20.918237 0.101205
mradermacher/Qwen3.5-9B.i1-IQ3_XS 3.852 20.825952 0.105562
AaryanK/Qwen3.5-9B.q3_k_l 4.493 21.068526 0.109296
DevQuasar/Qwen.Qwen3.5-9B.Q3_K_L 4.493 21.070038 0.109460
bartowski/Qwen_Qwen3.5-9B-IQ3_XXS 4.052 21.074602 0.113778
DevQuasar/Qwen.Qwen3.5-9B.Q3_K_M 4.299 21.186911 0.117853
unsloth/Qwen3.5-9B-UD-IQ3_XXS 3.74 21.337685 0.122042
byteshape/Qwen3.5-9B-IQ4_XS-3.60bpw 3.766 21.935245 0.142608
mradermacher/Qwen3.5-9B.i1-Q3_K_S 3.967 21.834745 0.146521
unsloth/Qwen3.5-9B-Q3_K_S 4.02 22.041631 0.151734
mradermacher/Qwen3.5-9B.i1-IQ3_XXS 3.533 21.757513 0.155960
Mungert/Qwen3.5-9B-q2_k_m 4.11 22.583041 0.187712
bartowski/Qwen_Qwen3.5-9B-Q2_K_L 4.649 23.033036 0.195621
DevQuasar/Qwen.Qwen3.5-9B.Q3_K_S 3.967 23.241273 0.204858
byteshape/Qwen3.5-9B-IQ3_S-3.15bpw 3.291 23.628691 0.221494
byteshape/Qwen3.5-9B-IQ3_S-3.00bpw 3.137 24.952801 0.278109
byteshape/Qwen3.5-9B-Q3_K_S-3.46bpw 3.614 25.713151 0.310829
byteshape/Qwen3.5-9B-IQ3_S-2.81bpw 2.938 27.095131 0.362968
 SIZE VS KLD RANKINGS - Qwen3.5-9B-bf16 

Efficiency Score: √(Normalized Size² + Normalized KLD²) - bolded KLD Score <0.01 - lower is better

Rank Quantization Size (GiB) KLD Eff. Score
1 mradermacher/Qwen3.5-9B.i1-IQ4_XS 4.722 0.028870 0.209539
2 Mungert/Qwen3.5-9B-iq4_xs 4.743 0.027766 0.210595
3 byteshape/Qwen3.5-9B-IQ4_XS-4.20bpw 4.384 0.051704 0.210931
4 byteshape/Qwen3.5-9B-IQ4_XS-4.43bpw 4.626 0.041636 0.215789
5 bartowski/Qwen_Qwen3.5-9B-IQ4_XS 4.846 0.025705 0.219361
6 Mungert/Qwen3.5-9B-iq3_s 4.418 0.059535 0.228461
7 byteshape/Qwen3.5-9B-Q5_K_S-4.60bpw 4.802 0.038704 0.228678
8 Mungert/Qwen3.5-9B-iq3_m 4.418 0.059772 0.228923
9 unsloth/Qwen3.5-9B-UD-Q3_K_XL 4.707 0.046158 0.229921
10 mradermacher/Qwen3.5-9B.i1-IQ4_NL 4.952 0.027867 0.232240
11 Mungert/Qwen3.5-9B-iq4_nl 4.972 0.026716 0.233334
12 unsloth/Qwen3.5-9B-IQ4_XS 4.814 0.043811 0.236552
13 byteshape/Qwen3.5-9B-Q5_K_S-4.75bpw 4.958 0.032144 0.236871
14 bartowski/Qwen_Qwen3.5-9B-IQ4_NL 5.070 0.024696 0.242012
15 mradermacher/Qwen3.5-9B.i1-Q4_K_S 4.974 0.043795 0.251854
16 bartowski/Qwen_Qwen3.5-9B-Q3_K_M 4.533 0.067070 0.252138
17 bartowski/Qwen_Qwen3.5-9B-Q4_K_S 5.197 0.020576 0.252761
18 unsloth/Qwen3.5-9B-IQ4_NL 5.002 0.042506 0.252937
19 unsloth/Qwen3.5-9B-Q4_K_S 5.024 0.040750 0.252950
20 Mungert/Qwen3.5-9B-iq3_xs 4.289 0.076315 0.254829
21 eaddario/Qwen3.5-9B-Q3_K 4.306 0.075912 0.255008
22 byteshape/Qwen3.5-9B-IQ4_XS-4.98bpw 5.198 0.024250 0.255212
23 bartowski/Qwen_Qwen3.5-9B-IQ3_M 4.349 0.076311 0.258679
24 bartowski/Qwen_Qwen3.5-9B-Q3_K_L 4.727 0.062688 0.259151
25 bartowski/Qwen_Qwen3.5-9B-Q4_0 5.074 0.044698 0.262704
26 mradermacher/Qwen3.5-9B.i1-Q4_0 4.960 0.052661 0.262913
27 byteshape/Qwen3.5-9B-Q5_K_S-5.10bpw 5.329 0.023510 0.268630
28 eaddario/Qwen3.5-9B-Q4_K 5.243 0.037505 0.271296
29 mradermacher/Qwen3.5-9B.i1-IQ3_M 4.112 0.087748 0.271508
30 eaddario/Qwen3.5-9B-Q4_K_M-naive 5.243 0.038486 0.272310
31 mradermacher/Qwen3.5-9B.i1-Q4_K_M 5.241 0.039594 0.273283
32 eaddario/Qwen3.5-9B-Q4_K-U 5.290 0.036301 0.274885
33 llmware/Qwen3.5-9B-Q4_K_M 5.290 0.036925 0.275498
34 unsloth/Qwen3.5-9B-Q4_K_M 5.290 0.037104 0.275676
35 bartowski/Qwen_Qwen3.5-9B-IQ3_XS 4.197 0.087739 0.276002
36 mradermacher/Qwen3.5-9B.i1-Q3_K_M 4.299 0.087404 0.280946
37 Mungert/Qwen3.5-9B-iq3_xxs 3.982 0.094229 0.281356
38 bartowski/Qwen_Qwen3.5-9B-Q4_K_M 5.485 0.016754 0.281813
39 mradermacher/Qwen3.5-9B.i1-IQ3_S 3.971 0.094688 0.282033
40 mradermacher/Qwen3.5-9B.i1-Q3_K_L 4.493 0.082614 0.282064
41 keyuan01/qwen3.5-9b-mix 4.508 0.082440 0.282674
42 unsloth/Qwen3.5-9B-Q3_K_M 4.353 0.088135 0.285815
43 AaryanK/Qwen3.5-9B.q4_0 4.948 0.067778 0.286669
44 unsloth/Qwen3.5-9B-Q4_0 5.010 0.064799 0.286779
45 bartowski/Qwen_Qwen3.5-9B-Q4_1 5.512 0.023208 0.287966
46 unsloth/Qwen3.5-9B-UD-Q4_K_XL 5.556 0.015238 0.288895
47 Mungert/Qwen3.5-9B-q3_k_m 4.861 0.073549 0.290196
48 eaddario/Qwen3.5-9B-Q4_K-B 5.485 0.033141 0.292174
49 AaryanK/Qwen3.5-9B.q4_k_s 4.974 0.071165 0.294908
50 DevQuasar/Qwen.Qwen3.5-9B.Q4_K_S 4.974 0.071280 0.295117
51 unsloth/Qwen3.5-9B-Q4_1 5.436 0.042209 0.295744
52 mradermacher/Qwen3.5-9B.i1-Q4_1 5.410 0.044785 0.295947
53 Mungert/Qwen3.5-9B-q4_k_m 5.564 0.034431 0.301487
54 byteshape/Qwen3.5-9B-Q4_K_S-3.92bpw 4.095 0.100597 0.302487
55 DevQuasar/Qwen.Qwen3.5-9B.Q4_K_M 5.241 0.062447 0.303452
56 AaryanK/Qwen3.5-9B.q4_k_m 5.241 0.062641 0.303751
57 lmstudio-community/Qwen3.5-9B-Q4_K_M 5.241 0.063009 0.304321
58 mradermacher/Qwen3.5-9B.i1-IQ3_XS 3.852 0.105562 0.305304
59 bartowski/Qwen_Qwen3.5-9B-Q3_K_S 4.300 0.101205 0.314005
60 steampunque/Qwen3.5-9B.Q4_K_H 5.663 0.047851 0.324685
61 AaryanK/Qwen3.5-9B.q5_0 5.872 0.019535 0.324810
62 unsloth/Qwen3.5-9B-Q5_K_S 5.924 0.009137 0.327254
63 bartowski/Qwen_Qwen3.5-9B-Q3_K_XL 5.556 0.058763 0.327527
64 mradermacher/Qwen3.5-9B.i1-Q5_K_S 5.872 0.027342 0.328869
65 AaryanK/Qwen3.5-9B.q5_k_s 5.872 0.034770 0.333982
66 DevQuasar/Qwen.Qwen3.5-9B.Q5_K_S 5.872 0.034819 0.334020
67 bartowski/Qwen_Qwen3.5-9B-IQ3_XXS 4.052 0.113778 0.334185
68 AaryanK/Qwen3.5-9B.q3_k_l 4.493 0.109296 0.343797
69 bartowski/Qwen_Qwen3.5-9B-Q5_K_S 6.078 0.008110 0.343888
70 DevQuasar/Qwen.Qwen3.5-9B.Q3_K_L 4.493 0.109460 0.344191
71 eaddario/Qwen3.5-9B-Q5_K 6.024 0.026344 0.344536
72 unsloth/Qwen3.5-9B-UD-IQ3_XXS 3.740 0.122042 0.345356
73 unsloth/Qwen3.5-9B-Q5_K_M 6.126 0.007290 0.349012
74 mradermacher/Qwen3.5-9B.i1-Q5_K_M 6.074 0.025498 0.349436
75 AaryanK/Qwen3.5-9B.q5_k_m 6.074 0.032233 0.353487
76 DevQuasar/Qwen.Qwen3.5-9B.Q5_K_M 6.074 0.032304 0.353535
77 DevQuasar/Qwen.Qwen3.5-9B.Q3_K_M 4.299 0.117853 0.355143
78 AaryanK/Qwen3.5-9B.q4_1 5.410 0.084915 0.355835
79 bartowski/Qwen_Qwen3.5-9B-Q4_K_L 6.188 0.015064 0.357446
80 unsloth/Qwen3.5-9B-UD-Q5_K_XL 6.281 0.006419 0.365840
81 ZeroWw/Qwen3.5-9B.q8q4 5.944 0.060661 0.367509
82 Mungert/Qwen3.5-9B-q5_k_m 6.336 0.006714 0.371882
83 bartowski/Qwen_Qwen3.5-9B-Q5_K_M 6.392 0.006604 0.377988
84 AaryanK/Qwen3.5-9B.q5_1 6.334 0.034313 0.382466
85 byteshape/Qwen3.5-9B-IQ4_XS-3.60bpw 3.766 0.142608 0.401233
86 mradermacher/Qwen3.5-9B.i1-Q3_K_S 3.967 0.146521 0.417162
87 mradermacher/Qwen3.5-9B.i1-Q6_K 6.854 0.003735 0.428270
88 AaryanK/Qwen3.5-9B.q6_k 6.854 0.004779 0.428327
89 DevQuasar/Qwen.Qwen3.5-9B.Q6_K 6.854 0.004801 0.428328
90 lmstudio-community/Qwen3.5-9B-Q6_K 6.854 0.004905 0.428335
91 Mungert/Qwen3.5-9B-q6_k_m 6.872 0.003609 0.430232
92 eaddario/Qwen3.5-9B-Q6_K 6.854 0.021010 0.431700
93 unsloth/Qwen3.5-9B-Q3_K_S 4.020 0.151734 0.432604
94 mradermacher/Qwen3.5-9B.i1-IQ3_XXS 3.533 0.155960 0.432711
95 unsloth/Qwen3.5-9B-Q6_K 6.946 0.003080 0.438303
96 bartowski/Qwen_Qwen3.5-9B-Q5_K_L 6.976 0.006068 0.441758
97 bartowski/Qwen_Qwen3.5-9B-Q6_K 7.134 0.002813 0.458852
98 bartowski/Qwen_Qwen3.5-9B-Q6_K_L 7.592 0.002371 0.508922
99 Mungert/Qwen3.5-9B-q2_k_m 4.110 0.187712 0.531250
100 bartowski/Qwen_Qwen3.5-9B-Q2_K_L 4.649 0.195621 0.569058
101 unsloth/Qwen3.5-9B-UD-Q6_K_XL 8.156 0.001910 0.570588
102 DevQuasar/Qwen.Qwen3.5-9B.Q3_K_S 3.967 0.204858 0.574089
103 ZeroWw/Qwen3.5-9B.q5_k 8.435 0.031931 0.607067
104 byteshape/Qwen3.5-9B-IQ3_S-3.15bpw 3.291 0.221494 0.610162
105 eaddario/Qwen3.5-9B-Q8_0 8.873 0.001198 0.648989
106 lmstudio-community/Qwen3.5-9B-Q8_0 8.873 0.001410 0.648989
107 ZeroWw/Qwen3.5-9B.q8_p 8.873 0.001412 0.648989
108 unsloth/Qwen3.5-9B-Q8_0 8.873 0.001433 0.648989
109 AaryanK/Qwen3.5-9B.q8_0 8.873 0.001445 0.648989
110 DevQuasar/Qwen.Qwen3.5-9B.Q8_0 8.873 0.001464 0.648989
111 bartowski/Qwen_Qwen3.5-9B-Q8_0 8.890 0.001405 0.650848
112 ZeroWw/Qwen3.5-9B.q6_k 9.089 0.004625 0.672675
113 byteshape/Qwen3.5-9B-IQ3_S-3.00bpw 3.137 0.278109 0.765743
114 ZeroWw/Qwen3.5-9B.q8_0 10.649 0.001679 0.843194
115 byteshape/Qwen3.5-9B-Q3_K_S-3.46bpw 3.614 0.310829 0.859064
116 byteshape/Qwen3.5-9B-IQ3_S-2.81bpw 2.938 0.362968 1.000000
117 unsloth/Qwen3.5-9B-UD-Q8_K_XL 12.083 0.001243 1.000000

eval dataset: https://gist.github.com/cmhamiche/788eada03077f4341dfb39df8be012dc 103 chunks at -c 512

ik_llama.cpp: https://github.com/Thireus/ik_llama.cpp/releases/tag/main-b4608-b33a10d

nvidia drivers: 595.97

edit: updated the plot with shapes instead or dots.

submitted by /u/TitwitMuffbiscuit
[link] [comments]