GGUF (llama.cpp) vs MLX Round 2: Your feedback tested, two models, five runtimes. Ollama adds overhead. My conclusion. Thoughts?

Two weeks ago I posted here that MLX was slower than GGUF on my M1 Max. You gave feedback, pointed out I picked possibly the worst model for MLX. Broken prompt caching (mlx-lm#903), hybrid attention MLX can't optimize, bf16 on a chip that doesn't do bf16.

So I went and tested almost all of your hints and recommendations.
Two mature models (Gemma 12B QAT, Qwen3 30B-A3B), five runtimes, and the bf16→fp16 fix u/bakawolf123 suggested for M1/M2 chips. Also compiled llama.cpp from source to check if LM Studio adds overhead. Same M1 Max 64GB.

After the fp16 conversion, most scenarios are single-digit differences. But its still not a "Just use MLX decision".

Here is Qwen3 30B-A3B effective tok/s (higher is better)

Scenario	MLX (bf16)	MLX (fp16)	GGUF Q4_K_M
Creative writing	53.7	52.7	56.1
Doc classification	26.4	32.8	33.7
Ops agent (8 turns)	35.7	38.4	41.7
Prefill stress (8K ctx)	6.0	8.6	7.6

Generation speed is basically tied with this model: 58 tok/s GGUF vs 55-56 MLX. The "57 vs 29" from Part 1 was the model, not the engine.

Interesting: Runtimes matter more than the engine.
Qwen3 ops agent (higher is better)

Runtime	Engine	eff tok/s
LM Studio	llama.cpp GGUF	41.7
llama.cpp (compiled)	llama.cpp GGUF	41.4
oMLX	MLX	38.0
Ollama	llama.cpp GGUF	26.0 (-37%)

LM Studio adds no overhead compared to raw llama.cpp. Verified by compiling with Metal support myself.
Ollama runs the same engine and is 37% slower for this model.
Consistently slower compared to LM Studio GGUF across both articles, all benchmarks I did models. Something in the Go wrapper seems to be expensive.

On the MLX side: oMLX is 2.2x faster than LM Studio MLX on multi-turn. But I also tested Gemma 12B, where LM Studio's caching works fine. Interestingly oMLX and LM Studio MLX produce similar numbers there. So oMLX fixes caching problems, not MLX performance in general. Still the best MLX runtime though.
Credit to the devs, it's well-engineered software. However: I don't have stability data yet. So not sure how stability behaves over time.

bf16 fix for anyone on M1/M2:

pip install mlx-lm mlx_lm.convert --hf-path <your-model> --mlx-path <output> --dtype float16 

Under a minute, no quality loss, recovers 40-70% of prefill penalty. M3+ has native bf16 so this doesn't apply there.

What I came across during research is the MLX quant quality concern: MLX 4-bit and GGUF Q4_K_M are not the same thing despite both saying "4-bit." But there is some movement in that area.

GGUF K-quants allocate more bits to sensitive layers, MLX applies uniform depth. The llama.cpp project measured a 4.7x perplexity difference between uniform Q4_0 and Q4_K_M on a 7B model. I haven't tested this myself yet. Would be interesting to see if that shows up in real output quality with the models I benchmarked. JANG-Q is working on bringing adaptive quantization to MLX.

Where I landed:

• LM Studio + GGUF for most things. Better quants, no workarounds, decent effective speed, just works, stable.
• oMLX if you use Qwen 3.5 MLX for new models, especially multi modal like qwen 3.5(which is great!) or longer agentic conversations with the same system prompt. A noticeable speed boost. The caching layers of oMLX are just great.
• Skip Ollama. The overhead hurts.

Still looking for M2 and M4 data.
AlexTzk submitted M3 Max results (oMLX scales from 38 to 71 eff tok/s, roughly proportional to GPU cores). M2 and M4 are still missing.

Benchmark yourself if you feel like it

https://github.com/famstack-dev/local-llm-bench 

Contribute results as Pull Request and I'll add your hardware or just use it to test your use-case. But there is no need to contribute. Comment with your results and findings if you happen to run something would be great**.**
What makes this bench different? It uses real-world scenarios and measures effective tokens/s not just the generation. It is easy to add and test custom scenarios.

Now enough benchmarking and back to solving actual problems :)

Thoughts on this journey? Some more tips & tricks?

Also happy do discuss over the channel linked in my profile.

Full writeup with all charts and some research data: famstack.dev/guides/mlx-vs-gguf-part-2-isolating-variables