Dual DGX Sparks vs Mac Studio M3 Ultra 512GB: Running Qwen3.5 397B locally on both. Here's what I found.

I was spending about $2K/month on Claude API tokens for a personal AI assistant I run through Slack. After about 45 days of that cost pain I decided to go local. Bought both a dual DGX Spark setup and a Mac Studio M3 Ultra 512GB, each cost me about $10K after taxes. Same price, completely different machines. Here is what I learned running Qwen3.5 397B A17B on both.

The Mac Studio

MLX 6 bit quantization, 323GB model loaded into 512GB unified memory. 30 to 40 tok/s generation. The biggest selling point is memory bandwidth at roughly 800 GB/s. That bandwidth is what makes token generation feel smooth on such a massive model in a single box. Setup was easy. Install mlx vlm, point it at the model, done. The weakness is raw compute. Prefill is slow (30+ seconds on a big system prompt with tool definitions) and if you want to do batch embedding alongside inference, you are going to feel it. I also had to write a 500 line async proxy because mlx vlm does not parse tool calls or strip thinking tokens natively.

The Dual Sparks

INT4 AutoRound quantization, 98GB per node loaded across two 128GB nodes via vLLM TP=2. 27 to 28 tok/s generation. The biggest selling point is processing speed. CUDA tensor cores, vLLM kernels, tensor parallelism. Prefill is noticeably faster than the Mac Studio. Batch embedding that takes days on MLX finishes in hours on CUDA. The entire open source GPU ecosystem just works. The weakness is memory bandwidth at roughly 273 GB/s per node, which is why generation tops out lower than the Mac Studio despite having more compute.

The setup was brutal though. Only one QSFP cable works (the second crashes NCCL). Node2's IP is ephemeral and disappears on reboot. The GPU memory utilization ceiling is 0.88 and you have to binary search for it because going to 0.9 starves the OS and 0.85 OOMs at 262K context. Every wrong guess costs you 15 minutes while checkpoint shards reload. You have to flush page cache on BOTH nodes before every model load or you get mystery OOM failures. Some units thermal throttle within 20 minutes. It took me days to get stable.

Why I kept both

I am building a RAG pipeline with Qwen3 Embedding 8B and Qwen3 Reranker 8B for a personal knowledge base. On the Mac Studio, those models would compete with the main model for the same 512GB memory pool. On the Sparks, they get dedicated CUDA and never touch inference memory.

So the architecture ended up being: Mac Studio handles inference only (full 512GB for the model and KV cache). Sparks handle RAG, embedding, reranking, and everything else. They talk over Tailscale.

Head to head numbers

If you can only buy one

I cannot tell you which is better because if one were clearly better I would have returned the other. They optimize for different things.

Mac Studio if you want it to just work, you want that 800 GB/s bandwidth for smooth generation, and you are not planning heavy embedding workloads alongside inference. An RTX 6000 Pro build was my third option but I did not want to build a custom PC on top of everything else I was planning on for this.

Dual Sparks if you are comfortable with Linux and Docker, you want CUDA and vLLM natively, you plan to run RAG or embedding alongside inference, and you are willing to spend days on initial setup for a more powerful platform long term.

The Mac Studio gives you 80% of the experience with 20% of the effort. The Sparks give you more capability but they extract a real cost in setup time.

Break even math

$2K/month API spend. $20K total hardware. 10 months to break even. After that it is free inference forever with complete privacy and no rate limits.

I wrote a longer version of this with more detail on the full build out at https://substack.com/home/post/p-192255754 . Building a series covering the full stack including vLLM tuning, RAG without LangChain, and QLoRA fine tuning a 397B MoE. Happy to answer questions.