 | The last Llama (Scout/Maverick) was released a year ago. Since then US based releases have been super rare: Granite 3.3, GPT-OSS 20B & 120B, Nemotron 3 Nano / Super and now Gemma 4. Can't even compare to the solid Chinese open model output or Qwens, DeepSeeks, Kimis, MiniMaxes, GLMs, MiMos, Seeds, etc.. Gemma 4 is like a breath of fresh air. Not just the model itself, but the rollout, the beauty, the innovation: K=V in global attention, Per-Layer Embeddings, tri-modal minis (E4B, E2B), etc. Most of my local LLM usage used to be via rented GPUs: Google Cloud, AWS, etc. But about a month ago I decided to bring it all home, and bought a shiny M5 Max MacBook Pro 128GB. It is a beast of a laptop, but also opens up the kind of models I can run locally: 128GB of unified RAM and all. Besides the cost, the true benefit of running models locally is privacy. I never fell easy sending my data to "OpenRouter => Model A" or even hosting it in AWS on P4d/P4de instances (NVIDIA A100): it is still my data, and it is not home. where I am. But my laptop is. When it comes to LLMs, unless it is research or coding finding utility is difficult. But I have kids, and they have school, and if anything is super messy in terms of organization, variety of disconnected systems where the kids data lives, communication inconsistencies, that would be US public schools. But being a parent is fun, and this mess is a great fit for LLMs to make sense of. Local LLMs solve the last piece: my kids data stay on my laptop at home. So it began. I loaded all I could to my 128GB friendly beast and start looking at which models are good for what. The flow is not difficult: go to many different school affiliated websites, some have APIs, some I need to playwright screen scape, some are a little of both plus funky captchas and logins, etc. Then, when on "a" website, some teachers have things inside a slide deck on a "slide 13", some in some obscure folders, others on different systems buried under many irrelevant links. LLMs need to scout all this ambiguity and come back to be with a clear signals of what is due tomorrow, this week; what the grades are, why they are what they are, etc. Again, a great use case for LLM, since it is lots of unorganized text with a clear goal to optimize for. You maybe thinking just about now: "OpenClaw". And you would be correct, this is what I have started from, but then I realized that OpenClaw is as good as the set of LLMs behind it. Also if I schedule a vanilla OS cron that invokes a "school skill", the number of tokens sent to LLM goes from 10K to about 600. And while I do have an OpenClaw running on VPS / OpenRouter, this was not (maybe yet) a good use of it. In order to rank local models I scavenged a few problems over the years that I had to solve with big boys: Claude, OpenAI, Grok and Gemini. They are nice enough to record everything we talk about, which is anything but local, but in this case gave me a chance to collect a few problems and convert them to prompts with rubrics. I then wrote a script to start making sense of what works for me vs. what is advertised and/or works for others. The script grew fast, and was missing look and feel, so I added UI to it: https://github.com/tolitius/cupel Besides the usual general problems, I used a few specific prompts that had tool use and muli-turns (multiple steps composed via tool calling) focused specifically on school related activities. After a few nights and trial and error, I found that " And then Gemma 4 came around. Interestingly, at least for my use case, " (here tested Gemma 4 via OpenRouter to avoid any misconfiguration on my side + 2x faster) But I suspect I still need to learn "The Way of Gemma" to make it work much better. It really is a giant leap forward given its size vs. quality. After all, at 31B, although dense, it stands side by side with 122B. [link] [comments] |
M5 Max 128GB, 17 models, 23 prompts: Qwen 3.5 122B is still a local king
Reddit r/LocalLLaMA / 4/8/2026
💬 OpinionSignals & Early TrendsTools & Practical UsageModels & Research
Key Points
- The author argues that among locally runnable large language models, Qwen 3.5 122B remains a top choice, noting strong performance compared with many other open models available for local use.
- They highlight recent momentum in open-weight models and specifically praise Gemma 4’s architectural and rollout innovations, such as K=V global attention and per-layer embeddings.
- After previously relying on rented GPU capacity, the author describes switching to a home setup using an M5 Max MacBook Pro with 128GB unified RAM to run models locally.
- The piece emphasizes privacy benefits of local inference versus routing data through intermediaries or cloud hosting services, while keeping all children/school-related data on-device.
- The author outlines a practical workflow where LLMs help aggregate messy, multi-site school information (often buried across logins, slide decks, and web pages) to extract deadlines, grades, and due items.
