AI Navigate

Has increasing the number of experts used in MoE models ever meaningfully helped?

Reddit r/LocalLLaMA / 3/16/2026

💬 OpinionSignals & Early TrendsIdeas & Deep AnalysisModels & Research
Read original →
共有:

Key Points

  • The post revisits whether increasing the number of experts in Mixture-of-Experts (MoE) models yields meaningful performance gains, citing past debates around Qwen3-30B-A3B and Qwen3-30b-A6B.
  • It notes that adjusting the MoE expert count is easy to configure in Llama-CPP, but there hasn’t been notable recent experimentation with larger numbers of experts.
  • The author is explicitly asking the community if anyone has conducted new tests or measurements with more MoE experts.
  • The discussion underscores ongoing uncertainty about MoE scaling and the need for empirical comparisons of accuracy, compute, and memory costs when increasing the number of experts.

I remember there was a lot of debate as to whether or not this was worthwhile back when Qwen3-30B-A3B came out. A few people even swore by "Qwen3-30b-A6B" for a short while.

It's still an easy configuration in Llama-CPP, but I don't really see any experimentation with it anymore.

Has anyone been testing around with this much?

submitted by /u/ForsookComparison
[link] [comments]

関連おすすめサービス

※当サイトはアフィリエイト広告を利用しています
🎧

Notta搭載AI議事録イヤホン ZENCHORD1

AI時代の仕事術。Notta搭載で会議の議事録を自動生成するスマートイヤホン。

🎙️

AI搭載ボイスレコーダー Plaud

世界100万人が愛用。AIで文字起こし・要約を自動化するボイスレコーダー。

🖼️

画像高画質化AIツール Aiarty Image Enhancer

AIで画像を高画質化。写真・イラストを簡単にアップスケール。