AI Navigate
インサイト最新記事AI大全カオスマップ

The use Q8 a waste of resources?

Reddit r/LocalLLaMA / 5/3/2026

💬 OpinionIdeas & Deep AnalysisTools & Practical Usage
Read original →
共有:

Key Points

  • The post asks whether running a high-quantized LLM (Q8) is an inefficient use of disk space and VRAM compared with using lower-quantized variants like Q6 K.
  • The author compares models in terms of supported context length (e.g., 75k vs 145k) and expected performance (tokens per second), weighing the tradeoffs against hardware cost.
  • Key concerns include whether moving from Q8 to Q6 would significantly reduce model intelligence or overall capability.
  • The author also asks about the impact of quantization on vision capabilities and whether “Q6 K XL” offers a meaningful improvement over “Q6 K.”

I can run G4 31B Q8 XL with ctx 75k and Gwen's 27B and 35B Q8 XL ctx 145k, but I'm wondering if I'm wasting GB of SSD and VRAM.

Is it worth upgrading to Q6 K? To save disk space and gain a little more T/s and more context? Or does intelligence deteriorate significaly "Kld" or "kl"?

Is Vision affected by using Q6?

Q6 K XL is much better than "Q6 K" normal?

submitted by /u/Spiderboyz1
[link] [comments]

関連おすすめサービス

※当サイトはアフィリエイト広告を利用しています
🎧

Notta搭載AI議事録イヤホン ZENCHORD1

AI時代の仕事術。Notta搭載で会議の議事録を自動生成するスマートイヤホン。

🎙️

AI搭載ボイスレコーダー Plaud

世界100万人が愛用。AIで文字起こし・要約を自動化するボイスレコーダー。

🖼️

画像高画質化AIツール Aiarty Image Enhancer

AIで画像を高画質化。写真・イラストを簡単にアップスケール。