KV Cache Quantization for Self-Forcing Video Generation: A 33-Method Empirical Study

arXiv cs.LG / 3/31/2026

💬 OpinionDeveloper Stack & InfrastructureIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper studies how KV-cache growth limits self-forcing long-horizon video generation and evaluates KV-cache quantization and cache-policy variants to improve memory behavior over longer rollouts.
Across 33 KV-cache compression methods with 610 prompt-level observations, the authors benchmark peak VRAM, runtime, realized compression ratio, VBench quality, BF16-referenced fidelity (SSIM/LPIPS/PSNR), and terminal drift.
A FlowCache-inspired soft-prune INT4 approach is identified as the most practical operating point, achieving about 5.42–5.49× compression and cutting peak VRAM from 19.28 GB to ~11.7 GB with only modest runtime overhead.
Methods targeting maximum compressed fidelity (e.g., PRQ_INT4, QUAROT_KV_INT4) are found to be poor deployment choices due to unacceptable runtime or memory costs.
The study concludes that compression alone can fail when the implementation still reconstructs/retains large BF16 buffers during attention/refresh stages, and provides an empirical harness, workflow, and dashboard to guide future KV-cache integration research.

Abstract

Self-forcing video generation extends a short-horizon video model to longer rollouts by repeatedly feeding generated content back in as context. This scaling path immediately exposes a systems bottleneck: the key-value (KV) cache grows with rollout length, so longer videos require not only better generation quality but also substantially better memory behavior. We present a comprehensive empirical study of KV-cache compression for self-forcing video generation on a Wan2.1-based Self-Forcing stack. Our study covers 33 quantization and cache-policy variants, 610 prompt-level observations, and 63 benchmark-level summaries across two evaluation settings: MovieGen for single-shot 10-second generation and StoryEval for longer narrative-style stability. We jointly evaluate peak VRAM, runtime, realized compression ratio, VBench imaging quality, BF16-referenced fidelity (SSIM, LPIPS, PSNR), and terminal drift. Three findings are robust. First, the strongest practical operating region is a FlowCache-inspired soft-prune INT4 adaptation, which reaches 5.42-5.49x compression while reducing peak VRAM from 19.28 GB to about 11.7 GB with only modest runtime overhead. Second, the highest-fidelity compressed methods, especially PRQ_INT4 and QUAROT_KV_INT4, are not the best deployment choices because they preserve quality at severe runtime or memory cost. Third, nominal compression alone is not sufficient: several methods shrink KV storage but still exceed BF16 peak VRAM because the current integration reconstructs or retains large BF16 buffers during attention and refresh stages. The result is a benchmark harness, analysis workflow, and empirical map of which KV-cache ideas are practical today and which are promising research directions for better memory integration. Code, data products, and the presentation dashboard are available at https://github.com/suraj-ranganath/kv-quant-longhorizon/.