GRASPrune: Global Gating for Budgeted Structured Pruning of Large Language Models

arXiv cs.AI / 4/22/2026

📰 NewsDeveloper Stack & InfrastructureModels & Research

共有:

Key Points

GRASPrune is a post-pretraining structured pruning framework that jointly prunes FFN channels and KV head groups for large language model efficiency under a single global parameter/compute budget.
It learns lightweight gate scores using a projected straight-through estimator to enforce a hard pruning mask at every step, while keeping the original backbone weights frozen.
After selecting which units to keep, GRASPrune calibrates scaling factors to reduce scale mismatch from pruning and folds them into the remaining weights to produce a smaller dense checkpoint for inference without extra parameters.
On LLaMA-2-7B, GRASPrune prunes 50% of parameters and reports WikiText-2 perplexity of 12.18 while maintaining competitive average zero-shot accuracy, using only calibration (no full fine-tuning) on a single NVIDIA A100 80GB GPU.

Abstract

Large language models (LLMs) are expensive to serve because model parameters, attention computation, and KV caches impose substantial memory and latency costs. We present GRASPrune, a structured pruning framework applied after pretraining that jointly prunes FFN channels and KV head groups under a single global budget. Instead of learning importance scores without constraints and applying the budget only after training, GRASPrune learns lightweight gate scores with a projected straight-through estimator that enforces a hard mask satisfying the budget at every step while keeping the backbone weights frozen. After the mask is fixed, we calibrate scaling factors on the retained units to mitigate scale mismatch caused by pruning, and fold these factors into the pruned weights to obtain a smaller dense checkpoint with no extra parameters at inference. On LLaMA-2-7B, GRASPrune removes 50% of parameters and achieves 12.18 perplexity on WikiText-2 while maintaining competitive average zero-shot accuracy on five benchmarks, using four epochs on 512 unlabeled calibration sequences on a single NVIDIA A100 80GB GPU without any full model fine-tuning.