New method improves pruning of large reasoning models
OBC-Prune: Outcome-Based Calibration for Large Reasoning Model Pruning
This research provides a more effective method for reducing the cost of large reasoning models, which is important for making them more practical and accessible.
A new method called OBC-Prune improves the effectiveness of pruning large reasoning models by focusing on the causal importance of reasoning steps rather than just statistical salience.
It constructs pairs of correct and incorrect model outputs to estimate how important each reasoning token is for correct answers.
Experiments on DeepSeek-R1, Qwen 1.5B, 7B, and 14B models at 40% and 50% sparsity show consistent improvements over state-of-the-art calibration baselines on MATH500, LiveCodeBench, and AIME 2025 benchmarks.
OBC-Prune: Outcome-Based Calibration for Large Reasoning Model Pruning
Large reasoning models (LRMs) generate long chain-of-thought traces before answering, creating significant inference overhead. Pruning can reduce this cost, but its effectiveness depends on the calibration data used to estimate parameter importance. Recent work calibrates on the model's own rollouts instead of generic dataset, but treats all reasoning tokens uniformly, regardless of whether they contribute to successful reasoning. As a result, pruning protects weights by statistical salience rather than by their contribution to correct reasoning, so weights behind erroneous computation survive as readily as those behind correct computation. These erroneous patterns then get carried into the pruned model, degrading reasoning quality, producing both lower accuracy and longer reasoning traces. We propose Outcome-Based Calibration for Large Reasoning Model Pruning (OBC-Prune) to close this gap. OBC first constructs difficulty-matched pairs of correct and incorrect rollouts from problems the model answers inconsistently. It then estimates the causal importance of each reasoning sentence through intervention-based analysis, quantifying how removing its influence affects subsequent predictions. These causal importance scores are converted into per-token weights that rescale the calibration activations used by one-shot pruning methods (SparseGPT, Wanda, ALPS), without modifying the underlying pruning algorithms. Experiments on DeepSeek-R1-Distill-Qwen 1.5B, 7B, and 14B models at 40\% and 50\% sparsity demonstrate consistent improvements over state-of-the-art calibration baselines across most model sizes and sparsity levels on MATH500, LiveCodeBench, and AIME 2025. These results indicate that preserving causally important reasoning circuits is a substantially more effective pruning objective than uniformly preserving observed activations.
This report is categorized under AITOP's "Papers" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include OBC-Prune, large reasoning models, model pruning, DeepSeek-R1.
Primary source verification: published directly by arXiv: DeepSeek with first-party authority.