PASK让结构化生成更高效,在保持准确性的同时大幅降低内存使用,提升处理速度。
PASK模型在Qwen3-4B上测试,于8个BFCL子类别平均超越最强压缩基线17.39个百分点。该模型针对JSON、SQL和函数调用等结构化生成任务,将解析器结构转化为层组特定的KV持久化决策。在端到端服务中,PASK实现2.2倍更高吞吐量,3.3倍更低TPOT,仅使用全KV 0.53倍的峰值GPU内存。
Parser States Already Know: Structure-Conditioned KV Persistence for Structured Generation
Structured generation underpins large language model (LLM) agents that produce JSON, SQL, and function calls, where a single wrong field can cause the downstream action to fail. Constrained decoding already tracks parser transitions to enforce formal validity, and these transitions expose how generated tokens participate in schema-critical decisions such as required fields, arguments, and structural boundaries under the active grammar. Existing KV compression largely leaves this task-relevant structural signal unused. We introduce PASK (Parser-Aware Structural KV Persistence), which turns parser-derived structure into layer-group-specific KV persistence decisions. PASK addresses the mismatch between model-side KV sensitivity and task-level structured risk by using task-error sensitivity to set minimum protection floors and attention-output distortion to allocate residual KV capacity. An offline calibration stage compiles these signals into a persistence policy, leaving only lightweight structure-conditioned lookup online. At a targe total KV budget of 0.33, PASK outperforms the strongest compressed baseline by 17.39 percentage points on average across eight BFCL non-live and Live subcategories on Qwen3-4B. In end-to-end serving, PASK achieves up to 2.2x higher throughput and 3.3x lower TPOT, while using 0.53x the peak GPU memory of Full KV.