LOCUS method cuts token usage in language models
LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation
This research provides a method to make large language models more efficient in token usage, which can reduce costs for developers and users.
A new method called LOCUS reduces the number of output tokens in large language models by up to 39.84% for a 3B model and 14.87-17.58% for another 3B model, without changing the model's utility. It does this by applying low-rank post-training updates to the model's parameters.
LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation
Large language model serving costs scale directly with output sequence length, yet standard preference alignment often inflates response verbosity without improving utility. We study whether the parameterization of post-training updates affects generation length: low-rank subspaces alter sequence length without modifying the alignment loss. We present LOCUS, a method that selects a task-aware low-rank adaptation subspace to minimize output-token cost subject to a utility constraint. Within this subspace, post-training retains the native preference objective with a frozen backbone. Across Anthropic HH-RLHF dialogue preferences, we evaluate two $\sim$3B decoder backbones, Pythia-2.8B and Qwen2.5-3B, against protocol-matched full-parameter DPO and DrDPO branches and the released SamPO checkpoint. LOCUS reduces continuation length by up to 39.84\% on Pythia-2.8B and by 14.87--17.58\% on Qwen2.5-3B while updating only 0.24--0.28\% of model parameters, with no material change in the internal preference diagnostic.
