SWIFT：用小世界连通性打破稀疏微调的信息孤岛
Breaking Information Islands in Sparse Tuning via Small-World Connectivity
稀疏微调总有些特征维度互相够不着，SWIFT 用小世界网络思路加一层置换把路打通，效果能追上全参数微调，代码也放出来了。
论文提出 Small-World Induced Fine-Tuning（SWIFT），一个参数高效的稀疏微调框架。它把块对角稀疏更新用于局部特征混合，再用固定的确定性散射置换把混合输出跨特征组路由，且不引入额外可训练参数。作者从理论上证明，某些随机稀疏更新和块对角更新会形成信息孤岛，而充分的散射置换可保证正的块级扩张。在常识推理、自然语言生成和图像分类三类任务上，SWIFT 平均优于所评测的 PEFT 基线，并能匹敌或超过全参数微调。代码已在 GitHub 开源。
Breaking Information Islands in Sparse Tuning via Small-World Connectivity
Sparse tuning is widely used to adapt large language models due to its parameter efficiency. However, its effectiveness depends not only on the number of trainable parameters, but also on the feature-interaction topology of the sparse update's structural support. Limited or uneven connectivity in this support may isolate subsets of feature dimensions, forming information islands that restrict direct task-specific interactions through the trainable update. To address this issue, we propose Small-World Induced Fine-Tuning (SWIFT), a sparse tuning framework inspired by small-world connectivity. SWIFT combines block-diagonal sparse updates for local feature mixing with a fixed deterministic scattering permutation that routes the mixed outputs across feature groups without additional trainable parameters. We further show theoretically that certain random sparse-update models and block-diagonal updates can exhibit information islands, whereas a sufficiently scattering permutation guarantees positive block-level expansion when the trainable block-diagonal factor consists of equal-sized, structurally dense square blocks. Extensive experiments on commonsense reasoning, natural language generation, and image classification demonstrate that SWIFT outperforms the evaluated parameter-efficient fine-tuning (PEFT) baselines on average and can match or surpass full fine-tuning. Implementation is available at https://github.com/xianchaoguan/SWIFT