这篇论文解释了为什么RoPE在训练中某些频率更常用，还教你用频率缩放来扛更长上下文，很实用。
论文提出频率匹配原理，认为RoPE最优频率与数据依赖宽度W成反比（1/W），并通过合成数据和文本数据验证。发现语言模型中低频带源于自然语言的多尺度依赖结构。将频率选择与位置插值长度泛化关联：低频缩放扩展有效场但降低分辨率。实验表明自然语言具有近似自相似性，支持测试时频率缩放实现长上下文泛化。
How Data Shapes RoPE Frequency Usage: From Positional Scale Matching to Length Generalization
Rotary Position Embeddings (RoPE) provide transformers with a fixed grid of positional frequencies, yet trained models use these frequencies highly non-uniformly. We study what determines this frequency usage and propose a data-centered explanation: RoPE frequencies are selected to match the relative-distance structure of the training data. Viewing each frequency as a positional lens, we formalize a field-resolution tradeoff and show that, for a data-induced dependency profile of width $W$, the optimal frequency scales as $1/W$. This frequency-matching principle explains controlled observations on synthetic and text-based data, and suggests that the mid-low frequency bands observed in language models arise from the multi-scale dependency structure of natural language. We further connect frequency selection to position-interpolation-based length generalization: scaling frequencies down expands the effective field while reducing resolution. This helps when longer-context dependencies are approximate dilations of those seen during training, but can fail when relevant dependencies do not scale with context length. Empirically, we show that natural language exhibits approximate self-similarity across positional scales, explaining why test-time frequency scaling can support long-context generalization. Overall, our results identify a data-driven mechanism behind emergent RoPE frequency usage and show that long-context generalization depends on two forms of scale matching: between learned frequencies and training-time dependencies, and between frequency scaling and how those dependencies extend to longer contexts.