面向电磁信号的结构化预训练：用 RoPE-fs 时间对齐替代波形重建

论文指出，波形由信号生成、传播环境和采集过程共同决定，传统重建式预训练存在外观失配与时间失配两类问题，导致模型学到观测捷径而非稳定信号结构，作者将这一失败模式称为 observation-biased pretraining。

方法上引入 RoPE-fs，把 token 位置映射到采样率归一化的物理时间坐标以消除时间失配；并用离散结构单元的掩码预测替代波形重建，结构单元分为原子结构单元和涌现结构单元，可捕捉超越固定 patch 边界的变长组合模式。

在多个 EM 基准上，该方法在全监督和少样本适配两种设定下均优于现有电磁信号预训练基线，表明学到的表示在波形外观和时间尺度漂移时仍保持结构一致性。

Observations Drift, Structures Remain: Structural Pretraining with Time Alignment for Electromagnetic Signals

Electromagnetic signal modeling is increasingly moving toward self-supervised pretraining on heterogeneous I/Q observations. However, an observed waveform is jointly shaped by signal generation, propagation environments, and acquisition processes, which mix reusable structural regularities with observation-specific variations. Existing methods usually pursue unified pretraining with reconstruction objectives on mixed observations. This heterogeneity creates two key mismatches for reconstruction pretraining: (1) appearance mismatch, induced by propagation-dependent waveform variations; and (2) temporal mismatch, induced by acquisition-dependent sampling coordinates. These mismatches make observation-specific appearances and sampling coordinates easier to reconstruct than stable signal structures, inducing observational shortcuts. We identify this failure mode as observation-biased pretraining. To address this, we propose Structural Pretraining with Time Alignment for heterogeneous EM signals. It mitigates temporal mismatch with RoPE-fs, which maps token positions to sampling-rate-normalized physical-time positions, and reduces reliance on appearance shortcuts by replacing waveform reconstruction with masked prediction of discrete structural units. These units are organized into atomic structural units and emergent structural units that capture variable-length compositional patterns beyond fixed patch boundaries. Across diverse EM benchmarks, our method improves both full-supervision and few-shot adaptation over existing EM pretraining baselines, showing that the learned representations better preserve structural consistency when observations drift in waveform appearance and temporal scale.

This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include RoPE-fs, 自监督预训练, 电磁信号, 掩码预测.

Source attribution: curated from arXiv cs.LG (@Wenjin Gui, Junyu Shen, Haibo Xu, Yuchuang Sun, Luqing Luo, Wupeng Xie, Qian Huang, Fengxiang Wang, Yangang Sun, Maosong Sun).