Lagged Coupling in Neural Networks
Lagged Coupling: Internal Representations Become Readable Before They Become Causal
The findings explain why probe accuracy doesn't guarantee steerability in large language models, affecting AI safety research and model interpretability efforts globally.
Researchers studied the Pythia suite (160M-12B models) across eight checkpoints and four task families.
A linear probe could read target variables from the residual stream as early as step 1,000 at every scale.Steering along the same reading direction remained null-equivalent in 43 of 48 model-checkpoint cells.
The lag between internal readability and causal efficacy does not shrink with model scale.
Lagged Coupling: Internal Representations Become Readable Before They Become Causal
Across the full Pythia suite (160M-12B, eight checkpoints, four task families), a linear probe can read a target variable from the residual stream as early as step 1,000 at every scale -- yet steering along that same reading direction remains null-equivalent in 43 of 48 model-checkpoint cells. Internal readability systematically outruns causal efficacy, and the lag does not shrink with scale. We call this structure lagged coupling and decompose it into three dissociable tracks: (i) internal readability, saturated (AUROC >= 0.990) from the first checkpoint everywhere; (ii) behavioral readability, which develops gradually and progressively later at larger scales (12B reaches 0.909 only at the final checkpoint); (iii) causal efficacy, almost always null-equivalent, occasionally counterproductive early, with one isolated positive pulse (12B, step 8,000, z = +2.49) our grid cannot resolve. The ordering is dominantly read-before-write (11/11 units, no inversion). Representation headroom along the probe direction grows up to 57x with training and scale while causal write-in stays below 0.11% of headroom -- the variable is increasingly written into the representation and increasingly ignored by the readout. Under a fully pre-registered protocol, both single-onset hypotheses resolve INDETERMINATE (scale slope +0.24, 95% CI [-0.60, +0.87]; time vote 3:3) -- a disciplined negative explained by the three-track decomposition. A pre-registered OLMo-2 replication preserves the direction at attenuated magnitude. Our results caution against inferring steerability from probe accuracy and establish a developmental bottleneck: representation formation reliably outpaces causal readout consolidation.
This report is categorized under AITOP's "Papers" desk. Key entities and technical topics under continuous tracking include Pythia, AI models.
Source attribution: curated from arXiv cs.AI (@Xining Xun).