EHR 模型时间上下文研究：回溯 30 年病史预测结直肠癌
How Deep is the Signal? Time-Context up to 30 Years for Colorectal Cancer Prediction in a Primary-Care EHR Model
用 30 年病史训练 EHR 模型预测结直肠癌，发现回看 18 年效果最好，还顺带指出了 RoPE 处理长时间的问题。
一项新研究提出 time-context（预测点之前的日历回溯窗口）概念，评估 EHR 基础模型需要回看多久的病史。团队在伦敦西北部 3.6 万名初级保健患者的 14 亿条编码事件上预训练了 34.6M 参数的 decoder-only transformer，用 1,008 例结直肠癌病例和 9,582 名对照评估 2 到 30 年的 15 个嵌套回溯窗口。AUROC 从 2 年窗口的 0.551 升至 18 年窗口的 0.590，约 80% 的增益在 10 年内达成，XGBoost 基线峰值也达到 0.589。模拟显示 rotary position embeddings（RoPE）在默认基频下对超过约 20 年的时间会产生混叠，需重新调整 θ。
How Deep is the Signal? Time-Context up to 30 Years for Colorectal Cancer Prediction in a Primary-Care EHR Model
EHR foundation models read medical events within a fixed context window, and long context has so far been measured only in tokens. How far back in time a model must look to capture the earliest predictive signal is unknown. We introduce time-context, the calendar lookback window before the prediction point, and provide, to our knowledge, the first systematic evaluation of decade-scale time-context for an EHR foundation model. We pretrained a decoder-only transformer (34.6M parameters) on 1.4B coded events from 3.6M primary-care patients in North West London and fine-tuned it to predict incident colorectal cancer under a 12-month pre-diagnostic buffer. A frozen cohort of 1,008 cases and 9,582 matched controls, all registered for at least 30 years, was evaluated at 15 nested lookback windows from 2 to 30 years, so that every truncation is compared within the same patients. Discrimination rose from an AUROC of 0.551 at 2 years to 0.590 at 18 years, then plateaued, with about 80% of the gain reached within 10 years; an XGBoost baseline converged to the same level (peak AUROC 0.589). A simulation further suggests that rotary position embeddings (RoPE) over elapsed time alias beyond about 20 years at the default base frequency, motivating a retuned $θ$. Time-context is a measurable design axis for clinical prediction models, distinct from token-context, and should be reported.