想知道你用的模型到底吃了什么数据？LLMSurgeon 让数据审计变得可行，做模型安全、数据治理或开源复现的团队值得关注。
大语言模型的预训练数据混合比例决定了其能力与缺陷，但这一信息极少公开。LLMSurgeon 提出“数据混合手术”框架，仅通过模型生成的文本就能逆向估计其预训练语料的领域分布。该方法利用校准后的软混淆矩阵解决领域混淆问题，在开源模型上验证了高精度。这为审计基础模型的数据构成提供了无需访问训练数据的实用后验方案。
LLMSurgeon: Diagnosing Data Mixture of Large Language Models
The pretraining data mixture of Large Language Models (LLMs) constitutes their "digital DNA", shaping model behaviors, capabilities, and failure modes. Yet this composition is rarely disclosed, making post-hoc auditing of data combination or provenance difficult. In this work, we formalize $\textbf{Data Mixture Surgery (DMS)}$: given only generated text from a target LLM, estimate the domain-level distribution of its pretraining corpus under a predefined taxonomy. We propose $\textbf{LLMSurgeon}$, a strong framework that casts DMS as an inverse problem under the label-shift assumption. Rather than directly aggregating classifier outputs, LLMSurgeon estimates a calibrated $\textit{soft}$ confusion matrix and solves a constrained inverse problem to correct systematic domain confusion and recover the latent mixture prior. To evaluate, we introduce $\textbf{LLMScan}$, a recipe-verifiable evaluation suite built from open-source LLMs with transparent pretraining mixtures. Across LLMScan, LLMSurgeon recovers domain mixtures with high fidelity under fixed protocols. Our work presents a practical, post-hoc approach for auditing the digital DNA of foundation models without access to their training data.