这篇论文用HERMES提供了一种可调节粒度的数据标签方法，比固定标签更灵活，能发现不同粒度下混合规则的效果差异。
HERMES提出一种数据驱动的层次化标签系统，通过Learned Semantic Transform与3阶段残差向量量化将每个文档标注为粗到细的代码，粒度可达约130k个单元。在1B参数、25B token的预训练实验中，粗粒度下与KMeans类方法性能持平，但层次结构揭示了固定粒度管道无法测试的交互：特定前缀长度下，Stage-2规则对比使16任务能力宏观平均提升+0.0253，而更细粒度时候选池缩小约5倍，该优势消失。HERMES将数据混合设计从固定标签集选择重构为可复用的数据驱动粒度层次导航。
HERMES: A Multi-Granularity Labeling Substrate for Pre-training Data Mixtures
Most data-mixing methods assume the corpus has already been partitioned into groups, and the choice of those groups determines what a mixer can express. Existing labels, including provenance, topic or format taxonomies, and flat embedding clusters, commit to one semantic axis at one granularity; changing the resolution rebuilds the labels. We argue the bottleneck is the label system, not the mixer, and provide a hierarchical one. HERMES is a data-derived labeling substrate: a Learned Semantic Transform followed by 3-stage residual vector quantization annotates each document once into a coarse-to-fine code whose prefix length controls granularity up to approximately 130k cells. At coarse granularity HERMES sits at a plateau with KMeans-family methods on standard clustering metrics, so the contribution is the substrate, not the clusterer. On 1B-parameter, 25B-token pre-training, the hierarchy exposes an interaction fixed-granularity pipelines cannot test: at one prefix length, a combined Stage-2 rule contrast, equal-subbucket coverage versus size-proportional within-bucket quality top-30%, lifts a 16-task capability macro-average by +0.0253; at the next finer level, the same rule loses its measurable edge as candidate pools contract approximately 5x. HERMES reframes data mixture design from choosing among fixed label sets to navigating a reusable, data-derived granularity hierarchy.