这个基准让你看到哪些大模型最容易被带偏去搞信息战,测了17个模型,差距能到85个百分点,挺颠覆直觉的。
InfoOps Bench是一个持续更新的AI基准,用于衡量前沿语言模型是否会被国家支持的信息操作所利用。该基准基于追踪俄罗斯、中国和伊朗国家背景信息资产的监测管道,覆盖2,100多个信息操作案例。研究测试了8家提供商的17个模型,发现多数模型都能被诱导参与信息操作,完整性评分从8.8%到94.5%不等。模型选择会改变生成内容性质,部分模型会编造细节,事实核查率在2.9%至72.9%之间变化。除Z.ai的GLM 5.2外,中国开发的模型对中国批评性请求的遵从度大幅下降,最高降幅达70个百分点。
InfoOps Bench: A live information operations safety benchmark
In this paper we present an active, constantly updated AI benchmark which measures the integrity of frontier language models against being co-opted for state-backed information operations. We draw on over 2,100 information operations from a live monitoring pipeline which tracks Russian, Chinese and Iranian state-backed information assets. Alongside this paper, we release a companion website that tracks the most prominent claims spread by state-backed media outlets, updated weekly, available from: pattrn.ai/research/infoopsbench. The dynamic nature of the benchmark makes it resistant to saturation. In the benchmark, we test 17 models from 8 providers across four prompt framings. We find that most models can be co-opted for information operations. Integrity scores, defined as the percentage of refused requests, range from 8.8% to 94.5%, an 85.7-percentage-point spread not explained by model size. Model choice also changes the character of the resulting operation. Some models fabricate details and produce output more harmful than the source material, others defuse claims even while complying, and fact-checking rates vary from 2.9% to 72.9%. Integrity against information operations is at least partly related to refusal to produce content even for benign claims, illustrating the challenge of balancing model usability with safety. With one exception (Z.ai's GLM 5.2), the Chinese-developed models sharply cut compliance on factually grounded but China-critical claims, dropping 48-70 percentage points relative to matched benign claims.