六款大模型模仿个人写作风格评测:Gemini 3 Pro 最强但靠复制
How Well Can Large Language Models Impersonate Individual Authors? A Comparative Evaluation
测了六个主流模型谁能模仿你的文风,结果 Gemini 3 Pro 分数最高,其实是靠大段抄原文,得辨别真假模仿。
一项研究测试了 Gemini 3 Pro、Claude Opus、GPT 5.2、GPT 5.2 Pro、DeepSeek 3.2、Grok 4.1 Fast 六款模型模仿短评论文写作风格的能力,使用余弦距离指标和嵌入分布可视化评估。结果显示 Gemini 3 Pro 模仿得分最高,但其大量直接复制训练文本中的句子,比例远超人类作者水平。多数模型生成文本的嵌入位置与目标作者的嵌入簇明显分离。研究还发现训练文本风格一致性是模仿成败的最稳健预测因素,该结论对 LLM 介入的文本取证有参考价值。
How Well Can Large Language Models Impersonate Individual Authors? A Comparative Evaluation
This study investigates the authorship impersonation capacity of six widely used large language models (LLMs): Gemini 3 Pro, Claude Opus, GPT 5.2, GPT 5.2 Pro, DeepSeek 3.2, and Grok 4.1 Fast. The evaluation was conducted using short product review texts selected to ensure high within author consistency in individual writing style. Two types of in context prompt were designed: a simple prompt with minimal instruction and a complex prompt incorporating explicit stylometric guidance for mimicry. Authorship mimicry was evaluated using a cosine distance based distance to training metric, supplemented by visual analyses of embedding distributions. The results indicate that authorship impersonation remained challenging for most LLMs, with Gemini 3 Pro showing the strongest apparent mimicry performance. However, subsequent overlap analysis revealed that Gemini 3 Pro extensively copied and reused sentences and phrases from the training texts, at levels far exceeding those observed among human authors. Follow up factor analyses showed that training set consistency was the most robust predictor of mimicry success across models, whereas author distinctiveness did not generalise as a reliable predictor. Global embedding visualisations further revealed a consistent macro level pattern: for most models, generated texts occupied regions of the embedding space that were largely separated from the corresponding human training clusters. This indicates limited convergence towards target author style and suggests the presence of model specific lexical and phraseological preferences. The findings highlight the importance of distinguishing genuine stylistic imitation from direct textual reuse and have implications for the forensic analysis of textual evidence involving potential LLM enabled impersonation.
- Ethan Mollick11-03 06:26原文