预测市场评论立场检测:反事实增强与市场上下文解决数据不平衡

Stance Detection in Prediction Markets: Addressing Imbalanced Trader Commentary via Counterfactual Augmentation and Market Context

精选理由

做金融舆情或预测市场分析的团队会感兴趣——这篇论文用反事实增强解决了极端不平衡数据下的立场检测难题,50%合成数据是最佳实践,值得在类似场景中尝试。

AI 摘要

该研究首次将立场检测应用于预测市场(如Polymarket)的评论,这些评论包含价格无法捕捉的方向性信号。由于评论极端简短、存在特定行话且类别严重不平衡(仅8.7%的评论反对市场结果),研究通过微调RoBERTa-base模型,探索了四种输入配置和三种数据增强条件。结果表明,市场上下文是最关键的因素,能将反对类别的召回率从0.10提升至0.45;反事实增强在弱配置下有效(F1从0.10升至0.24),但在强配置下会降低性能;50%的增强比例是最优剂量。注意力可解释性分析为所有发现提供了机制支持。

原文 · arXiv: Anthropic

Stance Detection in Prediction Markets: Addressing Imbalanced Trader Commentary via Counterfactual Augmentation and Market Context

Prediction markets such as Polymarket aggregate crowd beliefs into real-time probability estimates, and the comments traders post beneath each market contain rich directional stance signals that prices alone cannot capture. This work introduces the first stance detection study applied to prediction market commentary, a domain characterized by extreme brevity, trader- specific vernacular, and severe class imbalance (only 8.7% of comments oppose the market outcome). RoBERTa-base is fine-tuned across a 4 x 3 ablation: four input configurations ({2- class, 3-class} x {with/without market context}) and three augmentation conditions (baseline, 50% synthetic, 100% synthetic). Synthetic minority-class samples are generated via LLM-driven Pro -> Anti counterfactual flips using the Anthropic API. Results show that (1) market context is the single most impactful factor, raising 3-class Anti recall from 0.10 to 0.45; (2) counterfactual augmentation is conditionally effective, improving Anti F1 in weak configurations (0.10 -> 0.24) while degrading strong ones (2-class-ctx macro F1: 0.68 -> 0.50 at full dose); and (3) 50% augmentation is the optimal dose, with 100% consistently hurting performance. Attention-based interpretability analysis provides mechanistic support for all three findings.