可解释智能体系统ConScamDetect实现对话诈骗高精度检测

An Explainable Agentic System for Detection of Conversational Scams with Summary-Based Memory

精选理由

这篇论文发了一个能识别跨周对话诈骗的系统,在公开数据集上准确率接近98%,还有用户研究证明它真的管用。

AI 摘要

该系统将单消息钓鱼检测扩展到对话级诈骗,并发布ConScamBench-278基准(含8类诈骗)。在单消息检测器上实现100%钓鱼召回,在LoveFraud02语料库中识别全部83个对话诈骗,在ConScamBench-278上达97.8%准确率(95% CI [95.4, 99.0])。两项用户研究(N=100和N=45)表明用户对基于AI的诈骗检测的信任和自信显著提升(p<0.001,Wilcoxon符号秩检验)。系统可用性评分74.7(95% CI [72.5, 76.9]),高于可用性基准。

原文 · arXiv cs.AI

An Explainable Agentic System for Detection of Conversational Scams with Summary-Based Memory

Following the rapid progress of generative Artificial Intelligence, there is a growing threat posed by conversational scams. These scams often span over multiple weeks or months, gradually build trust and request for money or sensitive information. Existing scam-detection systems mainly focus on isolated messages, which renders them inadequate against this evolving threat. This paper extends single-message phishing detection and presents an explainable agentic system for detecting sophisticated conversational scams. It also introduces ConScamBench-278, an initial public multi-category benchmark for conversational scam detection spanning eight scam types, released to support reproducible evaluation and future expansion. On isolated messages the single-message detector attains 100% phishing recall, while the conversation-level detector identifies all conversational scams in the public LoveFraud02 corpus (83/83) and reaches 97.8% accuracy (95% CI [95.4, 99.0]) on ConScamBench-278. Two user studies (N = 100 and N = 45) further motivate the system: participants report frequently experiencing uncertainty when judging suspicious conversations. In an uncontrolled pre/post comparison, users self-reported trust, self-confidence, and perceived need for AI-based scam detection all increased (p < 0.001, Wilcoxon signed-rank). The system also receives a System Usability Scale score of 74.7 (95% CI [72.5, 76.9]), above the established usability benchmark.