Papers

研究发现协议过滤可提升 LLM 分析网络流量的准确率

Do LLMs Understand Network Traces? A Comparative Study of Reasoning over Raw and Filtered Traffic Data

Why we picked it

搞网络运维的同学可以看看,这篇用 CICIDS2017 数据测了多个 LLM 解析流量的能力,结论很实用:先把流量按协议过滤再喂给模型,又省 token 又更准。

一篇发表于 arXiv 的研究评估了大语言模型在解析真实网络数据包方面的能力,使用的数据集是 CICIDS2017 中的良性流量样本。研究提出了一套可复现的问答式评估方法,让多个模型分别处理原始文本形式的流量转储和经过协议过滤后的表示,再与人工生成的标准答案比对。

Body

技术流程上,模型回答通过 LLM-as-a-judge 自动化管线打分,该管线经过人工评估验证。结果显示,协议过滤在显著减少提示词开销的同时降低了错误回答数量,从而提高答题准确率。

对运维和安全团队而言,这项工作意味着把 LLM 引入流量分析时,先做协议感知的过滤是一个低成本、有效的优化手段,而非直接投喂原始 pcap 数据。

Original · arXiv cs.AI

Do LLMs Understand Network Traces? A Comparative Study of Reasoning over Raw and Filtered Traffic Data

The effective delivery and monitoring of ICT services rely heavily on network traffic analysis. However, interpreting complex packet captures remains a challenging task that requires substantial technical expertise. Large language models (LLMs) offer promising opportunities to automate network traffic in- vestigation, yet their reliability in packet-level field extraction and flow-level sequence reconstruction remains an open research question. This study proposes a reproducible, question-answering- based evaluation methodology to assess the capabilities of LLMs in network trace analysis. Using real-world benign traffic from the CICIDS2017 dataset, we evaluate multiple models against a human-generated ground truth, comparing their performance on raw textual traffic dumps and protocol-filtered representations. Model responses are assessed through an automated LLM- as-a-judge pipeline, validated against human evaluations. The results demonstrate that protocol filtering significantly reduces prompt overhead while improving answer accuracy by reducing incorrect responses. These findings highlight the importance of protocol-aware traffic filtering as an effective optimization for improving the efficiency and reliability of LLM-assisted network management.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on Europe's AI governance and technology ecosystem. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include LLM, CICIDS2017, LLM-as-a-judge.

Source attribution: curated from arXiv cs.AI (@Irene Gosálvez-White, Roberto Magán-Carrión, Carlota de la Vega-Soriano, Pedro García-Teodoro).