编排开源小语言模型在恶意软件分析中超越单个大语言模型

Small, Free, and Effective: Orchestrating Open-Weight Small Language Models to Outperform Single LLM for Malware Analysis

精选理由

别总盯着大模型花钱了,用Qwen3-4B和Foundation-Sec-8B组队就能干翻GPT,这篇论文给了具体方法和数据。

AI 摘要

论文测试了11个开源小语言模型(SLM)、3个网络安全预训练模型和6个前沿大语言模型(LLM)在Meta的CyberSecEval恶意软件分析基准上的表现。设计了四种编排架构:多智能体管道、对抗性辩论、层级咨询和混合系统。混合系统(Qwen3-4B与Foundation-Sec-8B)达到35.30%准确率,超过最强网络安全基线22.54%和最强无基线前沿模型34.77%。有证据的Gemini达到38.22%,表明证据导向的编排可显著提升协作SLM的性能。

原文 · arXiv cs.AI

Small, Free, and Effective: Orchestrating Open-Weight Small Language Models to Outperform Single LLM for Malware Analysis

Malware analysis demands rapid interpretation of complex detonation reports spanning filesystem, network, and process behaviours. While large language models (LLMs) demonstrate impressive capabilities for technical artifact interpretation, the opacity and escalating API costs of closed-weight frontier models motivate exploration of open-weight alternatives. However, many open-weight models are large, demanding significant compute resources and incurring non-trivial hosting costs that place them beyond reach for resource-constrained deployments. This paper investigates whether orchestrated ensembles of small language models (SLMs) can match or exceed single LLM performance on structured questions about malware detonation reports. We established baselines by testing eleven open-weight SLMs, three cyber security pre-trained models, and six frontier LLMs on Meta's CyberSecEval Malware Analysis benchmark. We then designed and evaluated four orchestration architectures: (i) a multi-agent pipeline that decomposes analysis into structured evidence-collection and reasoning stages, (ii) an adversarial debate framework in which two agents iteratively critique each other's reasoning, (iii) a hierarchical consultation system that pairs a general-purpose SLM with a cyber-specialised expert model, and (iv) a hybrid architecture that combines evidence-grounded pipelines with adversarial debate reasoning. The hybrid system (Qwen3-4B with Foundation-Sec-8B) achieved 35.30% overall accuracy, exceeding the strongest cyber-specialised baseline (22.54%) and the strongest ungrounded frontier baseline (34.77%); when given the same evidence pipeline, grounded Gemini remained the strongest configuration at 38.22%. These findings show that evidence-grounded orchestration can substantially improve the performance of collaborative SLMs for supporting interpretation of malware detonation reports.