LLM与规则注入在OpenStack中的故障测试对比
Beyond Fixed Fault Models: Comparing LLM-Based and Rule-Based Fault Injection in OpenStack
阿里和DeepSeek的代码模型在云系统故障测试中表现如何？和传统规则注入有什么不同？
研究比较了Qwen2.5-Coder和DeepSeek-Coder两个代码大模型与ProFIPy规则注入器在OpenStack Nova和Cinder服务中的故障注入效果。在共享注入目标上，激活率和可观测故障率相近，但LLM生成的故障在Nova上导致更多灾难性结果，而ProFIPy产生更多静默和多组件影响。LLM输出在故障表现方式上存在差异，但在故障传播范围上表现出更高一致性。
Beyond Fixed Fault Models: Comparing LLM-Based and Rule-Based Fault Injection in OpenStack
Software Fault Injection (SFI) supports testing of cloud systems by introducing software defects and observing their manifestation. Rule-based injectors such as ProFIPy provide controlled and reproducible source-level mutations but require fault patterns to be encoded manually. Large Language Models (LLMs) offer a data-driven alternative by generating context-dependent software faults. We compare two code LLMs, Qwen2.5-Coder and DeepSeek-Coder, with ProFIPy in OpenStack's Nova and Cinder services. On shared injection targets, activation and observable-failure rates are comparable, but operational profiles differ: LLM-generated faults produce more Catastrophic outcomes on Nova, whereas ProFIPy produces more Silent and Multi-component effects. The sampled LLM outputs also differ in how they manifest failure, while showing greater agreement in their propagation scope. These findings show that LLM-based fault injection extends the behavioral coverage of fixed fault models without establishing general superiority, and that practical adoption still requires controlled generation, runtime validation, system-level oracles, and reproducible experimental provenance.
