AutoRAGBench Tests Coding Agents Against Human RAG Designs
The most actionable finding is that agents starting from a human-designed baseline gain 16.6 points on average, so the practical recipe is human scaffolding plus agent refinement rather than full automation. Teams evaluating Codex CLI or choosing an agent stack for RAG engineering can use AutoRAGBench's nine-task setup to compare the nine tested configurations. Engineers building retrieval pipelines should treat it as a reference point for allocating agent compute budgets within a fixed time and single-GPU constraint.