Wayne Chi Explains Interaction-as-a-Judge for Agent Evaluation
The core takeaway is that evaluation signal can be harvested from real user interactions instead of hand-written test suites, shifting agent benchmarking toward live platform data. Researchers and teams building computer-use agents can apply this pattern to score traces with LLM judges or reward models trained on user complaints. Code Arena and Agent Arena already supply the battle and complaint data, giving practitioners a concrete starting point for evaluation pipelines.
