Artificial Analysis launches Code2Video-AA video benchmark
The 300x spread in cost per task means the ranking alone is misleading: GPT-6.1 Sol delivers fourth place at 90% less cost than the leader, which changes which model is actually practical for batch video generation. Teams building automated marketing video pipelines can use the five-axis Elo breakdown to match models to their budget and quality needs, and the open-weights results give a concrete quality gap for self-hosting comparisons.
Artificial Analysis released Code2Video-AA, an independent implementation of HeyGen's Code2Video Bench, which tests how well models turn 114 product and brand marketing briefs into HTML compositions rendered with HyperFrames. A Gemini 4 Argon judge scores pairwise video comparisons across five axes—Engagement, Prompt Intent, Composition, Temporal Consistency and Craft—using over 35,000 verdicts to rank 27 models by Elo. Claude Opus 5.5 (max with fallback) leads by 93 points at US$3.08 per task, roughly 10x the cost of GPT-6.1 Sol (max, 1369), which trails by 167 points. The top open-weights models, DeepSeek V4.1 Flash and MiMo-V2.6 variants, score 1052-1053 at US$0.01-0.08 per task, at least 483 Elo points behind the leader.
Introducing Code2Video-AA, our independent implementation of HeyGen’s Code2Video Bench, which evaluates how well models create product and brand marketing videos in code
Code2Video-AA tests models across 114 design briefs, tasking them with turning each product or brand marketing brief into a @HyperFrames_ HTML composition, which we render to video. Model outputs are evaluated through pairwise comparisons with other model outputs across five axes: Engagement, Prompt Intent, Composition, Temporal Consistency, and Craft. We use the outcomes of these comparisons to calculate an Elo rating for each axis, which are then averaged to produce the headline Code2Video-AA Elo. Code2Video-AA was built in partnership with @HeyGen.
Claude Opus 5.5 (max with fallback, 1536) leads Code2Video-AA by 93 points, but costs $3.08 per task, ~10x more than GPT-6.1 Sol (max, 1369), which trails by 167 points.
Key findings:
➤ Claude Opus 5.5 leads on all five judge axes: Models are judged across 5 axes: Engagement, Prompt Intent, Composition, Temporal Consistency, and Craft. Claude Sonnet 5.5 (max with fallback, 1443) and GPT-6 Astra (max, 1427) are statistically tied for second in overall score
➤ Cost per task varies more than 300x, from $0.01 to $3.89: GPT-6.1 Sol (max), fourth, has a Cost per Task of $0.31, about 90% lower than Claude Opus 5.5 (max with fallback) at $3.08
➤ Open weights models trail by at least 483 points: DeepSeek V4.1 Flash (max, 1053), MiMo-V2.6-Flash (1052) and MiMo-V2.6-Pro (1052) are the top open weights models, at $0.01 to $0.08 per task
Code2Video-AA details:
➤ Models are tested on a single response without tools. We render it with HyperFrames, with no network access
➤ A judge (Gemini 4 Argon, high) compares pair of videos across five axes: engagement, prompt intent, composition, temporal consistency and craft, with the final score being calculated from a combination of each individual axis Elo
➤ Over 35,000 verdicts rank 27 models by Elo, with 95% confidence intervals
More results, example videos and how Code2Video-AA works follow in the thread below
