Gemini 3.8 Flash Benchmark Results
Gemini 3.8 Flash shows regression in document parsing compared to 3.7, suggesting users should stick with the previous version for document analysis tasks.
Gemini 3.8 Flash was tested on ParseBench, showing slight improvements in tables and content faithfulness compared to 3.7 Flash. The model performed worse on charts and semantic formatting, with overall scores similar to 3.7 and 3.6 Flash versions. The benchmark needs evolution toward more challenging documents as frontier models struggle with real-world document parsing.
Gemini 3.8 Flash might be maxxed on other benchmarks, but not quite on document parsing. We benchmarked it on ParseBench. It does slightly better on tables + content faithfulness compared to Gemini 3.7 Flash, but a little worse on charts and semantic formatting. The overall scores are similar to those of 3.7 and 3.6 Flash. We definitely need to evolve the benchmark towards higher difficulty documents. In the meantime though, parsing real-world docs remains a real challenge for frontier models! Bindu Reddy @bindureddy 🚨 Gemini 3.8 Flash Is Bench Maxxed And Is A Regression on 3.7 Flash 🫢 - 3.8 is worse on our benchmark than 3.7 - This version appears to be overfit to public benchmarks It does worse on our hidden questions and regresses in data analysis. Edit - That said, Gemini Flash 3.7 is an AWESOME model. Google should stop with more Flash releases and drop Gemini 4.0 🔗 View Quoted Tweet 💬 9 🔄 0 ❤️ 11 👀 3083 📊 9 ⚡
