swyx argues independent AA-* rebuilds are needed to catch benchmaxxing
The core implication is that official benchmark scores are increasingly unreliable signals for model selection, and community-run alternative evals are becoming the de facto quality check. Builders choosing models for production should cross-check lab claims against independent altbenches before committing to an API or fine-tuning pipeline. Following Artificial Analysis's rebuilds offers a practical sniff test without waiting for lab corrections.
