别用前沿 LLM 做视觉任务:$6 训练专用检测器更划算
HF 的 Merven 把自己选视觉模型的经验做成了一套工具:检测、OCR 打包成 Agent 可调用的工具,还有从标注到训练到评估的完整训练管线,$6 就能训出一个检测器。做 Agent 视觉相关项目的开发者可以直接抄这套流程。
Merven (HF 工程师) 在 AI Engineer WF 演讲中分享:直接调用前沿 LLM 做物体检测成本高、效果差,训练一个专用检测器只需 $6,运行成本也极低。
她把 Keypoint 检测、OCR 等首选视觉模型封装成 Agent 可调用的工具,并做了一个端到端目标检测训练管线:从数据标注、VLM 当裁判评估,到训练和 eval 全流程打通。她还为 Agent 补上了决策模型支持,并更新了 LFM3 和 Qwen3.
6/8 等 VLM。
it's a skill issue to use frontier LLMs to detect objects or do vision
instead, if you use LLMs as vision interns you can train a detector for $6 that runs at little cost. but your agent is a bad vision intern, it only knows old models that are proprietary and makes rookie mistakes
so I built something for it, I basically distilled myself for you to use🤝 I put my favorite models from keypoint detection to OCR as a tool for your agent to call and I also made an e2e object detection trainer (from labelling to VLM as a judge to training and eval)
if you're interested in it and the results, my talk at @aiDotEngineer WF is finally out, you can watch it here https://t.co/2cdSRc3bOa many thanks @swyx
on the next post you can find the github repository, I also shipped decision model support and updated VLMs (LFM3, Qwen3.6/8)
This report is categorized under AITOP's "How-to" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include object detection, vision models, agents, OCR.
Source attribution: curated from Merve Noyan (@mervenoyann).