AITOPAI热报
IndustryMulti-source

同一模型不同评测框架结果不同：Hugging Face 工程师详解 harness engineering

DeepLearning.AI@DeepLearningAI
Post· 11 sources
View the post
Why we picked it

同一个模型换个评测框架跑分就不一样，Hugging Face 的工程师要在 AI Dev NYC 现场讲怎么搭靠谱的 harness，搞评测的可以去听。

Hugging Face 机器学习工程师 Joel Niklaus 将在 AI Dev NYC 上开设两场 workshop，主题为 harness engineering 和基准测试。他指出同一模型在不同评测框架下会产生不同结果。活动于 11 月 30 日至 12 月 1 日在纽约 Chelsea Piers 举行，演讲者来自 OpenAI、Neo4j、Oracle、Sonar 等。

Original · DeepLearning.AI

Same model, different harness, different results. Joel Niklaus, Machine Learning Engineer at @huggingface , leads two workshops at AI Dev NYC on harness engineering and benchmarks. Speakers from @OpenAI , @neo4j , @Oracle , @Sonar and more round out two days of technical sessions. 📅Nov 30–Dec 1 · Chelsea Piers, NYC 🎟Get your tickets ai-dev.deeplearning.ai/speakers/joel-… i0 #AIDev e #DeepLearningAI A #AIEngineeringSkills ls 💬 0 🔄 0 ❤️ 0 👀 618 ⚡

#Hugging Face#AI Dev NYC#基准测试#harness engineering

Keep reading

Recent stories with the same tag or section
ModelsBuildersMulti-sourceSelected03:23
NVIDIA Nemotron 3.5 Transcribe Tops Hugging Face Trending

An open-weight 30B MoE with only ~4B active parameters makes speech-to-text with speaker diarisation and translation runnable on modest hardware, undercutting the usual cost of per-language ASR pipelines. Builders working on voice agents or meeting transcription can self-host one checkpoint instead of chaining multiple services. With 40-language coverage, it is a practical candidate for multilingual Southeast Asian speech workloads.

#NVIDIA#Nemotron#speech recognition#open-source models
Event
官方账号NVIDIA AI@NVIDIAAI11 sourcesSource
ProductsBuildersSelected00:28
Hugging Face and partners launch RoboColosseum open robot eval initiative

Real-world robot evaluation has lacked shared standards, and publishing failed rollouts alongside successes gives the field a reproducible benchmark rather than cherry-picked demos. Labs and hobbyists can apply for free eval hours to test policies on heterogeneous hardware. eaic_sg's involvement ties the initiative to the Singapore robotics ecosystem.

#Hugging Face#LeRobot#Unitree#robotics
官方账号Hugging Face@huggingfaceSource