NVIDIA open-sources Nemotron 3 Diarization for voice agents
Speaker diarization has been a missing piece in voice agent stacks, where most demos hide the inability to tell callers apart. Because the model works on one-second chunks, developers can plug it into streaming pipelines without waiting for turn ends. The commercial-friendly license and day-zero Transformers support make it a practical alternative to closed speech APIs for anyone building voice robots or assistants.
NVIDIA has open-sourced Nemotron 3 Diarization, a model that tracks speakers in live conversations and ships under a commercial-friendly license.
Hugging Face's Andi Marafioti reports strong quality on one-second speech chunks, making it usable in real-time voice agents.
The model has day-zero integration with the Transformers library.Marafioti tested it with Pollen Robotics' Reachy Mini and speech-to-speech running on a DGX Spark, where the robot detected a new voice, asked for a name, and remembered it.
Our new Nemotron 3 Diarization model + @pollenrobotics Reachi Mini + DGX Spark Great work Andi @huggingface 🙌 Andi Marafioti @andimarafioti Voice agents still don’t understand who’s speaking to them. That’s a huge gap compared with humans, hidden by all the “phone-call” demos. But that changes today! NVIDIA is open-sourcing Nemotron 3 Diarization: a model that can reliably track speakers in live conversations, under a commercial-friendly license! In my tests, the quality is really good with one-second speech chunks. So we can use it for voice agents! I tested it with Reachy Mini and speech-to-speech running on a DGX Spark. It’s super fun to see the robot notice a new voice, ask for a name, and remember it. The model has day-zero integration with Transformers! Kudos to the NVIDIA team for shipping useful tools for the whole community!
This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include Nemotron, NVIDIA, Hugging Face, speech.
Source attribution: curated from NVIDIA AI (@NVIDIAAI). Cross-source corroboration: 11 independent sources have covered this development (including AIGCLINK, Thomas Wolf, Hugging Face: Blog), confirming industry-wide signal strength.
