ModelsMulti-sourceSelected73°

T-Rex Tactile Methodology for Robotics

Why we picked it

T-Rex addresses a critical gap in robotics by making tactile sensing as important as vision, with open data available on HuggingFace for researchers worldwide.

T-Rex is a new open-source tactile methodology for robotics developed by NVIDIA and Berkeley.

Image · Jim Fan
Body

It includes a mixture-of-transformer architecture with two asynchronous clocks: a slow visuomotor expert and a fast tactile expert processing 4 touch ticks per vision tick.

The release includes a 50-hour tactile dataset with 5,500 episodes collected on 22-DOF tactile hardware.T-Rex extends EgoScale pretraining with tactile robot play data to improve contact-rich manipulation.

Original · Jim Fan

The sense of touch is the most criminally under-explored modality in robotics. Imagine doing sleight of hand wearing thick oven mitts. That's exactly how a robot feels today if it were alive. A magnetic piece snapping into place, a paper cup peeling out of a stack, a USB negotiating its way into the port - all invisible to the camera. Learning how to feel must be a full-stack co-designed effort. We are open-sourcing a principled methodology called "T-Rex": 1. Tactile as first-class citizen of the model. Our mixture-of-transformer runs two clocks asynchronously: a slow visuomotor expert plans the motion, and a fast tactile expert refines it in real time with high-frequency corrections at 4 "touch ticks" per vision tick. Forces change faster than frames arrive, so the architecture had to as well. 2. Open data. The largest tactile dataset ever released to our knowledge: a 50-hour (~5,500 episodes) high-quality, carefully synchronized robot play corpus, collected on SOTA tactile hand hardware with 22 degrees of freedom. Available today on HuggingFace! 3. Training recipe: T-Rex extends our prior work, EgoScale. Human egocentric videos for pretraining, a diverse dose of tactile robot play for mid-training. Our experiments show this bridges contact-free pretraining to contact-rich manipulation remarkably well. Pixels are cheap and everywhere, but they run out of steam at the moment of contact. Tactile will carry the last mile. The next scaling curve will be measured in hours of touch. T-Rex is a great collaboration between NVIDIA and Berkeley: 🧵

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Models" desk. Key entities and technical topics under continuous tracking include robot tactile sensing, T-Rex, NVIDIA, UC Berkeley.

Source attribution: curated from Jim Fan (@DrJimFan). Cross-source corroboration: 4 independent sources have covered this development (including NVIDIA AI, techcrunch, Clement Delangue), confirming industry-wide signal strength.