NVIDIA Project GR00T, a general-purpose foundation model for humanoid robots, designed to further its work driving breakthroughs in robotics and embodied AI.
Project GR00T is an initiative to develop general-purpose foundation models for humanoid robots that take multimodal instructions and past interactions as input and output robot actions. These advanced models are modular, with systems for both high-level reasoning and planning and low-level fast, accurate, and reactive motion.
Project GR00T is an active research project. If you are a humanoid robot maker, robot software/ hardware vendor, request early access.
As part of the initiative, the company also unveiled a new computer, Jetson Thor, for humanoid robots based on the NVIDIA Thor system-on-a-chip (SoC), as well as significant upgrades to the NVIDIA Isaac™ robotics platform, including generative AI foundation models and tools for simulation and AI workflow infrastructure.
“Building foundation models for general humanoid robots is one of the most exciting problems to solve in AI today,” said Jensen Huang, founder and CEO of NVIDIA. “The enabling technologies are coming together for leading roboticists around the world to take giant leaps towards artificial general robotics.”
Robots powered by GR00T, which stands for Generalist Robot 00 Technology, will be designed to understand natural language and emulate movements by observing human actions — quickly learning coordination, dexterity and other skills in order to navigate, adapt and interact with the real world. In his GTC keynote, Huang demonstrated several such robots completing a variety of tasks.
Built for Humanoid Robots
Jetson Thor was created as a new computing platform capable of performing complex tasks and interacting safely and naturally with people and machines. It has a modular architecture optimized for performance, power and size.
The SoC includes a next-generation GPU based on the NVIDIA Blackwell architecture with a transformer engine delivering 800 teraflops of 8-bit floating point AI performance to run multimodal generative AI models like GR00T. With an integrated functional safety processor, a high-performance CPU cluster and 100Gb of Ethernet bandwidth, it significantly simplifies design and integration efforts.
NVIDIA is building a comprehensive AI platform for leading humanoid robot companies such as 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, Figure AI, Fourier Intelligence, Sanctuary AI, Unitree Robotics and XPENG Robotics, among others.
“We are at an inflection point in history, with human-centric robots like Digit poised to change labor forever. Modern AI will accelerate development, paving the way for robots like Digit to help people in all aspects of daily life,” said Jonathan Hurst, cofounder and chief robot officer at Agility Robotics. “We’re excited to partner with NVIDIA to invest in the computing, simulation tools, machine learning environments and other necessary infrastructure to enable the dream of robots being a part of daily life.”
“Embodied AI will not only help address some of humanity’s biggest challenges, but also create innovations which are currently beyond our reach or imagination,” said Geordie Rose, cofounder and CEO of Sanctuary AI. “Technology this important shouldn’t be built in silos, which is why we prioritize long-term partners like NVIDIA.”
Major Updates to Isaac Platform
The Isaac tools that GR00T utilizes are capable of creating new foundation models for any robot embodiment in any environment. Among these tools are Isaac Lab for reinforcement learning, and OSMO, a compute orchestration service.
Embodied AI models require massive amounts of real and synthetic data. The new Isaac Lab is a GPU-accelerated, lightweight, performance-optimized application built on Isaac Sim specifically for running thousands of parallel simulations for robot learning.
To scale robot development workloads across heterogeneous compute, OSMO coordinates the data generation, model training and software/hardware-in-the-loop workflows across distributed environments.
NVIDIA also announced Isaac Manipulator and Isaac Perceptor — a collection of robotics pretrained models, libraries and reference hardware.
Isaac Manipulator offers state-of-the-art dexterity and modular AI capabilities for robotic arms, with a robust collection of foundation models and GPU-accelerated libraries. It provides up to an 80x speedup in path planning and zero-shot perception increases efficiency and throughput, enabling developers to automate a greater number of new robotic tasks. Among early ecosystem partners are Yaskawa, Universal Robots, a Teradyne company, PickNik Robotics, Solomon, READY Robotics and Franka Robotics.
Isaac Perceptor provides multi-camera, 3D surround-vision capabilities, which are increasingly being used in autonomous mobile robots adopted in manufacturing and fulfillment operations to improve efficiency and worker safety as well as reduce error rates and costs. Early adopters include ArcBest, BYD and KION Group as they aim to achieve new levels of autonomy in material handling operations and more.
How It Works
Click to Enlarge
Model Operation
Project GR00T enables humanoid embodiments to perform tasks from multimodal instructions and dynamically react to the world through self-observation.
Development and Training
The model uses multimodal learning from a variety of data sources—including instructions, demonstrations, video, and imitation learning for robots—to acquire new skills, accelerating the development of humanoid robots.
NVIDIA Isaac Lab is ideal for training Project GR00T at scale. You can also take advantage of the NVIDIA OSMO robot cloud computing orchestration service to coordinate workflows across NVIDIA DGX systems for training, NVIDIA OVX systems for simulation, and NVIDIA IGX systems for hardware-in-the-loop validation.
