Nvidia's AI Lab: Powering the Future of Robotics

08/12/2025 Artificial Intelligence

It's amazing to see how far some companies can come. Back in 2009, when Bill Dally joined Nvidia's research lab, it was a tiny place, focusing on how to make computer graphics look super realistic with something called ray tracing. Fast forward to today, and that little lab has grown into a powerhouse with over 400 people, playing a huge role in Nvidia's transformation from a gaming GPU startup to a $4 trillion giant, mostly because of the AI boom. I mean, who would have thought?

Now, Nvidia's setting its sights on the next big thing: robotics and AI. It sounds like something straight out of a sci-fi movie, but it's real. The company recently showed off a bunch of new AI models, libraries, and other tools aimed at helping robotics developers. They're basically building the infrastructure for the future of robots.

Dally, who's now Nvidia's chief scientist, actually started out as a consultant for them back in 2003, while he was still teaching at Stanford. When he was ready to leave his position at Stanford, Nvidia jumped at the chance to bring him on board full-time. He said that Nvidia really made a compelling case for him to join their research lab, and he eventually agreed. Sometimes, the best opportunities come when you least expect them. It’s all about timing, I guess.

Expanding Horizons

When Dally took charge of the lab, his main goal was to grow it and diversify their work. So, researchers started exploring areas beyond just ray tracing, like circuit design and VLSI (very large-scale integration), which is basically putting millions of transistors on a single chip. The lab hasn't stopped growing since then.

For a while, the focus was on making better GPUs for AI. Nvidia was ahead of the curve, starting to experiment with AI GPUs way back in 2010 – more than a decade before the current AI craze. They realized the potential early on and started tailoring their GPUs for AI, developing software to support it, and working with researchers around the world. It was a risky bet, but it paid off big time.

Now that Nvidia dominates the AI GPU market, they're looking for new areas to expand into. That's what led them to physical AI and robotics. Nvidia wants to be the brains behind all the robots in the world. To achieve this, they're focusing on developing the essential technologies needed to make it happen.

Sanja Fidler, Nvidia's vice president of AI research, joined the team in 2018 and brought her expertise in simulation models for robots. She was already working on this at MIT when Jensen Huang, Nvidia's CEO, expressed interest. She couldn't resist the opportunity to join Nvidia and create a research lab in Toronto called Omniverse, which focuses on building simulations for physical AI. It's like a virtual playground for robots.

One of the first challenges was finding enough 3D data to build these simulated worlds. They needed a ton of images and the technology to turn them into 3D models that the simulators could use. They invested in something called differentiable rendering, which makes rendering AI-friendly. Omniverse released its first model that turns images into 3D models, GANverse3D, in 2021. They also developed the Neuric Neural Reconstruction Engine to create 3D models and simulations from videos of robots and self-driving cars.

The team is now working on making these models faster so that robots can react in real-time. They want robots to be able to process information much faster than humans. If they can make these models significantly faster, they'll be incredibly useful for robotics and physical AI applications.

Despite all the excitement surrounding robots, the Nvidia research team remains grounded. They believe that we're still a few years away from having a humanoid robot in every home. But, with the progress they're making in visual AI, generative AI, and data collection, they're confident that robots will continue to improve and become more capable.