AI can speak fluently, reason through problems, and describe the world.
Getting it to do something in the world is a much harder problem.
In this episode of Automated, Brian Heater speaks with Dhruv Batra, co-founder and chief scientist at Yutori, about the gap between language models that generate answers and intelligent systems that can take action.
Dhruv explains why many post-ChatGPT robot demos are still little more than a physical wrapper around a language model. A robot may be able to hold a convincing conversation, but that does not prove it can navigate, manipulate objects, understand physical space, or respond to the movement of people around it.
The conversation begins with an even more fundamental question: What does it mean for an AI system to have a belief? Dhruv traces the idea back to Bayesian probability and explains why a rational decision-maker should never assign exactly zero probability to an event. Once a possibility reaches zero, no amount of new evidence can change that belief.
Brian and Dhruv also examine the human language used to describe artificial intelligence. Words like “belief” and “thinking” carry rich meanings in everyday life, but AI researchers often use them in much narrower technical ways. Dhruv argues that this trade-off can create confusion, while also giving researchers something precise enough to measure and improve.
They trace Dhruv’s path from probabilistic machine learning and computer vision to visual question answering, Grad-CAM, embodied AI at Meta FAIR, and eventually Yutori. Along the way, he explains how his team trained virtual robots to navigate directly from pixels to actions without building a map, using LiDAR, or separating perception from planning.
The conversation also explores why simulation has earned a complicated reputation in robotics, where sim-to-real transfer works well for locomotion but remains far more difficult for raw camera images and dexterous manipulation. Dhruv recalls the blunt question roboticists asked whenever he presented work in simulation: “Did you touch a robot?”
At Yutori, Dhruv is now applying many of those embodied AI ideas to web agents. These systems perceive a browser through screenshots and take actions such as clicking buttons, entering information, and completing tasks. He calls them “robots of the web.” Unlike traditional automation, they do not depend on a website remaining perfectly structured or unchanged.
The result is a wide-ranging conversation about uncertainty, paradigm shifts, robot learning, web agents, and the difference between an AI system that can answer a question and one that can act on your behalf.
Connect with Dhruv Batra
https://www.linkedin.com/in/dhruv-batra-dbatra/
Learn more about Dhruv Batra
https://dhruvbatra.com/
Learn more about Yutori
https://yutori.com/
We’d love to hear from you. Have thoughts or guest suggestions?
Reach us at podcast@automate.org
You can find the transcript and more episodes of Automated at automated.fm
Unlock full access to Automated and explore everything automation.
Subscribe today and leave a review on YouTube, Apple Podcasts, Spotify, and the Automated Newsletter.
https://www.youtube.com/@automatedpodcast
https://podcasts.apple.com/us/podcast/automated-with-brian-heater/id1837762221
https://open.spotify.com/show/60olq6brlBEIJWggx2fMR6
https://www.automate.org/automation/automated-newsletter
LinkedIn
https://www.linkedin.com/showcase/automated-podcast-by-a3/
Instagram
https://www.instagram.com/automatedpod/
1 h et 1 min
46 min
50 min
54 min
52 min
46 min
58 min
42 min