In an era of rapid AI progress, leveraging accelerated computing and big data has unlocked new possibilities to develop generalist AI models. As AI systems like ChatGPT showcase remarkable performance in the digital realm, we are compelled to ask: Can we achieve similar breakthroughs in the physical world — to create generalist humanoid robots capable of performing everyday tasks? In this talk, I will present our data-centric research principles and approaches for building general-purpose robot autonomy in the open world. I will discuss our recent works leveraging real-world, synthetic, and web data for training robotic foundation models. By combining these advances with cutting-edge developments in humanoid robotics, I will outline a roadmap for the next generation of autonomous robots.
Yuke Zhu is an Associate Professor in the Computer Science Department of UT-Austin, where he directs the Robot Perception and Learning (RPL) Lab. He is also a Director and Distinguished Research Scientist at NVIDIA Research, where he co-leads the Generalist Embodied Agent Research (GEAR) lab. He focuses on developing intelligent algorithms for generalist robots and embodied agents to reason about and interact with the real world. He obtained his Ph.D. degree from Stanford University. He received the NSF CAREER Award, the IEEE RAS Early Academic Career Award, and various faculty fellowships and research awards from Amazon, JP Morgan, and Sony Research.