Attention and Inattention for Minimalist Robot Learners

Attention and Inattention for Minimalist Robot Learners

Abstract: Industry is placing big bets on "brute forcing" robotic control through scaling data, compute, and models, but a key blind spot of such scaling methods is that they are profligate in their use of expensive resources: power, compute, time, data, etc. Towards developing more minimalist robotic control stacks, my research group studies how agents can select and attend to task-relevant information during sensing, representation, decision making, and policy learning. I will speak about my group's work on exploiting privileged sensors at training time, combining pre-training language and vision models to compute task-relevant representations, and task-relevant world model learning.

Bio: Dinesh Jayaraman is an assistant professor at the University of Pennsylvania's CIS department and GRASP lab. He leads the Perception, Action, and Learning (Penn PAL) research group, which works at the intersections of computer vision, robotics, and machine learning. Dinesh received his PhD (2017) from UT Austin, before becoming a postdoctoral scholar at UC Berkeley (2017-19). Dinesh's research has received a Best Paper Award at CORL '22, a Best Paper Runner-Up Award at ICRA '18, a Best Application Paper Award at ACCV ‘16, the NSF CAREER award '23, an Amazon Research Award '21, and been covered in several press outlets. His webpage is at: https://www.seas.upenn.edu/~dineshj/