Michael Posa, an assistant professor with appointments in the departments of mechanical engineering and applied mechanics, computer and information science, and electrical and systems engineering of Penn Engineering, has received funding from the National Science Foundation’s CAREER Award to work on a new teaching method wherein robots interact with objects in the real world and observations from those interactions are used to create a training lesson plan or model. The new approach puts the real world first and simulation second, a prioritization that Dr. Posa believes is key to building robots’ real-world intelligence.

“I want a robot to be able to enter a room it has never been in and quickly characterize and manipulate objects to perform an assigned task,” Dr. Posa said, “whether that’s assisting in the home, conducting search and rescue operations or manufacturing items.”

But a robot is only as smart as scientists train it to be. The reason researchers have not yet been able to create robots with this real-world intelligence is due to the way they are teaching them. Robots currently learn through repetitive training in simulations and controlled laboratory settings. They may be great at performing an extremely precise motion over and over again, but have a hard time quickly reacting to diverse stimuli in an uncontrolled environment. To improve that ability, scientists need to teach robots how to process information in novel environments where there is no time or opportunity to waste.

To accomplish this, scientists must enable robots to learn from small data sets. Unlike machine learning models like ChatGPT, where big data sets of language, images and video can be found on the internet, data in robotics is hard to come by. Advanced robots remain expensive and require specialized skills to operate. While shared data sets are growing in scale, they are many orders of magnitude smaller than what is available in big data sets.

“As an example, for a robot to prepare meals in the home, it must master a huge range of different foods, tools and cooking strategies,” said Dr. Posa. “While humans intuitively understand and learn from small data—teaching a human how to prepare a meal just once or twice would be enough for them to successfully accomplish the task on their own—computers require far more repetition in an environment with little to no interruption.”

In addition to working with small data, future life-improving robots will need to be able to perform and react to many different, notoriously hard-to-model, discontinuous movements such as jumping, sticking, sliding, chopping and bouncing, all while adhering to the laws of physics.

With research underway, Dr. Posa plans to create a cloud-based lesson plan accessible to students around the world, using the award to bridge gaps in both research and outreach.

“Our future engineers are learning algebra and calculus now,” said Dr. Posa. “We want to show them how what they are learning in school relates to a real-world problem, solution and career path in robotics.”