Learning-from-observation 2.0: Automatic Acquisition of Robot Behavior from Human Demonstration

This book presents recent breakthroughs in the field of Learning-from-Observation (LfO) resulting from advancement in large language models (LLM) and reinforcement learning (RL) and positions it in the context of historical developments in the area. LfO involves observing human behaviors and generating robot actions that mimic these behaviors. While LfO may appear similar, on the surface, to Imitation Learning (IL) in the machine learning community and Programing-by-Demonstration (PbD) in the robotics community, a significant difference lies in the fact that these methods directly imitate human hand movements, whereas LfO encodes human behaviors into the abstract representations and then maps these representations onto the currently available hardware (individual body) of the robot, thus indirectly mimicking them. This indirect imitation allows for absorbing changes in the surrounding environment and differences in robot hardware. Additionally, by passing through this abstract representation, filtering can occur, distinguishing between important and less important aspects of human behavior, enabling imitation with fewer demonstrations and less demanding demonstrations. The authors have been researching the LfO paradigm for the past decade or so.  Previously, the focus was primarily on designing necessary and sufficient task representations to define specific task domains such as assembly of machine parts, knot-tying, and human dance movements. Recent advancements in Generative Pre-trained Transformers (GPT) and RL have led to groundbreaking developments in methods to obtain and map these abstract representations. By utilizing GPT, the authors can automatically generate abstract representations from videos, and by employing RL-trained agent libraries, implementing robot actions becomes more feasible.

Katsushi Ikeuchi received his Ph.D. in Information Engineering from the University of Tokyo.  He worked at MIT-AI as a postdoctoral researcher, CMU-RI as a research professor, U Tokyo as a professor before joining Microsoft in 2015. At MIT-AI, he was engaged in the development of algorithms for the world first bin-picking system using photometric stereo. At CMU-RI, he started the Learning-from-Observation project, focusing on developing robots that can acquire behavior from human demonstrations. At U Tokyo, he applied this LfO to develop a humanoid robot that can perform the Aizu Bandaisan Dance, Knot typing, and assemble mechanical parts. He has served as general or program chair for a dozen international conferences including IROS1995, CVPR1996, ICRA 2009 and ICCV 2017. He served as Editor-in-Chief of Springer-IJCV for more than 10 years. He has received Distinguished Researcher Award from IEEE-PAMI-TC and the Medal of Honor with purple ribbon from the Emperor of Japan.  He is a Fellow of IEEE, IEICE, IPSJ, RSJ, and IAPR.

Naoki Wake received his Ph.D. degree in Information Science and Technology at the University of Tokyo, Japan in 2019. He currently works at Microsoft as a Research Scientist for Industrial Solutions and Engineering.  His current research involves the development of multimodal perception systems for robots and co-speech gesturing systems. His past research has spanned auditory neuroscience, neurorehabilitation, and speech processing.

Jun Takamatsu received his Ph.D. degree in Computer Science from the University of Tokyo, Japan, in 2004.  From 2004 to 2008, he was with the Institute of Industrial Science, the University of Tokyo. In 2007, he was with Microsoft Research Asia as a Visiting Researcher.  From 2008 to 2021, he was with Robotics Laboratory, Nara Institute of Science and Technology, Japan, as an Associate Professor. He was also with Carnegie Mellon University as a Visitor in 2012 and 2013 and with Microsoft as a Visiting Scientist in 2018.  Now, he is with Microsoft as a Senior Research Scientist. His research interests are in robotics including learning-from-observation, task/motion planning, feasible motion analysis, 3D shape modeling and analysis, and physics-based vision.Kazuhiro Sasabuchi received his Ph.D. degree in Information Science and Technology at the University of Tokyo, Japan in 2019.  He has worked across various fields in robotics including human-robot interaction, hardware design, field robotics, robot systems, robot teaching, reinforcement learning, and mobile manipulation.  He currently works at Microsoft as a Research Scientist for Industrial Solutions and Engineering.  His interests are in practical robot systems which leverage composable skills, cloud operations, large language models, human interaction, simulation, and machine-learning.