Generalizable Learning for Robotic Autonomy

The primary goal of this thesis is to develop generalizable learning methods for planning and perception in robotic manipulation by designing deep learning architectures and leveraging pre-trained foundation models. These methods aim to establish a general prior that facilitates a comprehensive understanding of various tasks in robotics. Humans can easily apply knowledge to new situations, even in unfamiliar domains. This ability comes from the capacity for abstract thinking and the intuitive understanding, developed through diverse experiences. In robotics, a major goal is to give robots similar adaptability. Ideally, robots would quickly adjust to new environments and tasks they haven't encountered before, much like humans do. This capability would significantly advance robotic systems, allowing them to handle a wider range of real-world scenarios. Building autonomous robots capable of adapting to diverse situations without external intervention has long been a significant challenge in advancing real-world robotics applications. Achieving this level of adaptation requires developing robots that are not limited to specific tasks but can generalize their skills across various scenarios. This generalization must occur at multiple levels, enabling the robot to form a comprehensive understanding of unfamiliar situations. For example, at the planning level, the robot should quickly adjust its decision-making process, while at the perception level, it should effectively interpret unknown environments. This ability to generalize to new situations is often referred to as zero-shot or few-shot learning. In zero-shot learning, the robot can adapt to a new task without any prior examples, whereas, in few-shot learning, a small number of examples (even just one, in the case of one-shot learning) is sufficient for adaptation. This capability is crucial for developing general-purpose autonomous robots, particularly in achieving zero-shot and one-shot learning, which are essential for enabling autonomous adaptation across a wide range of situations with minimal data requirements during deployment.