Projects

Phase 2 Projects

Autonomous active object learning through robot manipulation


Project leaders: Wolfram Burgard (Freiburg), Sven Behnke (Bonn)

Researchers: Andreas Eitel (Freiburg), Seongyong Koo (Bonn)

Administration: Susanne Bourjaillat (Freiburg), Nikola Kokisch (Bonn)

Associates:

Summary:

Service robots will be ubiquitous helpers of our everyday life. They are envisioned to perform a variety of tasks ranging from cleaning our apartments to mowing our lawn or feeding our pets. For solving such tasks successfully, robots must be able to properly reason about and manipulate objects, which requires rich knowledge about the corresponding objects.

To obtain such knowledge, robots need not only means for analyzing the geometry and appearance of objects, but also for retrieving their categories and inferring their characteristics. This includes, for example, physical properties, such as material and weight, or parts and their functions, as well as other forms of information, such as object brand or price.
For learning and identifying objects, a substantial amount of efforts of our scientific community concentrated on building dedicated object detection and recognition algorithms that are trained on extensive data sets. While this is a key enabling technology, we are still not at the point that they reliably work for large numbers of objects and with the many different types of objects encountered in everyday life. In this project, we aim at actively learning all the object information with minimal human supervision by leveraging the possibility of physical robot-object interaction and by analyzing the information available in the World Wide Web. Such object knowledge is key to endowing robots with a rich set of object manipulation skills. To use an object appropriately, a robot has to reason about the function of the object and its parts. The robot furthermore has to transfer its skills from a generic model to the specific shape and characteristics of the object. If it can distinguish the many variants of objects, the robot can adapt its skills to the corresponding object from experience.

In ALROMA, we propose a novel active learning perspective. Instead of building impractically extensive datasets and designing overly-complex techniques to cope with all the situations occurring in the real-word, we will provide the robot with prior knowledge about objects and give it access to the information available on the World Wide Web. The robot will then actively learn the objects by testing estimated hypotheses through the interaction with the environment, e.g., by taking object data or by pushing or picking an object. In a similar manner, the robot will improve its skills from the experience gathered through the interaction with the objects. For our research, we will use state-of-the-art robots equipped with capable manipulators (e.g., Willow Garage PR2 or Rethink Robotics Baxter). The robot will retrieve knowledge about objects that is not directly observable from the Internet. It will learn novel skills for manipulating the objects from demonstrations, which it generalizes and improves for similar objects and situations.

This project is a collaborative effort of ALU-FR and UBONN that both have extensive prior work in this field. We plan for an intense scientific interaction between our groups such as physical meetings and lab rotations of our researchers. ALROMA will advance the state of the art in active and robot learning, object manipulation and object discovery and thus  will contribute to the next generation of service robots.

Phase 1 Projects

Autonomous active object learning through robot manipulation


Project leaders: Wolfram Burgard (Freiburg), Sven Behnke (Bonn)

Researchers: Andreas Eitel (Freiburg), Seongyong Koo (Bonn)

Administration: Susanne Bourjaillat (Freiburg), Nikola Kokisch (Bonn)

Associates:

Summary:

Service robots will be ubiquitous helpers of our everyday life. They are envisioned to perform a variety of tasks ranging from cleaning our apartments to mowing our lawn or feeding our pets. For solving such tasks successfully, robots must be able to properly reason about and manipulate objects, which requires rich knowledge about the corresponding objects.

To obtain such knowledge, robots need not only means for analyzing the geometry and appearance of objects, but also for retrieving their categories and inferring their characteristics. This includes, for example, physical properties, such as material and weight, or parts and their functions, as well as other forms of information, such as object brand or price.
For learning and identifying objects, a substantial amount of efforts of our scientific community concentrated on building dedicated object detection and recognition algorithms that are trained on extensive data sets. While this is a key enabling technology, we are still not at the point that they reliably work for large numbers of objects and with the many different types of objects encountered in everyday life. In this project, we aim at actively learning all the object information with minimal human supervision by leveraging the possibility of physical robot-object interaction and by analyzing the information available in the World Wide Web. Such object knowledge is key to endowing robots with a rich set of object manipulation skills. To use an object appropriately, a robot has to reason about the function of the object and its parts. The robot furthermore has to transfer its skills from a generic model to the specific shape and characteristics of the object. If it can distinguish the many variants of objects, the robot can adapt its skills to the corresponding object from experience.

In ALROMA, we propose a novel active learning perspective. Instead of building impractically extensive datasets and designing overly-complex techniques to cope with all the situations occurring in the real-word, we will provide the robot with prior knowledge about objects and give it access to the information available on the World Wide Web. The robot will then actively learn the objects by testing estimated hypotheses through the interaction with the environment, e.g., by taking object data or by pushing or picking an object. In a similar manner, the robot will improve its skills from the experience gathered through the interaction with the objects. For our research, we will use state-of-the-art robots equipped with capable manipulators (e.g., Willow Garage PR2 or Rethink Robotics Baxter). The robot will retrieve knowledge about objects that is not directly observable from the Internet. It will learn novel skills for manipulating the objects from demonstrations, which it generalizes and improves for similar objects and situations.

This project is a collaborative effort of ALU-FR and UBONN that both have extensive prior work in this field. We plan for an intense scientific interaction between our groups such as physical meetings and lab rotations of our researchers. ALROMA will advance the state of the art in active and robot learning, object manipulation and object discovery and thus  will contribute to the next generation of service robots.