Written Exam CHANGED: 9:00 – 11:00 Thursday 21.7, room V7.11
Instructors: Vien Ngo and Hung Ngo
Machine Learning & Robotics Lab, IPVS, University of Stuttgart.
Time:
- Lessons: Wed. 14:00-15:30 at room 0.108
- Tutorials: Tue. 17:30-19:00 at room 38.03
Description
Reinforcement learning is an area of machine learning in computer science, concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward. The problem, due to its generality, is studied in many other disciplines, such as game theory, control theory, operations research, information theory, simulation-based optimization, statistics, and genetic algorithms. (Wikipedia)
As encountered in nature, human or animal can learn by interacting with environment, and see the effect to adapt and respond to changes to achieve the goals. These creatures easily perceive cause and effect without having supervised data or teachers. Reinforcement learning is a type of machine learning frameworks which addresses such learning machinery. It aims to find an optimal policy to achieve the goal by interacting with the environment in absence of explicit teachers. For instance, the recent success of AlphaGO which is a computer program to play the game GO; in robotics, robots learn to understand environment and achieve new skills; in commercial smart ads where ads are optimally adapted-to-user chosen by learning through activities of users on the site; etc..
Schedule, slides & exercises
| date | topics | slides | exercises |
|---|---|---|---|
| 06.04 | Introduction | 01-Introduction | No tutorial on 05.06 |
| 13.04 | MDP, Dynamic Programming | 02-MDP | 01-exercise (due 12.04) 01-solution |
| 20.04 | TD Learning | 03-TDLearning | 02-exercise (due 19.04) 16-RL-code |
| 26.04 | Q-Learning | (cont.) | 03-exercise (due 26.04) |
| 03.05 | RL with Func. Approx. | 04-FunctionApproximation | 04-exercise (due 03.05) |
| 10.05 | Deep RL | (cont.) | 05-exercise (due 10.05) |
| 17.05 | Semester Break | No Lectures | |
| 24.05 | Integrated Learning-Planning | 05-ModelRL-Dyna-Explore | 06-exercise (due 24.05) |
| 31.05 | Exploration-Exploitation | (cont.) | 07-exercise-revised (due 31.05) |
| 08.06 | Policy Search | 06-PolicySearch | 08-exercise (due 07.06) |
| 15.06 | (cont.) | 09-exercise (due 14.06) e09-code |
|
| 22.06 | (cont.) | 10-exercise (due 21.06) |
|
| 29.06 | Hierarchical RL | 07-HierarchicalRL | 11-exercise (due 28.06) |
| 06.07 | Inverse RL | 08-InverseRL | 12-exercise (due 05.07) |
| 13.07 | Review | Q&A lecture | 13-exercise (due 12.07) EXERCISE Canceled (Vien's Travel) |
Literature
- R. Sutton and A. Barto, Reinforcement Learning, 1998. This book is freely available online
- C. Szepesvari, Algorithms for Reinforcement Learning, 2010. Draft version is freely available online.
- Steven M. LaValle, Planning Algorithms, 2006, Cambridge University Press. This book is freely available online.