Reinforcement Learning (SS 16)

Written Exam CHANGED: 9:00 – 11:00 Thursday 21.7, room V7.11

Instructors: Vien Ngo and Hung Ngo

Machine Learning & Robotics Lab, IPVS, University of Stuttgart.

Time:

  • Lessons: Wed. 14:00-15:30 at room 0.108
  • Tutorials: Tue. 17:30-19:00 at room 38.03

Description

Reinforcement learning is an area of machine learning in computer science, concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward. The problem, due to its generality, is studied in many other disciplines, such as game theory, control theory, operations research, information theory, simulation-based optimization, statistics, and genetic algorithms. (Wikipedia)

As encountered in nature, human or animal can learn by interacting with environment, and see the effect to adapt and respond to changes to achieve the goals. These creatures easily perceive cause and effect without having supervised data or teachers. Reinforcement learning is a type of machine learning frameworks which addresses such learning machinery. It aims to find an optimal policy to achieve the goal by interacting with the environment in absence of explicit teachers. For instance, the recent success of AlphaGO which is a computer program to play the game GO; in robotics, robots learn to understand environment and achieve new skills; in commercial smart ads where ads are optimally adapted-to-user chosen by learning through activities of users on the site; etc..

Schedule, slides & exercises

datetopicsslidesexercises
06.04Introduction01-IntroductionNo tutorial on 05.06
13.04MDP, Dynamic Programming02-MDP01-exercise (due 12.04)
01-solution
20.04TD Learning03-TDLearning02-exercise (due 19.04)
16-RL-code
26.04Q-Learning(cont.)03-exercise (due 26.04)
03.05RL with Func. Approx.04-FunctionApproximation04-exercise (due 03.05)
10.05Deep RL(cont.)05-exercise (due 10.05)
17.05Semester BreakNo Lectures
24.05Integrated Learning-Planning05-ModelRL-Dyna-Explore06-exercise (due 24.05)
31.05Exploration-Exploitation(cont.)07-exercise-revised (due 31.05)
08.06Policy Search06-PolicySearch08-exercise (due 07.06)
15.06(cont.)09-exercise (due 14.06)
e09-code
22.06(cont.)10-exercise (due 21.06)
29.06Hierarchical RL07-HierarchicalRL11-exercise (due 28.06)
06.07Inverse RL08-InverseRL12-exercise (due 05.07)
13.07ReviewQ&A lecture13-exercise (due 12.07)
EXERCISE Canceled (Vien's Travel)

Literature

  • R. Sutton and A. Barto, Reinforcement Learning, 1998. This book is freely available online
  • C. Szepesvari, Algorithms for Reinforcement Learning, 2010. Draft version is freely available online.
  • Steven M. LaValle, Planning Algorithms, 2006, Cambridge University Press. This book is freely available online.