Maths for Intelligent Systems (WS 14/15)

Instructors: Nathan Ratliff and Marc Toussaint
Machine Learning & Robotics Lab, IPVS, University of Stuttgart.

Co-listed with Theoretical and Methodological Foundations of Autonomous Systems

If you’re attending the course but aren’t yet on the course mailing list, please email Nathan Ratliff to be added. Important information regarding lectures, notes, homework, solutions, etc. is broadcast via the mailing list.

Time:

  • Lessons: Thurs 15:45-17:15 at room 0.463
  • Tutorial: Tue 17:30-19:00 at room 0.108
  • Office hours: Nathan – Thurs 14:00-15:00 at room 2.228

Zielgruppe: Master-Studiengang Informatik (Vertiefungslinien und Katalog MINF), Diplom-Studiengang Informatik (Hauptstudium), Master-Studiengang Softwaretechnik (Vertiefungslinien), Diplom-Studiengang Softwaretechnik (Hauptstudium), International Master in Computer Science – Studienprofil AUT

Description

This course prepares students for further study of autonomous robotic and learning systems. Methodologies in autonomous systems are highly mathematical and require on a concrete understanding of core areas such as linear algebra, functional analysis, differential geometry, optimization, probability, statistics, and decision theory. The course especially emphasizes geometric intuition behind more abstract ideas with concrete examples taken from robotics and machine learning. The first half of the course covers topics from higher mathematics that form the backbone and description language for many of the advanced tools and modeling techniques used in machine learning and robotics, and second half builds the theoretical foundations of decision theory exploring important tools from probability and statistics along the way. Specific topics include, coordinate free linear algebra, numerical linear algebra, multidimensional quadratic functions and Gaussians, algorithmic covariance, the method of lagrange multipliers and KKT conditions in optimization, probabilistic graphical models, Monte Carlo sampling, maximum likelihood and maximum entropy, statistical bounds, Markov decision processes, hidden Markov models, spectral algorithms, and more.

Topics:

  • Coordinate free Linear Algebra
  • Review of Vector Calculus and some Differential Geometry
  • Optimization
  • Probability
  • Statistics
  • Decision theory

Reference material. (Books, links, etc.)

Homework policy

Homework shapes your understanding of the material and gives you an opportunity to think critically about the concepts and their application. We’ll have weekly homework assignments, and we’ll discuss the solutions during the Tutorial sections. The homework, itself, doesn’t affect your final grade, but you need to have done at least 50% of the problems in order to take the final exam. (That’s 50% total, not on a per-week basis.)

Note again, though, that doing the homework is a very important part of the learning process, especially for this material. Please try your hand at each of the problems. During the tutorial section, if you have a solution that makes sense to you, mark down that you’ve done the problem. Even if the solution ends up begin “wrong”, it’s more important that it makes sense to you, and that though the discussion you revise your understanding so that you ultimately find the mistake and the correct solution makes sense. These problems are meant to be difficult, and in some cases more difficult than what I’d expect you to answer on the fly during a test. But doing them will force you to come to terms with what you do and do not understand, which itself is sometimes unclear initially with complicated topics.

Schedule, slides & exercises
DateLectureNotes & ReferencesHomework
16.10.14Introduction to the course and coordinate free linear algebraLecture 1 notesHomework 1
23.10.14Linear algebra II: SVD, Eigenvectors, fundamental spacesLecture 2 notesHomework 2
30.10.14Linear algebra III: Computation in coordinatesLecture 3 notes Homework 3
06.11.14Linear algebra IV: Eigen-algorithmsSame as previous notes.Homework 4
20.11.14Multi-dimensional calculusLecture 5 notesHomework 5
27.11.2014Smooth Mappings and Linear AlgebraLecture 6 notesHomework 6
Software (manipulator Jacobian)
04.12.14Optimization I: Analytical optimizationLecture 7 notesHomework 7
11.12.14Optimization II: Numeric methodsLecture 8 notes (preliminary)Homework 8
Links to software and data
18.12.14None! Canceled. Christmas!
08.01.15Probability I: The multi-dimensional basicsLecture 9 notesHomework 9
15.01.15ReviewReview homework
22.01.15Probability II: Maximum Likelihood and Structured in Probability and OptimizationLecture 11 notesHomework 11
29.01.15Statistical Bounds and Information TheoryLecture 12 notes I - Information Geometry and Natural Gradients
Lecture 12 notes II - Statistical bounds
Homework 12
05.02.15Decision theoryLecture 13 notesHomework 13
12.02.15Review and interesting applications