# Optimization Course SS 15 U Stuttgart

Please subscribe to this mailing list.

Optimization is one of the most fundamental tools of modern sciences. Many phenomena -- be it in computer science, artificial intelligence, logistics, physics, finance, or even psychology and neuroscience -- are typically described in terms of optimality principles. The reason is that it is often easier to describe or design an optimality principle or cost function rather than the system itself. However, if systems are described in terms of optimality principles, the computational problem of optimization becomes central to all these sciences.

This lecture aims give an overview and introdution to various approaches to optimization together with practical experience in the exercises. The focus will be on continuous optimization problems and we will cover methods ranging from standard convex optimization and gradient methods to non-linear black box problems (evolutionary algorithms) and optimal global optimization. Students will learn to identify, mathematically formalize, and derive algorithmic solutions to optimization problems as they occur in nearly all disciplines. A preliminary list of topics is:

• gradient methods, log-barrier, conjugate gradients, Rprop
• constraints, KKT, primal/dual
• Linear Programming, simplex algorithm
• (sequential) Quadratic Programming
• Markov Chain Monte Carlo methods
• 2nd order methods, (Gauss-)Newton, (L)BFGS
• blackbox stochastic search, including a discussion of evolutionary algorithms
Students should bring basic knowledge of linear algebra and analysis.
Organization
• This is the central website of the lecture. Link to slides, exercise sheets, announcements, etc will all be posted here.
• See the 01-introduction slides for further information.
Schedule, slides & exercises
 date topics slides exercises(due on 'date'+1) 14.4. Introduction 01-introduction 21.4. Unconstrained Opt.(Pt.1) 02-unconstrainedOpt e01-introduction 28.4. Unconstrained Opt.(Pt.2) e02-unconstrainedOpt 06.5. Constrained Opt. 03-constrainedOpt e03-newtonMethods 13.5. e04-constraints 20.5. e05-lagrange 27.5. NO LECTURES 02.6. e06-primaldual 09.6. Convex Opt. 04-convexOpt e07-convexOpt 16.6. Global Bayesian Optimization 05-globalBayesianOptimization e08-ILPrelaxation 23.6. Philipp Hennig's slides on Entropy Search CANCELED 30.6. Blackbox Optimization 06-blackBoxOpt e09-globalOptim 07.7. e10-blackBoxOpt 14.7. SCRIPT 15-Optimization-script e11-blackBoxOpt_2
Literature