Optimization Course SS 14 U Stuttgart
Optimization is one of the most fundamental tools of modern
sciences. Many phenomena  be it in computer science, artificial
intelligence, logistics, physics, finance, or even psychology and
neuroscience  are typically described in terms of optimality
principles. The reason is that it is often easier to describe or
design an optimality principle or cost function rather than the system
itself. However, if systems are described in terms of optimality
principles, the computational problem of optimization becomes central
to all these sciences.
This lecture aims give an overview and introdution to various
approaches to optimization together with practical experience in the
exercises. The focus will be on continuous optimization problems and
we will cover methods ranging from standard convex optimization and
gradient methods to nonlinear black box problems (evolutionary
algorithms) and optimal global optimization. Students will learn to
identify, mathematically formalize, and derive algorithmic solutions
to optimization problems as they occur in nearly all disciplines.
A preliminary list of topics is:
 gradient methods, logbarrier, conjugate gradients, Rprop
 constraints, KKT, primal/dual
 Linear Programming, simplex algorithm
 (sequential) Quadratic Programming
 Markov Chain Monte Carlo methods
 2nd order methods, (Gauss)Newton, (L)BFGS
 blackbox stochastic search, including a discussion of evolutionary algorithms
 Organization

 This is the central website of the lecture. Link to slides, exercise sheets, announcements, etc will all be posted here.
 More information to come
 Schedule, slides & exercises

date topics slides exercises
(due on 'date'+1)15.4. Introduction 01introduction  22.4. Unconstrained Optimization 02unconstrainedOpt e01introduction 29.4. Unconstrained Optimization (cont.) [Nathan did the lecture. Sorry I was ill.] e02unconstrainedOpt 6.5. Constrained Optimization 03constrainedOpt e03newtonMethods 13.5. Constrained Optimization & Convex Problems 04convexProblems e04constraints 20.5. Constrained Optimization & Convex Problems (cont.) questions time (optional to attend) 27.5. Blackbox Optimization 05blackBoxOpt e05constrainedOpt 3.6. Blackbox Optimization (cont.) e06convexOpt 10.6. [Pfingstferien] 17.6. Bayesian Optimization 06globalBayesianOptimization e07stochasticSearch 24.6. Bayesian Optimization (cont.) [no exercise] 1.7. Consider this to prepare for exam: 14Optimizationscript e08globalOptim
../data/gp01pred.m ../data/test.m  Literature