70007

Computational Optimisation

Module aims

In this module you will have the opportunity to:

• learn numerical methods for the solution of non-linear optimisation problems
• apply optimisation in engineering (e.g., the design of energy efficient chemical processes), machine learning (e.g., learning classifiers from data), and finance (e.g., optimal portfolio allocation)
• use analytical techniques and numerical algorithms to solve constrained and unconstrained problems
• identify convexity in a mathematical model and appreciate the importance of convexity both in theory and in practice
• identify necessary and sufficient conditions of optimality for different classes of optimisation models
• be able to apply an appropriate numerical method given the characteristics of the optimisation model
• understand the meaning of Lagrange multipliers
• be able to implement first/second order methods for constrained and unconstrained models

Learning outcomes

Upon successful completion of this module you will be able to:

• use advanced mathematical methods to establish convexity in complex problems
• specify necessary and sufficient conditions for optimality
• classify optimisation algorithms as first or second order
• determine appropriate optimisation algorithms for given problems given the size and structure of the optimisation models
• apply sensitivity analysis to optimisation problems using Lagrange multipliers

Module syllabus

This module covers the following topics:

• Prove convexity of a function and of an optimization problem;
• Specify the optimality conditions for both constrained and unconstrained optimization problems;
• Derive basic zeroth-, first-, and second-order optimization algorithms and evaluate their convergence;
• Apply optimization algorithms to machine learning applications, e.g., recognizing hand-writing and robust principal component analysis;
• Program basic zeroth-, first-, and second-order optimization algorithms.

 Recommended: COMP50011 Computational Techniques; COMP60016 Operations Research and COMP70015 Mathematics for Machine Learning

Teaching methods

This module primarily operates as a lecture-style module, but we place significant emphasis on (i) interaction with the students during lecture, e.g. via Mentimeter, (ii) discussion of modern research implications, e.g. via research papers that have recently appeared, (iii) a community of learners, e.g. via the online discussion boards,  (iv) open ended coursework that encourages students to try different possibilities.

A lecture-type format is important to develop the mathematical concepts and ideas. But we augment the lecture with little quizzes during the lecture that also enable student interaction. These Mentimeter questions reinforce students to understand and inspire students to raise their hands when they have questions (since they know that their colleagues also have questions). This module is fairly close to the research, so we typically discuss 1-2 papers during class and show their relationships to the class. Through the online discussion boards, we encourage active conversation about the material.  

Assessments

The module features two courseworks, one for each part covered by the lecturers, to better pace and consolidate the learning of the algorithms and modelling techniques explained in the course. These courseworks components equates 20% of the module marks. The remaining 80% will be assessed in a final written exam, which will test the understanding of the methods presented in the course as well as of their underpinning theory.                                                     
You will be provided with detailed written feedback on your individual coursework exercises. There will also be in-class and/ or written feedback explaining common pitfalls and suggestions for improvement.

Reading list

Module leaders

Mr Matthew Sheldon
Professor Ruth Misener