Syllabus for Theoretical Foundations for Data Science

Learning outcomes

On the completion of the course the student should be able to:

• formulate decision problems, including action space and loss function, in particular for hypothesis testing and estimation problems,
• derive confidence bounds using limit theorems such as Glivenko-Cantelli lemma and Dvoretzky-Kiefer-Wolfowitz inequality,
• use concentration inequalities to derive bounds for specific distributions and give proofs of basic inequalities,
• derive Bayes-optimal rules for simple decision problems,
• obtain finite sample bounds for estimators via Vapnik-Chervonenkis dimension,
• obtain lower bounds on minimax risk,
• select appropriate model complexity measures to balance bias and variance, for example by penalization,
• apply the above to derive/implement algorithms, including those designed for use under resource constraints.

Content

Uniform limit theorems and empirical processes, concentration of measure, optimality criteria in statistical decision theory, generalisation bounds and learning theory, minimax lower-bounds and information theory, low-dimensional approximations,

algorithms for decision procedures under constraints, for eg. computing resources and applicable law.

Instruction

Lectures and problem solving sessions.

Assessment

Written assignments that are also presented orally in connection with the teaching (7.5 credits).

If there are special reasons for doing so, an examiner may make an exception from the method of assessment indicated and allow a student to be assessed by another method. An example of special reasons might be a certificate regarding special pedagogical support from the disability coordinator of the university.