Markov Processes
Syllabus, Master's level, 1MS012
- Code
- 1MS012
- Education cycle
- Second cycle
- Main field(s) of study and in-depth level
- Mathematics A1N
- Grading system
- Fail (U), Pass (3), Pass with credit (4), Pass with distinction (5)
- Finalised by
- The Faculty Board of Science and Technology, 20 April 2009
- Responsible department
- Department of Mathematics
Entry requirements
120 credits including Inference Theory, or Probability and Statistics and Stochastic Modelling
Learning outcomes
In order to pass the course (grade 3) the student should
- have a general knowledge of the theory of stochastic processes, in particular Markov processes, and be prepared to use Markov processes in various areas of applications;
- be familiar with Markov chains in discrete and continuous time with respect to state diagram, recurrence and transience, classification of states, periodicity, irreducibility, etc., and be able to calculate transition probabilities and intensities;
- be able to give an account of existence and uniqueness for stationary and asymptotic distributions of Markov chains and, whenever applicable, compute such distributions as solutions of a balance equation;
- be able to calculate absorption probabilities and expected absorption time for Markov chains using the principle of conditioning with respect to the first jump;
- be able to choose a suitable Markov model in various cases and make suitable calculations, in particular modelling of birth-death processes;
- have a knowledge of Markov processes with a continuous state space, in particular a preparatory knowledge of Brownian motion and diffusion, and some understanding of the connection between the theory of Markov processes and differential equations;
- have a knowledge of some general Markov method, e.g. Markov Chain Monte Carlo.
Content
The Markov property. Chapman-Kolmogorov's relation, classification of Markov processes, transition probability. Transition intensity, forward and backward equations. Stationary and asymptotic distribution. Convergence of Markov chains. Birth-death processes. Absorption probabilities, absorption time. Brownian motion and diffusion. Geometric Brownian motion. Generalised Markov models. Applications of Markov chains.
Instruction
Lectures and problem solving sessions.
Assessment
Written and, possibly, oral examination at the end of the course. Moreover, compulsory assignments and project may be given during the course.
Reading list
- Reading list valid from Autumn 2023
- Reading list valid from Autumn 2022
- Reading list valid from Spring 2022
- Reading list valid from Spring 2019
- Reading list valid from Autumn 2015
- Reading list valid from Autumn 2013
- Reading list valid from Autumn 2009
- Reading list valid from Spring 2009, version 2
- Reading list valid from Spring 2009, version 1
- Reading list valid from Autumn 2008
- Reading list valid from Autumn 2007