Syllabus for Markov Processes



  • 10 credits
  • Course code: 1MS012
  • Education cycle: Second cycle
  • Main field(s) of study and in-depth level: Mathematics A1N
  • Grading system: Fail (U), Pass (3), Pass with credit (4), Pass with distinction (5)
  • Established: 2007-03-15
  • Established by:
  • Revised: 2022-10-18
  • Revised by: The Faculty Board of Science and Technology
  • Applies from: Autumn 2023
  • Entry requirements:

    120 credits. Probability Theory I. Proficiency in English equivalent to the Swedish upper secondary course English 6.

  • Responsible department: Department of Mathematics

Learning outcomes

On completion of the course, the student should be able to:

  • have a general knowledge of the theory of stochastic processes, in particular Markov processes, and be prepared to use Markov processes in various areas of applications;
  • be familiar with Markov chains in discrete and continuous time with respect to state diagram, recurrence and transience, classification of states, periodicity, irreducibility, etc., and be able to calculate transition probabilities and intensities;
  • be able to give an account of existence and uniqueness for stationary and asymptotic distributions of Markov chains and, whenever applicable, compute such distributions as solutions of a balance equation;
  • be able to calculate absorption probabilities and expected absorption time for Markov chains using the principle of conditioning with respect to the first jump;
  • be able to choose a suitable Markov model in various cases and make suitable calculations, in particular modelling of birth-death processes;
  • have a knowledge of Markov processes with a continuous state space, in particular a preparatory knowledge of Brownian motion and diffusion, and some understanding of the connection between the theory of Markov processes and differential equations;
  • have a knowledge of some general Markov method, e.g. Markov Chain Monte Carlo.


The Markov property. Chapman-Kolmogorov's relation, classification of Markov processes and transition probability. Transition intensity, forward and backward equations. Stationary and asymptotic distribution. Convergence of Markov chains. Birth-death processes. Absorption probabilities and absorption time. Brownian motion and diffusion as well as Geometric Brownian motion. Generalised Markov models. Applications of Markov chains.


Lectures and problem solving sessions. Written assignments as well as project which is presented orally.


Written examination (10 credits).

If there are special reasons for doing so, an examiner may make an exception from the method of assessment indicated and allow a student to be assessed by another method. An example of special reasons might be a certificate regarding special pedagogical support from the disability coordinator of the university.

Reading list

Reading list

Applies from: Autumn 2023

Some titles may be available electronically through the University library.

  • Lawler, Gregory F. Introduction to stochastic processes

    2. ed.: 2006

    Find in the library

  • Stirzaker, David Stochastic processes and models

    Oxford: Oxford Univ. Press, 2005

    Find in the library

Last modified: 2022-04-26