[Univ of Cambridge] [Dept of Engineering]

Statistical Pattern Processing - Module 4F10

The lectures of this part of the course aim to describe the basic concepts of statistical pattern processing and some of the standard techniques used in pattern classification.

Principal lecturers: Professor PC Woodland, Dr. WJ Byrne
Departmental course module page: http://www.eng.cam.ac.uk/teaching/courses/y4/4f10.html
Any queries, problems, or errors in the handouts, please contact me by email: bill.byrne@eng.cam.ac.uk

A related 4th year module is 4F13 Machine Learning

[ Handouts | Examples Papers | On-line Material | Further Reading ]


The lecture notes should be available online just before the lectures.
  • Handout 1: Introduction and Bayes' Decision Theory.
    Statistical pattern processing, Bayesian decision theory, classification cost, ROC curves
    Notes available in [pdf]
  • Handout 2: Multivariate Gaussian and Decision Boundaries.
    Multivariate Gaussian PDFs, maximum likelihood estimation, decision boundaries, generalisation
    Notes available in [pdf]
  • Handout 3: Gaussian Mixture Models.
    Gaussian mixture models, parameter estimation, introduction to EM.
    Notes available in [pdf]
  • Handout 4: Expectation Maximisation.
    Latent variables both continuous and discrete, proof of EM, factor analysis.
    Notes available in [pdf]
  • Handout 5: Linear Classifiers.
    Single layer perceptron, perceptron learning algorithm, Fisher's linear discriminant analysis, limitations.
    Notes available in [pdf]
  • Handout 6 and 7: Multi-Layer Perceptrons.
    Basic structure, posterior modelling, regression.
    Notes available in [pdf] and [pdf]
  • Handout 8: Support Vector Machines.
    Maximum margin classifiers, handling non-separable data, training SVMs, non-linear SVMs, kernel functions.
    Notes available in [pdf]
  • Handout 9: Gaussian Processes.
    inear and basis function regression, Gaussian processes, covariance functions, classification
    Notes available in [pdf]
    Carl Rasmussen's notes available in [pdf]
  • Handout 10: Relevance Vector Machines and Classification.
    RVM weight prior, training using EM, classification.
    2012 notes available in [pdf]
  • Handout 11: Classification and Regression Trees.
    Decision trees, query selection, multivariate decision trees.
    2012 notes available in [pdf] (mortgage examples [ps])
  • Handout 12: Non-Parametric Techniques.
    Parzen windows, K-nearest neighbours, nearest neighbour rule.
    2012 notes available in [pdf]
  • Handout 13: Speaker Verification and Identification.
    GMMs and MAP estimation, SVM-based verification, dynamic kernels.
    2012 notes available in [pdf]

Examples papers

The examples class for the first examples paper is planned for the end of week 4. For the second examples paper the end of week 8.
  • The first examples paper for 4F10 statistical pattern processing are available in [pdf].
    Solutions for examples paper 1 are available in [pdf].
  • The second examples paper for 4F10 statistical pattern processing are available in [pdf].
    Solutions for examples paper 2 are. available in [pdf].

On-line material

WARNING: I have no control over any website beyond the engineering department. The links provided were valid when I checked them. If a link has changed, or the contents are impropriate please email me ASAP.

Recommended reading

  • Judea Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, San Mateo, CA, 1997. ISBN 1558604790.
  • (*) Richard Duda, Peter Hart and David Stork: Pattern Classification, Second Edition, John Wiley & Sons Inc, 2000. ISBN 0471056693
  • (*) Christopher Bishop, Neural Networks for Pattern Recognition, Clarendon Press, 1995. ISBN 0198538642
  • (*) Christopher Bishop, Pattern Recognition and Machine Learning, Springer 2006.

[ Cambridge University | CUED | MIL | Home]