[Univ of Cambridge] [Dept of Engineering]

Statistical Pattern Processing - Module 4F10

Any queries, problems, or errors in the handouts on lectures 1-7, please contact Phil Woodland by email pcw@eng.cam.ac.uk or for lectures 8-14 please contact Mark Gales mjfg@eng.cam.ac.uk

Handouts

These lectures of this part of the course aim to describe the basic concepts of statistical pattern processing and some of the standard techniques used in pattern classification.

The lecture notes should be available online just after the lectures.

  • Lecture 1: Introduction and Bayes' Decision Theory.
    Statistical pattern processing, Bayesian decision theory, generalisation.
    Notes available in [pdf]
  • Lecture 2: Multivariate Gaussian and Decision Boundaries.
    Multivariate Gaussian PDFs, maximum likelihood estimation, decision boundaries, classification cost, ROC curves
    Notes available in [pdf]
  • Lecture 3: Gaussian Mixture Models.
    Gaussian mixture models, parameter estimation, introduction to EM.
    Notes available in [pdf]
  • Lecture 4: Expectation Maximisation.
    Latent variables both continuous and discrete, proof of EM, factor analysis.
    Notes available in [2-up pdf] [pdf]
  • Lecture 5: Linear Classifiers.
    Single layer perceptron, perceptron learning algorithm, Fisher's linear discriminant analysis, limitations.
    Notes available in [2-up pdf] [pdf]
  • Lecture 6: Multi-Layer Perceptrons.
    Basic structure, posterior modelling, regression.
    Notes available in [pdf]
  • Lecture 7: Multi-Layer Perceptrons (cont).
    Error back propogation, learning rates, second order methods.
    Notes available in [pdf]
  • Lecture 8: Support Vector Machines.
    Maximum margin classifiers, handling non-separable data.
  • Lecture 9: Support Vector Machines (cont).
    Training SVMs, non-linear SVMs, kernel functions.
  • Lecture 10-11: Gaussian Processes.
    Gaussian processes, covariance functions, non-linear regresion, Gaussian processes for classification.
  • Lecture 12: Classification and Regression Trees.
    Decision trees, query selection, multivariate decision trees.
  • Lecture 13: Non-Parametric Techniques.
    Parzen windows, K-nearest neighbours, nearest neighbour rule.
  • Lecture 14: Application: Speaker Verification and Identification.
    Speaker recognition/verification task, GMMs and MAP adaptation, SVM-based verification.

Examples papers

The examples papers for 4F10 statistical pattern processing will be available online. Examples paper 1 available in [pdf]. Note that the first examples class will be on Monday 12th February.
The solutions for the 4F10 statistical pattern processing examples papers will be available online.

Recommended Reading

  • (*) Richard Duda, Peter Hart and David Stork: Pattern Classification, Second Edition, John Wiley & Sons Inc, 2000. ISBN 0471056693
  • (*) Christopher Bishop, Neural Networks for Pattern Recognition, Clarendon Press, 1995. ISBN 0198538642
top

[ Cambridge University | CUED | MIL | Mark Gales Home] Phil Woodland Home]