[Univ of Cambridge] [Dept of Engineering]

Statistical Pattern Processing - Module 4F10

The lectures of this part of the course aim to describe the basic concepts of statistical pattern processing and some of the standard techniques used in pattern classification.

Any queries, problems, or errors in the handouts, please contact me by email mjfg@eng.cam.ac.uk .

A related 4th year module is 4F13 Machine Learning

[ Handouts | Examples Papers | Exam Questions | On-line Material | Further Reading ]


The lecture notes should be available online just before the lectures.
  • Handout 1: Introduction and Bayes' Decision Theory.
    Statistical pattern processing, Bayesian decision theory, classification cost, ROC curves
    Notes available in [pdf] [slides]
  • Handout 2: Multivariate Gaussian and Decision Boundaries.
    Multivariate Gaussian PDFs, maximum likelihood estimation, decision boundaries, generalisation
    Notes available in [pdf] [slides]
  • Handout 3: Gaussian Mixture Models.
    Gaussian mixture models, parameter estimation, introduction to EM.
    Notes available in [pdf] [slides]
  • Handout 4: Expectation Maximisation.
    Latent variables both continuous and discrete, proof of EM, factor analysis.
    Notes available in [pdf] [slides]
  • Handout 5: Mixture and Product of Experts.
    Gating functions, mixture versus product of experts
    Notes available in [pdf] [slides]
  • Handout 6: Restricted Boltzmann Machines.
    RBM Structure, contrastive divergence
    Notes available in [pdf] [slides]
  • Handout 7: Linear Classifiers.
    Single layer perceptron, perceptron learning algorithm, Fisher's linear discriminant analysis, limitations.
    Notes available in [pdf] [slides]
  • Handout 8: Multi-Layer Perceptrons.
    Basic structure, regression., deep topologies and initialisation
    Notes available in [pdf] [EBP details]
  • Handout 9: Support Vector Machines.
    Maximum margin classifiers, handling non-separable data, training SVMs, non-linear SVMs, kernel functions.
    Notes available in [pdf] [slides]
  • Handout 10: Classification and Regression Trees.
    Decision trees, query selection, multivariate decision trees.
    Notes available in [pdf] [slides]
  • Handout 11: Non-Parametric Techniques.
    Parzen windows, K-nearest neighbours, nearest neighbour rule.
    Notes available in [pdf] [slides]
  • Handout 12: Speaker Verification and Identification.
    GMMs and MAP estimation, SVM-based verification, dynamic kernels.
    Notes available in [pdf]

Examples papers

The examples class for the first examples paper is planned for the end of week 4. For the second examples paper the end of week 8.
  • The first examples paper for 4F10 statistical pattern processing are available in [pdf].
    Solutions for examples paper 1 are available in [pdf].
  • The second examples paper for 4F10 statistical pattern processing are available in [pdf].
    Solutions for examples paper 2 are available in [pdf].

Exam Questions

The following past 4F10 exam questions, which require knowledge of Gaussian Processes (see 4F13), are NOT covered by the current 4F10 syllabus.
  • 2012: Qu 4
  • 2011: Qu 5
  • 2010: Qu 3
  • 2009: Qu 3
  • 2008: Qu 3
Past exam questions are available at the CUED web-page.

On-line material

WARNING: I have no control over any website beyond the engineering department. The links provided were valid when I checked them. If a link has changed, or the contents are impropriate please email me ASAP.

Recommended reading

  • Judea Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, San Mateo, CA, 1997. ISBN 1558604790.
  • (*) Richard Duda, Peter Hart and David Stork: Pattern Classification, Second Edition, John Wiley & Sons Inc, 2000. ISBN 0471056693
  • (*) Christopher Bishop, Neural Networks for Pattern Recognition, Clarendon Press, 1995. ISBN 0198538642
  • (*) Christopher Bishop, Pattern Recognition and Machine Learning, Springer 2006.
  • (*) D.J.C. Mackay, Information Theory, Inference and Learning Algorithms, CUP, 2004. (CUED: NO.277)

[ Cambridge University | CUED | MIL | Home]