Statistical Pattern Processing  Module 4F10
The lectures of this part of the course aim to describe the
basic concepts of statistical pattern processing and some of the
standard techniques used in pattern classification.
Any queries, problems, or errors in the handouts, please contact me by email mjfg@eng.cam.ac.uk .
A related 4th year module is 4F13 Machine Learning
[
Handouts 
Examples Papers 
Exam Questions 
Online Material 
Further Reading
]
Handouts
The lecture notes should be available online just before the lectures.
 Handout 1: Introduction and Bayes' Decision Theory.
Statistical pattern processing, Bayesian decision theory, classification cost, ROC curves
Notes available in [pdf] [slides]
 Handout 2: Multivariate Gaussian and Decision Boundaries.
Multivariate Gaussian PDFs, maximum likelihood estimation, decision boundaries, generalisation
Notes available in [pdf] [slides]
 Handout 3: Gaussian Mixture Models.
Gaussian mixture models, parameter estimation, introduction to EM.
Notes available in [pdf] [slides]
 Handout 4: Expectation Maximisation.
Latent variables both continuous and discrete, proof of EM, factor analysis.
Notes available in [pdf] [slides]
 Handout 5: Mixture and Product of Experts.
Gating functions, mixture versus product of experts
Notes available in [pdf] [slides]
 Handout 6: Restricted Boltzmann Machines.
RBM Structure, contrastive divergence
Notes available in [pdf] [slides]
 Handout 7: Linear Classifiers.
Single layer perceptron, perceptron learning algorithm, Fisher's linear discriminant analysis,
limitations.
Notes available in [pdf] [slides]
 Handout 8: MultiLayer Perceptrons.
Basic structure, posterior modelling, regression., deep topologies and initialisation
Notes available in [pdf] [slides]
 Handout 9: Support Vector Machines.
Maximum margin classifiers, handling nonseparable data, training SVMs, nonlinear
SVMs, kernel functions.
Notes available in [pdf] [slides]
 Handout 10: Classification and Regression Trees.
Decision trees, query selection, multivariate decision trees.
Notes available in [pdf] [slides]
 Handout 11: NonParametric Techniques.
Parzen windows, Knearest neighbours, nearest neighbour rule.
Notes available in [pdf] [slides]
 Handout 12: Speaker Verification and Identification.
GMMs and MAP estimation, SVMbased verification, dynamic kernels.
Notes available in [pdf]
Examples papers
The examples class for the first examples paper is planned for the end of week 4. For the second examples
paper the end of week 8.
 The first examples paper for 4F10 statistical pattern processing are
available in [pdf].
Solutions for examples paper 1
are available in [pdf].
 The second examples paper for 4F10 statistical pattern processing are
available in [pdf].
Solutions for examples paper 2
are available in [pdf].
Exam Questions
The following past 4F10 exam questions, which require knowledge of Gaussian Processes (see 4F13), are NOT
covered by the current 4F10 syllabus.
 2012: Qu 4
 2011: Qu 5
 2010: Qu 3
 2009: Qu 3
 2008: Qu 3
Past exam questions are available at the CUED webpage.
Online material
WARNING: I have no control over any website beyond the engineering department. The links
provided were valid when I checked them. If a link has changed, or the contents are
impropriate please email me ASAP.
Recommended reading

Judea Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible
Inference, Morgan Kaufmann, San Mateo, CA, 1997. ISBN 1558604790.

(*) Richard Duda, Peter Hart and David Stork: Pattern Classification, Second Edition,
John Wiley & Sons Inc, 2000. ISBN 0471056693

(*) Christopher Bishop, Neural Networks for Pattern Recognition, Clarendon Press, 1995.
ISBN 0198538642

(*) Christopher Bishop, Pattern Recognition and Machine Learning, Springer 2006.
top
[ Cambridge University 
CUED 
MIL 
Home]
