Statistical Pattern Processing  Module 4F10
The lectures of this part of the course aim to describe the
basic concepts of statistical pattern processing and some of the
standard techniques used in pattern classification.
Principal lecturers: Professor
PC Woodland,
Dr. WJ Byrne
Departmental course module
page: http://www.eng.cam.ac.uk/teaching/courses/y4/4f10.html
Any queries, problems, or errors in the handouts, please contact me by email: bill.byrne@eng.cam.ac.uk
A related 4th year module is 4F13 Machine Learning
[
Handouts 
Examples Papers 
Online Material 
Further Reading ]
Handouts
The lecture notes should be available online just before the lectures.
 Handout 1: Introduction and Bayes' Decision Theory.
Statistical pattern processing, Bayesian decision theory, classification cost, ROC curves
Notes available in [pdf]
 Handout 2: Multivariate Gaussian and Decision Boundaries.
Multivariate Gaussian PDFs, maximum likelihood estimation, decision boundaries, generalisation
Notes available in [pdf]
 Handout 3: Gaussian Mixture Models.
Gaussian mixture models, parameter estimation, introduction to EM.
Notes available in [pdf]
 Handout 4: Expectation Maximisation.
Latent variables both continuous and discrete, proof of EM, factor analysis.
Notes available in [pdf]
 Handout 5: Linear Classifiers.
Single layer perceptron, perceptron learning algorithm, Fisher's linear discriminant analysis,
limitations.
Notes available in [pdf]
 Handout 6 and 7: MultiLayer Perceptrons.
Basic structure, posterior modelling, regression.
Notes available in [pdf] and [pdf]
 Handout 8: Support Vector Machines.
Maximum margin classifiers, handling nonseparable data, training SVMs, nonlinear
SVMs, kernel functions.
Notes available in [pdf]
 Handout 9: Gaussian Processes.
inear and basis function regression, Gaussian processes, covariance functions, classification
Notes available in [pdf]
Carl Rasmussen's notes available in [pdf]
 Handout 10: Relevance Vector Machines and Classification.
RVM weight prior, training using EM, classification.
2012 notes available in [pdf]
 Handout 11: Classification and Regression Trees.
Decision trees, query selection, multivariate decision trees.
2012 notes available in [pdf] (mortgage examples [ps])
 Handout 12: NonParametric Techniques.
Parzen windows, Knearest neighbours, nearest neighbour rule.
2012 notes available in [pdf]
 Handout 13: Speaker Verification and Identification.
GMMs and MAP estimation, SVMbased verification, dynamic kernels.
2012 notes available in [pdf]
Examples papers
The examples class for the first examples paper is planned for the end of week 4. For the second examples
paper the end of week 8.
 The first examples paper for 4F10 statistical pattern processing are
available in [pdf].
Solutions for examples paper 1 are
available in [pdf].
 The second examples paper for 4F10 statistical pattern processing are
available in [pdf].
Solutions for examples paper 2 are.
available in [pdf].
Online material
WARNING: I have no control over any website beyond the engineering department. The links
provided were valid when I checked them. If a link has changed, or the contents are
impropriate please email me ASAP.
Recommended reading

Judea Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible
Inference, Morgan Kaufmann, San Mateo, CA, 1997. ISBN 1558604790.

(*) Richard Duda, Peter Hart and David Stork: Pattern Classification, Second Edition,
John Wiley & Sons Inc, 2000. ISBN 0471056693

(*) Christopher Bishop, Neural Networks for Pattern Recognition, Clarendon Press, 1995.
ISBN 0198538642

(*) Christopher Bishop, Pattern Recognition and Machine Learning, Springer 2006.
top
[ Cambridge University 
CUED 
MIL 
Home]
