# Generalization and Maximum Likelihood from Small Data Sets

**Download:**
PDF.

“Generalization and Maximum Likelihood from Small Data Sets”
by W. Byrne.
In *IEEE-SP Workshop on Neural Networks in Signal Processing*, 1993.

## Abstract

An often encountered learning problem is maximum likelihood training of
exponential models. When the state is only partially specified by the
training data, iterative training algorithms are used to produce a
sequence of models that assign increasing likelihood to the training data.
Although the performance as measured on the training set continues to
improve as the algorithms progress, performance on related data sets may
eventually begin to deteriorate. The cause of this behavior can be seen
when the training problem is stated in the Alternating Minimization
framework. A modified maximum likelihood training criterion is suggested
to counter this behavior. It leads to a simple modification of the
learning algorithms which relates generalization to learning speed.
Training Boltzmann Machines and Hidden Markov Models is discussed under
this modified criterion.

**Download:**
PDF.

**BibTeX entry:**

@inproceedings{byrne93:_gener_,
author = {W. Byrne},
title = {Generalization and Maximum Likelihood from Small Data Sets},
booktitle = {{IEEE-SP} Workshop on Neural Networks in Signal Processing},
pages = {(7 pages)},
year = {1993}
}

Back to Bill Byrne publications.