Statistical mechanics of mutual information maximization
Neural Computing Research Group, Aston University Aston Triangle, Birmingham B4 7ET,
Accepted: 14 December 1999
An unsupervised learning procedure based on maximizing the mutual information between the outputs of two networks receiving different but statistically dependent inputs is analyzed (Becker S. and Hinton G., Nature, 355 (1992) 161). By exploiting a formal analogy to supervised learning in parity machines, the theory of zero-temperature Gibbs learning for the unsupervised procedure is presented for the case that the networks are perceptrons and for the case of fully connected committees.
PACS: 87.10.+e – General theory and mathematical aspects / 05.50.+q – Lattice theory and statistics (Ising, Potts, etc.) / 64.60.Cn – Order-disorder transformations; statistical mechanics of model systems
© EDP Sciences, 2000