Learning in a large committee machine: worst case and average case
Institut für theoretische Physik, Universität Würzburg Am Hubland,
D-97074 Würzburg, Germany
Accepted: 5 July 1996
Learning of realizable rules is studied for tree committee machines with continuous weights. No nontrivial upper bound exists for the generalization error of consistent students as the number of hidden units K increases. However, numerical considerations show that consistent students with a value of the generalization error significantly higher than predicted by the average-case analysis are extremely hard to find. An on-line learning algorithm is presented, for which the generalization error scales with the training set size as in the average-case theory in the limit of large K.
PACS: 87.10.+e – General, theoretical, and mathematical biophysics (including logic of biosystems, quantum biology, and relevant aspects of thermodynamics, information theory, cybernetics, and bionics) / 05.50.+q – Lattice theory and statistics; Ising problems / 64.60.Cn – Order-disorder and statistical mechanics of model systems
© EDP Sciences, 1996