Volume 38, Number 6, May III 1997
|Page(s)||477 - 482|
|Section||Cross-disciplinary physics and related areas of science and technology|
|Published online||01 September 2002|
On-line learning from finite training sets
Department of Physics, University of Edinburgh - Edinburgh EH9 3JZ, UK
2 Neural Computing Research Group, Aston University - Birmingham B4 7ET, UK
Accepted: 3 April 1997
We analyse on-line (gradient descent) learning of a rule from a finite set of training examples at non-infinitesimal learning rates η, calculating exactly the time-dependent generalization error for a simple model scenario. In the thermodynamic limit, we close the dynamical equation for the generating function of an infinite hierarchy of order parameters using “within-sample self-averaging”. The resulting dynamics is non-perturbative in η, with a slow mode appearing only above a finite threshold . Optimal settings of η for given final learning time are determined and the results are compared with offline gradient descent.
PACS: 87.10.+e – General, theoretical, and mathematical biophysics (including logic of biosystems, quantum biology, and relevant aspects of thermodynamics, information theory, cybernetics, and bionics) / 02.50.-r – Probability theory, stochastic processes, and statistics / 05.90.+m – Other topics in statistical physics and thermodynamics
© EDP Sciences, 1997
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.