| Issue |
Europhys. Lett.
Volume 34, Number 2, April II 1996
|
|
|---|---|---|
| Page(s) | 151 - 156 | |
| Section | Cross-disciplinary physics and related areas of science and technology | |
| DOI | https://doi.org/10.1209/epl/i1996-00431-5 | |
| Published online | 01 September 2002 | |
Finite-size effects in on-line learning of multilayer neural networks
1
Department of Computer Science and Applied Mathematics, Aston University, Birmingham
B4 7ET, UK
2
Department of Physics, University of Edinburgh, Kings Buildings,
Edinburgh EH9 3JZ, UK
Received:
7
July
1995
Accepted:
26
February
1996
We complement recent advances in thermodynamic limit analyses of mean on-line gradient descent learning dynamics in multilayer networks by calculating fluctuations possessed by finite-dimensional systems. Fluctuations from the mean dynamics are largest at the onset of specialisation as student hidden unit weight vectors begin to imitate specific teacher vectors, increasing with the degree of symmetry of the initial conditions. In light of this, we include a term to stimulate asymmetry in the learning process, which typically also leads to a significant decrease in training time.
PACS: 87.10.+e – General, theoretical, and mathematical biophysics (including logic of biosystems, quantum biology, and relevant aspects of thermodynamics, information theory, cybernetics, and bionics) / 02.50.-r – Probability theory, stochastic processes, and statistics
© EDP Sciences, 1996
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.