On-line learning from a finite training set: A solvable model
Institut für Theoretische Physik,
D-97074 Würzburg, Germany
2 Neural Computing Research Group, Aston University - B4 7ET Birmingham, UK
Accepted: 12 November 1999
We discuss the problem of on-line learning from a finite training set with feedforward neural networks. Defining a modified learning rule, which randomly chooses inputs and weights to be updated, the dynamics of learning can be treated within a diffusion approximation in the thermodynamic limit. No assumption on the generation of data is made. Explicit results for the stationary distribution and relaxation times can be found for a network with linear transfer function. Assuming self-averaging of the diffusion term, a general relation between on-line learning and batch learning with an effective temperature is established.
PACS: 02.50.-r – Probability theory, stochastic processes, and statistics / 07.05.Mh – Neural networks, fuzzy logic, artificial intelligence / 05.20.-y – Classical statistical mechanics
© EDP Sciences, 2000