*Europhys. Lett.*,

**49**(3), pp. 275-281 (2000)

## On-line learning from a finite training set: A solvable model

^{1}
Institut für Theoretische Physik,
Julius-Maximilians-Universität
Würzburg
D-97074 Würzburg, Germany

^{2}
Neural Computing Research Group, Aston University - B4 7ET Birmingham, UK

Received:
24
June
1999

Accepted:
12
November
1999

We discuss the problem of on-line learning from a finite training set with feedforward neural networks. Defining a modified learning rule, which randomly chooses inputs and weights to be updated, the dynamics of learning can be treated within a diffusion approximation in the thermodynamic limit. No assumption on the generation of data is made. Explicit results for the stationary distribution and relaxation times can be found for a network with linear transfer function. Assuming self-averaging of the diffusion term, a general relation between on-line learning and batch learning with an effective temperature is established.

PACS: 02.50.-r – Probability theory, stochastic processes, and statistics / 07.05.Mh – Neural networks, fuzzy logic, artificial intelligence / 05.20.-y – Classical statistical mechanics

*© EDP Sciences, 2000*