Unsupervised learning of distributions

Universität Augsburg, Memminger Str. 6, 86135 Augsburg, Germany

Received: 23 May 1997
Accepted: 16 September 1997

Abstract

We study unsupervised learning from non-uniformly distributed examples with a single symmetry-breaking orientation when both the distribution and the preferential direction are otherwise completely unknown. For asymptotically high dimensions N of the pattern space the distribution can be inferred exactly from p=O(N) examples up to a well-known remaining uncertainty in the preferential direction. We further discuss implications for supervised learning of a teacher perceptron with unknown transfer function, unsupervised learning with several preferential directions, and architecture optimization.