Abstract
This contribution demonstrates the possibility to achieve a Bayesian (or nearly Bayesian) classification of exponentially distributed data by perceptrons with at most two hidden layers. The number of hidden layers depends on how much is known about the sufficient statistics figuring in the corresponding exponential distributions. A practical applicability is illustrated by classification of normally distributed data. Experiments with such data proved that, in the learning based on correct classification information, the error backpropagation rule is able to create in the hidden layers surprisingly good approximations of apriori unknown sufficient statistics. This enables the trained network to imitate Bayesian classifiers and to achieve minimum classification errors.
Supported by the GA AV CR grant 2075703.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
BROWN, L. D. (1986): Fundamentals of Statistical Exponential Families. Lecture Notes vol. 9. Inst. of Mathem. Statist., Hayward, California.
CYBENKO, G. (1989): Approximation by Superpositions of a Sigmoidal Functions. Mathematics of Control, Signals and Systems, 2, 303–314.
DECLERIS, N. (1991): On the Case of Higher Order Nonlinear Neural-Type Functions in Adaptive Systems. N. Decleris (Ed.), System Science Manual, 129–136. European System Union, Athens.
DEVIJVER, P. and KITTLER, J. (1982): Pattern Recognition: A Statistical Approach. Englewood Cliffs, Prentice Hall.
FUNAHASHI, K. (1989): On the Approximate Realization of Continuous Mappings by Neural Networks. Neural Networks, 2, 183–192.
FUNAHASHI, K. (1998): Multilayer networks and Bayes decision theory. Neural Networks, 11, 209–213.
HAND, D. J. (1981): Discrimination and Clasification. New York, Wiley.
HORNIK, K., STINCHCOMBE, B. and WHITE, H. (1989): Multilayer Feedforward Networks and Universal Approximation. Neural Networks, 2, 359–366.
KÜCHLER, U. and SØRENSEN, M. (1989): Exponential Families of Stochastic Processes: A Unifying Semimartingale Approach. Int. Statist. Review 57, 123–144.
LAPEDES, A. S., and FARBER, R. H. (1988): How Neural Networks Work. Y. S. Lee (Ed.), Evolution, Learning and Cognition, 331–340. World Scientific, Singapore.
MÜLLER, B., REINHARD, J. and STRICKLAND, M. T. (1995): Neural Networks. 2nd edition. Springer, Berlin.
MORGAN, D.P. and SCOFIELD, C. L. (1994): Neural Networks and Speech Processing. Boston, Kluwer.
RUCK, D. W., ROGERS, S.K., KABRISKY, M., OXLEY M.E. and SUTER, B. W. (1990): The multilayer perceptron as an approximation to a Bayes optimal discriminant function. Transactions of IEEE on Neural Networks, 1, 296–298.
SPECHT, D. F. (1991): A General Regression Neural Network. Transactions of IEEE on Neural Networks, 2, 568–576.
VAJDA, I. (1996): About Perceptron Realization of Bayesian Decisions. IEEE International Conference on Neural Networks, vol. 1, 253–257.
VAJDA, I. and VESELÝ, A. (1997): Perceptron Approximations to Bayesian Discrimination and Classification of Random Signals. Neural Network World 3, 305–323.
VAJDA, I., VESELÝ, A., LONEK, B. and NIKOLOV, V. (1998): Neural Network Realizations of Bayes Decision Rules for Exponentially Distributed Data. Kybernetika (in print).
WHITE, H. (1990): Connectionist Nonparametric Regression: Multilayer Feed-forward Network can Learn Arbitrary Mappings. Neural Networks, 3, 535–549.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin · Heidelberg
About this paper
Cite this paper
Vajda, I. (1999). Neural Network Classification in Exponential Models with Unknown Statistics. In: Gaul, W., Locarek-Junge, H. (eds) Classification in the Information Age. Studies in Classification, Data Analysis, and Knowledge Organization. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-60187-3_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-60187-3_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65855-9
Online ISBN: 978-3-642-60187-3
eBook Packages: Springer Book Archive