Skip to main content
SpringerLink
Log in

Training Algorithm with Incomplete Data for Feed-Forward Neural Networks

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

A new algorithm is developed to train feed-forward neural networks for non-linear input-to-output mappings with small incomplete data in arbitrary distributions. The developed Training-EStimation-Training (TEST) algorithm consists of 3 steps, i.e., (1) training with the complete portion of the training data set, (2) estimation of the missing attributes with the trained neural networks, and (3) re-training the neural networks with the whole data set. Error back propagation is still applicable to estimate the missing attributes. Unlike other training methods with missing data, it does not assume data distribution models which may not be appropriate for small training data. The developed TEST algorithm is first tested for the Iris benchmark data. By randomly removing some attributes from the complete data set and estimating the values latter, accuracy of the TEST algorithm is demonstrated. Then it is applied to the Diabetes benchmark data, of which about 50% contains missing attributes. Compared with other existing algorithms, the proposed TEST algorithm results in much better recognition accuracy for test data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. LeCun, Y., Denker, J. S. and Solla, S. A.: Optimal brain damage, In: D. Touretzky (ed.), Advances in Neural Information Processing Systems 2, Morgan Kaufmann, 1990, pp. 598–605.

  2. Weigend, A. S., Rumelhart, D. E. and Huberman, B. A.: Generalization by weight elimination with application to forecasting, In: R. Lippmann, J. Moody, and D. Touretzky (eds.), Advances in Neural Information Processing Systems 3, San Mateo, CA, 1991, pp. 875–882.

  3. Jeong, D. G. and Lee, S. Y.: Merging Hebbian and error back-propagation learning rules for robust classifications, Neural Networks 9 (1996), 1213–1222.

    Google Scholar 

  4. Timm, H. and Klawonn, K.: Classification of data with missing data, Proc. 6th European Congress on Intelligent Techniques and Soft Computing 1 (September 7, 1998), 639–643, Aachen, Germany.

    Google Scholar 

  5. Little, R. J. A.: Regression with missing X's: a review, Journal of the American Statistical Association 87 (1992), 1227–1237.

    Google Scholar 

  6. Beale, E. M. L. and Little, R. J. A.: Missing values in multivariate analysis, Journal of Royal Statistical Society, B37 (1975), 129–145.

    Google Scholar 

  7. Dempster, A. P., Laird, N. M. and Rubin, D. B.: Maximum likelihood from incomplete data via the EM algorithm, Journal of Royal Statistical Society B39 (1977), 1–38.

    Google Scholar 

  8. Chen, C. F.: A Bayesian approach to nested missing-data problems, In: P. Goel and A. Zellner (eds.), Bayesian Inference and Decision Techniques, P. Goel and A. Zellner (eds.), Elsevier Press, New York, 1986, pp. 355–361.

    Google Scholar 

  9. Heitjan, D. F. and Little, R. J. A.: Multiple imputation in the fatal accident reporting system, Applied Statistics 40 (1991), 13–29.

    Google Scholar 

  10. Lange, K., Little, R. J. A. and Taylor, J. M. G.: Robust statistical inference using the t distribution, Journal of the American Statistical Association 84 (1989), 881–896.

    Google Scholar 

  11. Tresp, V., Ahmad, S. and Neuneier, R.: Training neural networks with deficient data, In: J. Cowan, G. Tesauro and J. Alspector (eds.), Advances in Neural Information Processing Systems 6 (1994), Morgan Kaufmann, pp. 128–135.

  12. Jordan, M. I. and Jacobs, R. A.: Hierarchical mixtures of experts and the EM algorithm, Neural Computation 6 (1994), 181–214.

    Google Scholar 

  13. Amari, S.: Information geometry of the EM and em algorithms for neural networks, Neural Networks 8 (1995), 1379–1408.

    Google Scholar 

  14. Ghahramani, Z. and Jordan, M.: Supervised learning from incomplete data via an EMapproach, In: J. Cowan, G. Tesauro, and J. Alspector (eds.), Advances in Neural Information Processing Systems 6 (1994), Morgan Kaufmann, pp. 120–127.

  15. Holmström, L. and Koistinen, P.: Using additive noise in backpropagation training, IEEE Trans. Neural Networks, 3 (1992), 24–38.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computation and Neural Systems Laboratory, Department of Electrical Engineering, Korea Advanced Institute of Science and Technology, 373–1 Kusong-dong, Yusong-gu, Taejon, 305–701, Korea, e-mail

    Song-Yee Yoon & Soo-Young Lee

Authors
  1. Song-Yee Yoon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Soo-Young Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yoon, SY., Lee, SY. Training Algorithm with Incomplete Data for Feed-Forward Neural Networks. Neural Processing Letters 10, 171–179 (1999). https://doi.org/10.1023/A:1018772122605

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1018772122605

Search

Navigation