Skip to main content
SpringerLink
Log in

On connectionism and rule extraction

  • Conference paper
Neural Nets WIRN Vietri-01

Part of the book series: Perspectives in Neural Computing ((PERSPECT.NEURAL))

  • 775 Accesses

Abstract

There are two major motivations for rule extraction from trained artificial neural networks. First, some of the proposed neural network architectures, like multiplayer perceptrons, are so complex that that it is difficult to understand the logic behind any decision or inference made by such a network. So from an engineering standpoint, rule extraction from such a complex network provides a way to understand and explain the logic behind any decision made by it. By the way, [11] define the rule extraction from neural networks task as follows: “Given a trained neural network and the examples used to train it, produce a concise and accurate symbolic description of the network.” So the objective of rule extraction is to provide a certain type of symbolic description of the network. A second major motivation for rule extraction is to bridge the divide between symbolic AI and connectionism; that is, to show that connectionist subsymbolic systems are just an implementation of higher-level symbolic systems. Thus rule-extraction and rule-insertion, whereby a connectionist network is created from a set of symbolic rules [16,17], provides a seamless integration between these two levels, the symbolic and the subsymbolic.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abe, S. and M.-S. Lan (1995). Fuzzy Rule Extraction Directly from Numerical Data for Function Approximation. IEEE Trans. Systems, Man & Cybernetics, 25:119–129.

    Article  MathSciNet  Google Scholar 

  2. Alexander, J. A. & Mozer, M. C. (1995). Template-based algorithms for connectionist rule extraction. In Tesauro, G., Touretzky, D., & Leen, T., editors, Advances in Neural Information Processing Systems (volume 7). MIT Press.

    Google Scholar 

  3. Andrews, R., Diederich, J. and Tickle, A. B. (1995). A survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-based Systems, 8, 6, 373–389.

    Article  Google Scholar 

  4. Andrews, R. and J. Diederich, eds, (1996). Proceedings of the NIPS’96 Workshop on Rule Extraction From Trained Artificial Neural Networks. NIPS Foundation.

    Google Scholar 

  5. Apolloni, B., Malchiodi, D., Orovas, C., and Palmas, G. (2000a). Learning rule representations from data. Under review.

    Google Scholar 

  6. Apolloni, B., Malchiodi, D., Orovas, C., and Palmas, G. (2000b). From synapses to rules. In Foundations of Connectionist-symbolic Integration: Representation, Paradigms, and Algorithms — Proceedings of the 14th European Conference on Artificial Intelligence.

    Google Scholar 

  7. Boden, M. (1994). Horses of a Different Color? In: Honavar, V. and Uhr, L. (Ed.) Artificial Intelligence and Neural Networks: Steps Toward Principled Integration. New York: Academic Press.

    Google Scholar 

  8. Buckley, J. J. and Hayashi, Y. (1994). Fuzzy neural networks. In: Yager, R. and Zadeh, L. (Ed.) Fuzzy Sets, Neural Networks and Soft Computing. New York: Van Nostrand Reinhold.

    Google Scholar 

  9. Carpenter, G.A. and Tan, A.-H. (1995). Rule extraction: From neural architecture to symbolic representation. Connection Science, 7, 3–27.

    Article  Google Scholar 

  10. Carpenter, G. A. and Grossberg, S. (1994). Fuzzy ARTMAP: A synthesis of neural networks and fuzzy logic for supervised categorization and nonstationary prediction. In: Yager, R. and Zadeh, L. (Ed.) Fuzzy Sets, Neural Networks and Soft Computing. New York: Van Nostrand Reinhold.

    Google Scholar 

  11. Craven, M. W. and Shavlik, J. W. (1994). Using sampling and queries to extract rules from trained neural networks. Machine Learning: Proceedings of the Eleventh International Conference, San Francisco, CA.

    Google Scholar 

  12. Fahlman, S. E. and Hinton, G. E. (1987). Connectionists Architectures for Artificial Intelligence. Computer, 20, 100–109.

    Article  Google Scholar 

  13. Feldman, J. A. and Ballard, D. A. (1982). Connectionists Models and Their Properties. Cognitive Science, 6, 205–254.

    Article  Google Scholar 

  14. Frasconi, P., Gori, M., Maggini, M., and Soda, G. (1995). Unified integration of explicit rules and learning by example in recurrent networks. IEEE Transactions on Knowledge and Data Engineering, vol. 7, no. 2, pp. 340–346.

    Article  Google Scholar 

  15. Fu, L. M. (1994). Rule generation from neural networks. IEEE Transactions on Systems, Man and Cybernetics, 24 (8), pp. 1114–1124.

    Article  Google Scholar 

  16. Giles, L. and C.W. Omlin (1993). Extraction, insertion and refinement of symbolic rules in dynamically-driven recurrent neural networks. Connection Science, 5(3,4):307–337, Special Issue on Architectures for Integrating Symbolic and Neural Processes.

    Article  Google Scholar 

  17. Giles, L. and Christian W. Omlin (1994). Extraction and insertion of symbolic information in recurrent neural networks. In V. Honavar and L. Uhr, editors, Artificial Intelligence and Neural Networks: Steps toward Principled Integration, pages 271–299. Academic Press.

    Google Scholar 

  18. Goonatilake, S. and S. Khebbal (Ed.), (1995), Intelligent Hybrid Systems. New York: Wiley.

    Google Scholar 

  19. Grossberg, S. (1982). Studies of Mind and Brain: Neural Principles of Learning Perception, Development, Cognition, and Motor Control. Boston: Reidell Press.

    MATH  Google Scholar 

  20. Grossberg, S. (1987). Competitive learning: From interactive activation to adaptive resonance. Cognitive Science, 11, 23–63.

    Article  Google Scholar 

  21. Grossberg, S. (1988). Nonlinear neural networks: principles, mechanisms, and architectures. Neural Networks, 1, 17–61.

    Article  Google Scholar 

  22. Gupta, M. M. and Rao, D. H. (1994). On the principles of fuzzy neural networks. Fuzzy Sets and Systems, 61, 1, 1–18.

    Article  MathSciNet  Google Scholar 

  23. Haugeland, J. (1996). What is Mind Design. Chapter 1 in Haugeland, J. (ed), Mind Design II, 1997, MIT Press, 1–28.

    Google Scholar 

  24. Honavar, V. and Uhr, L. (Ed.), (1994), Artificial Intelligence and Neural Networks: Steps Toward Principled Integration. New York, NY: Academic Press.

    Google Scholar 

  25. Honavar, V. and Uhr, L. (1995). Integrating Symbol Processing Systems and Connectionist Networks. In: Goonatilake, S. and Khebbal, S. (Ed.) Intelligent Hybrid Systems. New York: Wiley.

    Google Scholar 

  26. Horwitz, B., Friston, K. J., and Taylor, J. G. (2000). Neural modeling and functional brain imaging: an overview. Neural Networks, Vol. 13, No. 8–9, 829–846.

    Article  Google Scholar 

  27. Keller, J. M. and Hunt, D. (1985). Incorporating fuzzy membership functions into the perceptron algorithm. IEEE Trans. Pattern Anal. Machine Intelligence, 7, 693–699.

    Article  Google Scholar 

  28. Kohonen, T. (1988). An introduction to neural networks. Neural Networks, 1, 3–16.

    Article  Google Scholar 

  29. Kohonen, T. (1989). Self-organization and associative memory. 3rd ed. Berlin, Heidelberg: Spriger-Verlag.

    Google Scholar 

  30. Kosko, B. (1992). Neural Networks and Fuzzy Systems. Prentice Hall, Englewood Cliffs, NJ.

    MATH  Google Scholar 

  31. Levine, D. and Apariciov, M.. (Ed.) (1994). Neural Networks for Knowledge Representation. New York: Lawrence Erlbaum.

    Google Scholar 

  32. Lin, C. T. and Lee, C. S. G. (1994). Supervised and unsupervised learning with fuzzy similarity for neural network-based fuzzy logic control systems. In: Yager, R. and Zadeh, L. (Ed.) Fuzzy Sets, Neural Networks and Soft Computing. New York: Van Nostrand Reinhold.

    Google Scholar 

  33. McClelland, J. L. (1985). Putting knowledge in its place: A scheme for programming parallel processing structures on the fly. Cognitive Science,9,1130–146.

    Article  Google Scholar 

  34. McClelland, J.L., McNaughton, B.L., and O’Reilly, R.C. (1995). Why there are complementary learning systems in hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review. 102: 419–457.

    Article  Google Scholar 

  35. Moody, J. & Darken, C. (1989). Fast Learning in Networks of Locally-Tuned Processing Units, Neural Computation, 1(2), 281–294.

    Article  Google Scholar 

  36. Pal, S. K. & Mitra, S. (1992). Multi-layer perceptrons, fuzzy sets and classification, IEEE Transactions on Neural Networks, NN-3, 683–697.

    Article  Google Scholar 

  37. Reilly, D.L., Cooper, L.N. and Elbaum, C. (1982). A Neural Model for Category Learning. Biological Cybernetics, 45, 35–41.

    Article  Google Scholar 

  38. Roy, A. (2000). On Connectionism, Rule Extraction and Brain-like Learning. IEEE Transactions on Fuzzy Systems, Vol. 8, No. 2, pp. 222–227.

    Article  Google Scholar 

  39. Roy, A., Kim, L.S. & Mukhopadhyay, S. (1993). A Polynomial Time Algorithm for the Construction and Training of a Class of Multilayer Perceptrons. Neural Networks, Vol. 6, No. 4, pp. 535–545.

    Article  Google Scholar 

  40. Roy, A., Govil, S. & Miranda, R. (1995). An Algorithm to Generate Radial Basis Function (RBF)-like Nets for Classification Problems. Neural Networks, Vol. 8, No. 2, pp. 179–202.

    Article  Google Scholar 

  41. Roy, A., Govil, S. & Miranda, R. (1997a). A Neural Network Learning Theory and a polynomial Time RBF Algorithm. IEEE Transactions on Neural Networks,8, 6, pp. 1301–1313.

    Article  Google Scholar 

  42. Roy, A. & Mukhopadhyay, S. (1997b). Iterative Generation of Higher-Order Nets in Polynomial Time Using Linear Programming. IEEE Transactions on Neural Networks, 8, 2,402–412.

    Article  Google Scholar 

  43. Roy, A. (1999). Brain’s internal mechanisms — a new paradigm. Proceedings of the International Joint Conference on Neural Networks (IJCNN’99), Washington, D.C., paper no. 259.

    Google Scholar 

  44. Rumelhart, D.E., and McClelland, J.L. (eds.)(1986). Parallel Distributed Processing: Explorations in Microstructure of Cognition, Vol. 1: Foundations. MIT Press, Cambridge, MA., 318–362.

    Google Scholar 

  45. Rumelhart, D.E. (1989). The Architecture of Mind: A Connectionist Approach. Chapter 8 in Haugeland, J. (ed). Mind Design II, 1997, MIT Press, 205–232.

    Google Scholar 

  46. Shadmehr, R. and Holcomb, H. (1997). Neural correlates of motor memory consolidation. Science, Vol. 277, pp. 821–825.

    Article  Google Scholar 

  47. Setiono, R. and Liu, H. (1996). Symbolic Representation of Neural Networks. IEEE Computer, 29(3), pp. 71–76.

    Google Scholar 

  48. Setiono, R. and Liu, H. (1997). NeuroLinear: From neural networks to oblique decision rules. Neurocomputing, 17, pp. 1–24.

    Article  Google Scholar 

  49. Simpson, P. K. (1992). Fuzzy min-max neural networks — part I: Classification. IEEE Trans. On Neural Networks, 3, 5, 776–786.

    Article  Google Scholar 

  50. Smolensky, P. (1989). Connectionist Modeling: Neural Computation/Mental Connections. Chapter 9 in Haugeland, J. (ed), Mind Design II, 1997, MIT Press, 233–250.

    Google Scholar 

  51. Sun, R. (1994). Logic and Variables in Connectionist Networks: A Brief Overview. In: Honavar, V. and Uhr, L. (Ed.) Artificial Intelligence and Neural Networks: Steps Toward Principled Integration. New York: Academic Press.

    Google Scholar 

  52. Sun, R. (1994). Integrating Rules and Connectionism for Robust Commonsense Reasoning. John Wiley and Sons, New York, NY.

    MATH  Google Scholar 

  53. Sun, R. and Bookman, L. (Ed.) (1995), Computational Architectures Integrating Symbolic and Neural Processes. New York: Kluwer.

    MATH  Google Scholar 

  54. Takagi, H. & Hayasi, A. (1991). NN-driven fuzzy reasoning. Int. J. Approximate Reasoning. 5, 191–212.

    Article  MATH  Google Scholar 

  55. Towell, G. and Shavlik, J. (1993). The extraction of refined rules from knowledge-based neural networks. Machine Learning, 13, 1, 71–101.

    Google Scholar 

  56. Yuan, F., Feldkamp, L.A., Davis, L.I., & Puskorius, G.V. (1992). Training a hybrid neural-fuzzy system. Proceedings of IJCNN’92, Baltimore, Vol. II, pp. 739–744.

    Google Scholar 

  57. Yager, R. R. (1993). Generalized fuzzy and matrix associative holographic memories. J. of Int. and Fuzzy Systems, 1, 1, 43–53.

    Google Scholar 

  58. Zadeh, L. A. (1965). Fuzzy Sets. Information and Control, 8, 338–353.

    Article  MathSciNet  MATH  Google Scholar 

  59. Zadeh, L. A. (1973). Outline of a new approach to the analysis of complex systems and decision process. IEEE Trans. Systems, Man and Cybernetics, 3, 1, 28–44.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information Systems, Arizona State University, 85287-3606, Tempe, AZ, USA

    Asim Roy

Authors
  1. Asim Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. DMI, Università di Salerno, 84081, Baronissi (SA), Italy

    Roberto Tagliaferri (Associate Professor of Computer Science and Neural Nets DMI) (Associate Professor of Computer Science and Neural Nets DMI)

  2. Dipartimento di Scienze Fisiche „E.R. Caianiello“, Università di Salerno, 84081, Baronissi (SA), Italy

    Maria Marinaro (Full Professor in Theoretical Physics) (Full Professor in Theoretical Physics)

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag London Limited

About this paper

Cite this paper

Roy, A. (2002). On connectionism and rule extraction. In: Tagliaferri, R., Marinaro, M. (eds) Neural Nets WIRN Vietri-01. Perspectives in Neural Computing. Springer, London. https://doi.org/10.1007/978-1-4471-0219-9_31

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-0219-9_31

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-85233-505-2

  • Online ISBN: 978-1-4471-0219-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation