Skip to main content
SpringerLink
Log in

A systematic review and taxonomy of explanations in decision support and recommender systems

  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

Abstract

With the recent advances in the field of artificial intelligence, an increasing number of decision-making tasks are delegated to software systems. A key requirement for the success and adoption of such systems is that users must trust system choices or even fully automated decisions. To achieve this, explanation facilities have been widely investigated as a means of establishing trust in these systems since the early years of expert systems. With today’s increasingly sophisticated machine learning algorithms, new challenges in the context of explanations, accountability, and trust towards such systems constantly arise. In this work, we systematically review the literature on explanations in advice-giving systems. This is a family of systems that includes recommender systems, which is one of the most successful classes of advice-giving software in practice. We investigate the purposes of explanations as well as how they are generated, presented to users, and evaluated. As a result, we derive a novel comprehensive taxonomy of aspects to be considered when designing explanation facilities for current and future decision support systems. The taxonomy includes a variety of different facets, such as explanation objective, responsiveness, content and presentation. Moreover, we identified several challenges that remain unaddressed so far, for example related to fine-grained issues associated with the presentation of explanations and how explanation facilities are evaluated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. In general, advice-giving system is a term that can also include other types of systems, such as conversational agents or autonomous systems. In this paper, however, we use the term to refer only to the four types of systems listed above.

  2. There is a large number of surveys that provide an overview of explanations published elsewhere (Tintarev and Masthoff 2007b, 2011; Swartout and Moore 1993; Chandrasekaran et al. 1989; Dhaliwal and Benbasat 1996; Gregor and Benbasat 1999; Moulin et al. 2002; Lacave and Díez 2002, 2004; Sørmo et al. 2005; Nakatsu 2006; Papadimitriou et al. 2012; Scheel et al. 2014).

  3. http://www.periodicos.capes.gov.br/.

  4. Further details about which inclusion and exclusion criterion was fulfilled by each individual paper, along with the detailed results of our analysis, can be found online at: http://inf.ufrgs.br/prosoft/resources/2017/umuai-sr-explanations.

  5. The interested reader can find the detailed analysis of each of the studies on the following link: http://inf.ufrgs.br/prosoft/resources/sr-explanations.

  6. Note that more papers were published in computer science in general over time.

  7. See http://www.fatml.org for a recent workshop series on fair, accountable, and transparent machine learning approaches.

  8. We remind the reader that approaches that simply provide a rule extraction algorithm without detailing how the explanations are provided to the end user are excluded from our review as specified in our inclusion and exclusion criteria.

  9. One of the examined studies (Roitman et al. 2010) focused on safety in the context of decisions with critical consequences. We included it in the category effectiveness.

  10. Bilgic and Mooney’s work is not included in this review because it was published in a workshop and not part of the databases searched in our review. We discuss the choice of databases and possible research limitations later in the paper.

  11. Some authors measured the accuracy of the decisions or ratings of the study participants. Such a measurement is categorised as investigating the effectiveness of the explanations.

  12. Higher deltas mean that users tend to overestimate or underestimate suggested alternatives based on explanations.

  13. http://www.eugdpr.org/.

  14. http://www.aaai.org/Library/library.php.

References

  • Abu-Hakima, S., Oppacher, F.: Rationale: reasoning by explaining. In: Proceedings of the Fourth International Conference on Data Engineering, pp. 258–265 (1988)

  • Allgayer, J., Harbusch, K., Kobsa, A., Reddig, C., Reithinger, N., Schmauks, D.: XTRA: a natural-language access system to expert systems. Int. J. Man Mach. Stud. 31(2), 161–195 (1989)

    Article  Google Scholar 

  • Amer-Yahia, S., Galland, A., Stoyanovich, J., Yu, C.: From Del.Icio.Us to x.Qui.Site: recommendations in social tagging sites. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, SIGMOD ’08, pp. 1323–1326 (2008)

  • Artioli, E., Avanzolini, G., Martelli, L., Ursino, M.: An expert system based on causal knowledge: validation on post-cardiosurgical patients. Int. J. Bio Med. Comput. 41(1), 19–37 (1996)

    Article  Google Scholar 

  • Bader, R., Woerndl, W., Karitnig, A., Leitner, G.: Designing an Explanation Interface for Proactive Recommendations in Automotive Scenarios, pp. 92–104. Springer, Berlin (2012)

    Google Scholar 

  • Balleda, K., Satyanvesh, D., Sampath, N.V.S.S.P., Varma, K.T.N., Baruah, P.K.: Agpest: an efficient rule-based expert system to prevent pest diseases of rice amp; wheat crops. In: 2014 IEEE 8th International Conference on Intelligent Systems and Control (ISCO), pp. 262–268 (2014)

  • Banavar, G.: Learning to Trust Artificial Intelligence Systems: Accountability, Compliance and Ethics in the Age of Smart Machines. White paper, IBM Global Services (2016)

  • Barbieri, N., Bonchi, F., Manco, G.: Who to follow and why: Link prediction with explanations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, pp. 1266–1275 (2014)

  • Basu, A., Dutta, A.: Computer based support of reasoning in the presence of fuzziness. Decis. Support Syst. 2(3), 235–256 (1986)

    Article  Google Scholar 

  • Basu, A., Ahad, R.: Using a relational database to support explanation in a knowledge-based system. IEEE Trans. Knowl. Data Eng. 4(6), 572–581 (1992)

    Article  Google Scholar 

  • Basu, A., Majumdar, A.K., Sinha, S.: An expert system approach to control system design and analysis. IEEE Trans. Syst. Man Cybern. 18(5), 685–694 (1988)

    Article  Google Scholar 

  • Bau, D.Y., Brezillon, P.J.: Model-based diagnosis of power-station control systems. IEEE Expert 7(1), 36–44 (1992)

    Article  Google Scholar 

  • Bavota, G., Gethers, M., Oliveto, R., Poshyvanyk, D., Lucia, Ad: Improving software modularization via automated analysis of latent topics and dependencies. ACM Trans. Softw. Eng. Methodol. 23(1), 4:1–4:33 (2014)

    Article  Google Scholar 

  • Bedi, P., Sharma, R.: Trust based recommender system using ant colony for trust computation. Expert Syst. Appl. 39(1), 1183–1190 (2012)

    Article  Google Scholar 

  • Bedi, P., Agarwal, S.K., Sharma, S., Joshi, H.: Saprs: situation-aware proactive recommender system with explanations. In: 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 277–283 (2014)

  • Beiley, J., Duban, S.: Explanation and learning in medicine. In: Kibby, M. (ed.) Computer Assisted Learning, pp. 91–97. Pergamon, Amsterdam (1990)

    Chapter  Google Scholar 

  • Belahcene, K., Labreuche, C., Maudet, N., Mousseau, V., Ouerdane, W.: Explaining robust additive utility models by sequences of preference swaps. Theory Decis. 82(2), 151–183 (2017)

    Article  MATH  MathSciNet  Google Scholar 

  • Benaroch, M.: Roles of design knowledge in knowledge-based systems. Int. J. Hum. Comput. Stud. 44(5), 689–721 (1996)

    Article  Google Scholar 

  • Bielza, C., Gómez, M., Ríos-Insua, S., Fernándezdel Pozo, J.A.: Structural, elicitation and computational issues faced when solving complex decision making problems with influence diagrams. Comput. Oper. Res. 27(78), 725–740 (2000)

    Article  MATH  Google Scholar 

  • Bilgic, M., Mooney, R.J.: Explaining recommendations: satisfaction vs. promotion. In: Proceedings of Beyond Personalization 2005: A Workshop on the Next Stage of Recommender Systems Research at the 2005 International Conference on Intelligent User Interfaces. San Diego, CA (2005)

  • Blake, J.N., Kerr, D.V., Gammack, J.G.: Streamlining patient consultations for sleep disorders with a knowledge-based CDSS. Inf. Syst. 56, 109–119 (2016)

    Article  Google Scholar 

  • Blanco, R., Ceccarelli, D., Lucchese, C., Perego, R., Silvestri, F.: You should read this! let me explain you why: explaining news recommendations to users. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management, CIKM ’12, pp. 1995–1999 (2012)

  • Bofeng, Z., Na, W., Gengfeng, W., Sheng, L.: Research on a personalized expert system explanation method based on fuzzy user model. In: Fifth World Congress on Intelligent Control and Automation, vol. 5, pp. 3996–4000 (2004)

  • Bohanec, M., Zupan, B., Rajkovic̃, V.: Applications of qualitative multi-attribute decision models in health care. Int. J. Med. Inform. 5859, 191–205 (2000)

    Article  Google Scholar 

  • Bohnenberger, T., Jacobs, O., Jameson, A., Aslan, I.: Decision-Theoretic Planning Meets User Requirements: Enhancements and Studies of an Intelligent Shopping Guide, pp. 279–296. Springer, Berlin (2005)

    Google Scholar 

  • Borlea, l., Buta, A., Dusa, V., Lustrea, B.: DIASE—expert system fault diagnosis for Timisoara 220 kV substation. In: EUROCON 2005—The International Conference on “Computer as a Tool”, vol. 1, pp. 221–224 (2005)

  • Bosnić, Z., Vraćar, P., Radović, M.D., Devedžić, G., Filipović, N.D., Kononenko, I.: Mining data from hemodynamic simulations for generating prediction and explanation models. IEEE Trans. Inf. Technol. Biomed. 16(2), 248–254 (2012)

    Article  Google Scholar 

  • Bostandjiev, S., O’Donovan, J., Höllerer, T.: Tasteweights: a visual interactive hybrid recommender system. In: Proceedings of the Sixth ACM Conference on Recommender Systems, RecSys ’12, pp. 35–42 (2012)

  • Briguez, C.E., Budn, M.C., Deagustini, C.A., Maguitman, A.G., Capobianco, M., Simari, G.R.: Argument-based mixed recommenders and their application to movie suggestion. Expert Syst. Appl. 41(14), 6467–6482 (2014)

    Article  Google Scholar 

  • Buchanan, B.G., Shortliffe, E.H. (eds.): Explanations as a topic of AI research. In: Rule-Based Systems, pp. 331–337. Addison-Wesley, Massachusetts (1984)

  • Buchanan, B.G., Moore, J.D., Forsythe, D.E., Carenini, G., Ohlsson, S., Banks, G.: An intelligent interactive system for delivering individualized information to patients. Artif. Intell. Med. 7(2), 117–154 (1995)

    Article  Google Scholar 

  • Burattini, E., Gregorio, M.D., Tamburrini, G.: Hybrid expert systems: An approach to combining neural computation and rule-based reasoning. In: Leondes, C.T. (ed.) Expert Systems, pp. 1315–1354. Academic Press, Burlington (2002)

    Chapter  Google Scholar 

  • Buschner, S., Schirru, R., Zieschang, H., Junker, P.: Providing recommendations for horizontal career change. In: Proceedings of the 14th International Conference on Knowledge Technologies and Data-driven Business, i-KNOW ’14, pp. 33:1–33:4 (2014)

  • Bussone, A., Stumpf, S., O’Sullivan, D.: The role of explanations on trust and reliance in clinical decision support systems. In: 2015 International Conference on Healthcare Informatics, pp. 160–169 (2015)

  • Cagnoni, S., Coppini, G., Livi, R., Poli, R., Scarpelli, P.T., Valli, G.: A neural network expert system for computer-assisted analysis of blood-pressure data. In: Proceedings Computers in Cardiology, pp. 473–476 (1991)

  • Carenini, G., Moore, J.D.: An empirical study of the influence of user tailoring on evaluative argument effectiveness. In: Proceedings of the 17th International Joint Conference on Artificial Intelligence, IJCAI’01, pp. 1307–1312 (2001)

  • Carenini, G., Moore, J.D.: Generating and evaluating evaluative arguments. Artif. Intell. 170, 925–952 (2006)

    Article  Google Scholar 

  • Castro, C., Bose, A., Handschin, E., Hoffmann, W.: Comparison of different screening techniques for the contingency selection function. Int. J. Electr. Power Energy Syst. 18(7), 425–430 (1996)

    Article  Google Scholar 

  • Chandrasekaran, B., Mittal, S.: Deep versus compiled knowledge approaches to diagnostic problem-solving. Int. J. Hum. Comput. Stud. 51(2), 357–368 (1999)

    Article  Google Scholar 

  • Chandrasekaran, B., Tanner, M.C., Josephson, J.R.: Explaining control strategies in problem solving. IEEE Expert Intell. Syst. Appl. 4(1), 9-15–19-24 (1989)

    Google Scholar 

  • Chang, C.C., Hsieh, S.C.: Applying web service technology to build a wireless lan problem diagnosis expert system. In: 2010 International Conference on Computational Aspects of Social Networks, pp. 217–220 (2010)

  • Charissiadis, A., Karacapilidis, N.: Strengthening the Rationale of Recommendations Through a Hybrid Explanations Building Framework, pp. 311–323. Springer, Berlin (2015)

    Google Scholar 

  • Chelsom, J.J., Ellis, T.J., Carson, E.R., Cramp, D.G.: Blood gas analysis: a knowledge-based adviser for the interpretation of results. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 1360–1361 (1988)

  • Chen, L., Wang, F.: Sentiment-enhanced explanation of product recommendations. In: Proceedings of the 23rd International Conference on World Wide Web, WWW ’14 Companion, pp. 239–240 (2014)

  • Chen, W., Hsu, W., Lee, M.L.: Tagcloud-based explanation with feedback for recommender systems. In: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’13, pp. 945–948 (2013a)

  • Chen, Y.C., Lin, Y.S., Shen, Y.C., Lin, S.D.: A modified random walk framework for handling negative ratings and generating explanations. ACM Trans. Intell. Syst. Technol. 4(1), 12:1–12:21 (2013b)

    Article  Google Scholar 

  • Cheng, S.J., Chen, D.S., Peng, X.L.: An expert system for a thermal power station alarm processing. In: International Conference on Advances in Power System Control, Operation and Management, APSCOM-91, pp. 316–320 (1991)

  • Chiou, A., Yu, X.: Industrial decision support system (IDSS) in weed control and management strategies: expert advice using descriptive schemata and explanatory capabilities. In: IECON 2007—33rd Annual Conference of the IEEE Industrial Electronics Society, pp. 105–110 (2007)

  • Chouicha, M., Siller, T.: An expert system approach to liquefaction analysis part 1: development and implementation. Comput. Geotech. 16(1), 1–35 (1994)

    Article  Google Scholar 

  • Cleger-Tamayo, S., Fernandez-Luna, J.M., Huete, J.F.: Explaining neighborhood-based recommendations. In: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’12, pp. 1063–1064 (2012)

  • Davey-Wilson, I.: Development of a prolog-based expert system for groundwater control. Comput. Struct. 40(1), 185–189 (1991)

    Article  MATH  Google Scholar 

  • David, J.M., Krivine, J.P.: Designing knowledge-based systems within functional architecture: the DIVA experiment. In: Proceedings of the Fifth Conference on Artificial Intelligence Applications, pp. 173–180 (1989)

  • Davis, K.: DORIS (diagnostic oriented rockwell intelligent system). IEEE Aerosp. Electron. Syst. Mag. 1(7), 18–21 (1986)

    Article  Google Scholar 

  • de Braal, L., Ezquerra, N., Garcia, E., Cooke, C., Krawczynska, E.: PERFUSE: an interactive knowledge-based system for the interpretation and explanation of cardiac imagery. In: Proceedings of 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, vol. 3, pp. 1238–1239 (1996)

  • Deep, R., Czech, D.R., Dizek, S.G., Kennedy, D.K.: A bit-mapping classifier expert system in warranty selection. In: Proceedings of the IEEE 1988 National Aerospace and Electronics Conference, pp. 1222–1224 (1988)

  • Dhaliwal, J.S., Benbasat, I.: The use and effects of knowledge-based system explanations: theoretical foundations and a framework for empirical evaluation. Inf. Syst. Res. 7(3), 342–362 (1996)

    Article  Google Scholar 

  • Diederich, J.: Explanation and artificial neural networks. Int. J. Man Mach. Stud. 37(3), 335–355 (1992)

    Article  MathSciNet  Google Scholar 

  • Du, G., Ruhe, G.: Two machine-learning techniques for mining solutions of the releaseplanner decision support system. Inf. Sci. 259, 474–489 (2014)

    Article  Google Scholar 

  • Ehrlich, K., Kirk, S.E., Patterson, J., Rasmussen, J.C., Ross, S.I., Gruen, D.M.: Taking advice from intelligent systems: the double-edged sword of explanations. In: Proceedings of the 16th International Conference on Intelligent User Interfaces, IUI ’11, pp. 125–134 (2011)

  • Ezquerra, N., de Braal, L., Garcia, E., Cooke, C., Krawczynska, E.: Interactive, knowledge-guided visualization of 3D medical imagery. Future Gener. Comput. Syst. 15(1), 59–73 (1999)

    Article  Google Scholar 

  • Felfernig, A.: Koba4ms: selling complex products and services using knowledge-based recommender technologies. In: Seventh IEEE International Conference on E-Commerce Technology (CEC’05), pp. 92–100 (2005)

  • Felfernig, A., Gula, B.: An empirical study on consumer behavior in the interaction with knowledge-based recommender applications. In: The 8th IEEE International Conference on E-Commerce Technology and the 3rd IEEE International Conference on Enterprise Computing, E-Commerce, and E-Services (CEC/EEE’06), pp. 37–37 (2006)

  • Fong, J., Lam, H.P., Robinson, R., Indulska, J.: Defeasible preferences for intelligible pervasive applications to enhance eldercare. In: IEEE International Conference on Pervasive Computing and Communications Workshops, pp. 572–577 (2012)

  • Friedrich, G., Zanker, M.: A taxonomy for generating explanations in recommender systems. AI Mag. 32(3), 90–98 (2011)

    Article  Google Scholar 

  • Gallagher, S., Trainor, J., Murphy, M., Curran, E.: A knowledge based system for competitive bidding. In: Proceedings of the 1995 Second New Zealand International Two-Stream Conference on Artificial Neural Networks and Expert Systems, pp. 314–317 (1995)

  • Garca, A.J., Chesevar, C.I., Rotstein, N.D., Simari, G.R.: Formalizing dialectical explanation support for argument-based reasoning in knowledge-based systems. Expert Syst. Appl. 40(8), 3233–3247 (2013)

    Article  Google Scholar 

  • Gedikli, F., Ge, M., Jannach, D.: Understanding Recommendations by Reading the Clouds, pp. 196–208. Springer, Berlin (2011)

    Google Scholar 

  • Gedikli, F., Jannach, D., Ge, M.: How should i explain? A comparison of different explanation types for recommender systems. Int. J. Hum. Comput. Stud. 72(4), 367–382 (2014)

    Article  Google Scholar 

  • Giboney, J.S., Brown, S.A., Lowryc, P.B., Nunamaker Jr., J.F.: User acceptance of knowledge-based system recommendations: explanations, arguments, and fit. Decis. Support Syst. 72, 1–10 (2015)

    Article  Google Scholar 

  • Gkika, S., Lekakos, G.: Investigating the effectiveness of persuasion strategies on recommender systems. In: Proceedings of the 9th International Workshop on Semantic and Social Media Adaptation and Personalization, SMAP ’14, pp. 94–97. IEEE Computer Society, Washington, DC, USA (2014)

  • Glaser, B.G.: Basics of Grounded Theory Analysis: Emergence vs. Forcing. Sociology Pr, Mill Valley (1992)

    Google Scholar 

  • Gómez-Vallejo, H.J., Uriel-Latorre, B., Sande-Meijide, M., Villamarín-Bello, B., Pavón, R., Fdez-Riverol, F., Glez-Peña, D.: A case-based reasoning system for aiding detection and classification of nosocomial infections. Decis. Support Syst. 84, 104–116 (2016)

    Article  Google Scholar 

  • Gönül, M.S., Önkal, D., Lawrence, M.: The effects of structural characteristics of explanations on use of a dss. Decis. Support Syst. 42(3), 1481–1493 (2006)

    Article  Google Scholar 

  • Goud, R., Hasman, A., Peek, N.: Development of a guideline-based decision support system with explanation facilities for outpatient therapy. Comput. Methods Progr. Biomed 91(2), 145–153 (2008)

    Article  Google Scholar 

  • Gowri, K., Marsh, C., Bedard, C., Fazio, P.: Knowledge-based assistant for aluminum component design. Comput. Struct. 38(1), 9–20 (1991)

    Article  MATH  Google Scholar 

  • Grando, M.A., Moss, L., Glasspool, D., Sleeman, D., Sim, M., Gilhooly, C., Kinsella, J.: Argumentation-Logic for Explaining Anomalous Patient Responses to Treatments, pp. 35–44. Springer, Berlin (2011)

  • Gregor, S.: Explanations from knowledge-based systems and cooperative problem solving. Int. J. Hum. Comput. Stud. 54(1), 81–105 (2001)

    Article  MATH  Google Scholar 

  • Gregor, S., Benbasat, I.: Explanations from intelligent systems: theoretical foundations and implications for practice. MIS Q. 23(4), 497–530 (1999)

    Article  Google Scholar 

  • Grierson, D.E., Cameron, G.E.: A knowledge-based expert system for computer automated structural design. Comput. Struct. 30(3), 741–745 (1988)

    Article  Google Scholar 

  • Guida, G., Zanella, M.: Active operator support: a case study in steel production. In: IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century, vol. 4, pp. 3340–3345 (1995)

  • Guida, G., Mussio, P., Zanella, M.: User interaction in decision support systems: the role of justification. In: 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation, vol. 4, pp. 3215–3220 (1997)

  • Guy, I., Zwerdling, N., Carmel, D., Ronen, I., Uziel, E., Yogev, S., Ofek-Koifman, S.: Personalized recommendation of social software items based on social relations. In: Proceedings of the Third ACM Conference on Recommender Systems, RecSys ’09, pp. 53–60 (2009)

  • Guy, I., Zwerdling, N., Ronen, I., Carmel, D., Uziel, E.: Social media recommendation based on people and tags. In: Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’10, pp. 194–201 (2010)

  • Gvenir, H., Emeksiz, N.: An expert system for the differential diagnosis of erythemato-squamous diseases. Expert Syst. Appl. 18(1), 43–49 (2000)

    Article  Google Scholar 

  • Hair, D.C., Pickslay, K., Chow, S.: Explanation-based decision support in real time situations. In: Proceedings of the Fourth International Conference on Tools with Artificial Intelligence, TAI ’92, pp. 22–25 (1992)

  • Hanshi, W., Qiujie, F., Lizhen, L., Wei, S.: A probabilistic rating prediction and explanation inference model for recommender systems. China Commun. 13(2), 79–94 (2016)

    Google Scholar 

  • Hasling, D.W., Clancey, W.J., Rennels, G.: Strategic explanations for a diagnostic consultation system. Int. J. Man Mach. Stud. 20(1), 3–19 (1984)

    Article  Google Scholar 

  • Hatzilygeroudis, I., Prentzas, J.: Symbolic-neural rule based reasoning and explanation. Expert Syst. Appl. 42(9), 4595–4609 (2015)

    Article  Google Scholar 

  • Helms, G.L., Richardson, J.W., Cochran, M.J., Rister, M.: A farm level expert simulation system to aid farmers in selecting among crop insurance strategies. Comput. Electron. Agric. 4(3), 169–190 (1990)

    Article  Google Scholar 

  • Herlocker, J.L., Konstan, J.A., Riedl, J.: Explaining collaborative filtering recommendations. In: Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, CSCW ’00, pp. 241–250 (2000)

  • Hodgkinson, L., Walker, E.: An expert system for credit evaluation and explanation. J. Comput. Sci. Coll. 19(1), 62–72 (2003)

    Google Scholar 

  • Holman, J.G., Wolff, A.H.: An expert adviser for oliguria occurring on the intensive care unit. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 1442–1443 (1988)

  • Horan, J., O’Sullivan, B.: Towards diverse relaxations of over-constrained models. In: 2009 21st IEEE International Conference on Tools with Artificial Intelligence, pp. 198–205 (2009)

  • Horn, W., Popow, C., Miksch, S., Seyfang, A.: Quicker, more accurate nutrition plans for newborn infants. IEEE Intell. Syst. Appl. 13(1), 65–69 (1998)

    Article  Google Scholar 

  • Hornung, T., Ziegler, C.N., Franz, S., Przyjaciel-Zablocki, M., Schtzle, A., Lausen, G.: Evaluating hybrid music recommender systems. In: IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT), vol. 1, pp. 57–64 (2013)

  • Hu, Y., Koren, Y., Volinsky, C.: Collaborative filtering for implicit feedback datasets. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 263–272 (2008)

  • Hudson, D.L., Cohen, M.E.: Human–computer interaction in a medical decision support system. In: Proceedings of the Twenty-Second Annual Hawaii International Conference on System Sciences. Volume II: Software Track, vol. 2, pp. 429–435 (1989)

  • Hunt, J., Price, C.: Explaining qualitative diagnosis. Eng. Appl. Artif. Intell. 1(3), 161–169 (1988)

    Article  Google Scholar 

  • Hussain, S., Abidi, S.S.R.: Ontology driven CPG authoring and execution via a semantic web framework. In: 40th Annual Hawaii International Conference on System Sciences, HICSS 2007, pp. 135–135 (2007)

  • Hussein, T., Neuhaus, S.: Explanation of spreading activation based recommendations. In: Proceedings of the 1st International Workshop on Semantic Models for Adaptive Interactive Systems, SEMAIS ’10, pp. 24–28 (2010)

  • Jabri, M.A.: Knowledge-based system design using prolog: the PIAF experience. Knowl. Based Syst. 2(1), 72–79 (1989)

    Article  Google Scholar 

  • Jaimes, A., Gatica-Perez, D., Sebe, N., Huang, T.S.: Guest editors’ introduction: human-centered computing-toward a human revolution. Computer 40(5), 30–34 (2007)

    Article  Google Scholar 

  • Jamieson, P.W.: A model for diagnosing and explaining multiple disorders. Comput. Biomed. Res. 24(4), 307–320 (1991)

    Article  Google Scholar 

  • Janjua, N.K., Hussain, F.K.: Defeasible reasoning based argumentative Web-IDSS for virtual teams (VTs). In: 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, pp. 330–334 (2011)

  • Jannach, D., Adomavicius, G.: Recommendations with a purpose. In: Proceedings of the 2016 ACM Conference on Recommender Systems, RecSys ’16, pp. 7–10 (2016)

  • Jannach, D., Zanker, M., Felfernig, A., Friedrich, G.: Recommender Systems: An Introduction. Cambridge University Press, New York (2010)

    Book  Google Scholar 

  • Jannach, D., Resnick, P., Tuzhilin, A., Zanker, M.: Recommender systems—beyond matrix completion. Commun. ACM 59(11), 94–102 (2016)

    Article  Google Scholar 

  • Ji, K., Shen, H.: Jointly modeling content, social network and ratings for explainable and cold-start recommendation. Neurocomputing 218, 1–12 (2016)

    Article  Google Scholar 

  • Joch, J., Dudeck, J.: Decision support for infectious diseasesa working prototype. Int. J. Med. Inform. 64(23), 331–340 (2001)

    Article  Google Scholar 

  • Jugovac, M., Jannach, D.: Interacting with recommenders—overview and research directions. ACM Trans. Interact. Intell. Syst. 7(3), 46 (2017)

  • Jung, D., Burns, J.R.: Connectionist approaches to inexact reasoning and learning systems for executive and decision support. Decis. Support Syst. 10(1), 37–66 (1993)

    Article  Google Scholar 

  • Junker, U.: Quickxplain: preferred explanations and relaxations for over-constrained problems. In: AAAI’04, pp. 167–172. USA (2004)

  • Kadhim, M.A., Alam, M.A., Kaur, H.: Design and implementation of intelligent agent and diagnosis domain tool for rule-based expert system. In: 2013 International Conference on Machine Intelligence and Research Advancement, pp. 619–622 (2013)

  • Kagal, L., Pato, J.: Preserving privacy based on semantic policy tools. IEEE Secur. Priv. 8(4), 25–30 (2010)

    Article  Google Scholar 

  • Karwowski, W., Mulholland, N.O., Ward, T.L., Jagannathan, V.: A fuzzy knowledge base of an expert system for analysis of manual lifting tasks. Fuzzy Sets Syst. 21(3), 363–374 (1987)

    Article  Google Scholar 

  • Katarya, R., Jain, I., Hasija, H.: An interactive interface for instilling trust and providing diverse recommendations. In: International Conference on Computer and Communication Technology (ICCCT), pp. 17–22 (2014)

  • Keeney, R.L., Raiffa, H.: Decisions with Multiple Objectives: Preferences and Value Tradeoffs. Wiley Series in Probability and Mathematical Statistics. Wiley, Hoboken (1976)

    MATH  Google Scholar 

  • Kim, B.O., Lee, S.M.: A bond rating expert system for industrial companies. Expert Syst. Appl. 9(1), 63–70 (1995)

    Article  Google Scholar 

  • Kim, S.K., Park, J.I.: A structural equation modeling approach to generate explanations for induced rules. Expert Syst. Appl. 10(3), 403–416 (1996)

    Article  Google Scholar 

  • Kitchenham, B., Charters, S.: Guidelines for performing systematic literature reviews in software engineering. Technical Report EBSE-2007-01, School of Computer Science and Mathematics, Keele University (2007)

  • Kitchenham, B., Brereton, P.: A systematic review of systematic review process research in software engineering. Inf. Softw. Technol. 55(12), 2049–2075 (2013)

    Article  Google Scholar 

  • Klein, D.A., Shortliffe, E.H.: A framework for explaining decision-theoretic advice. Artif. Intell. 67(2), 201–243 (1994)

    Article  MATH  Google Scholar 

  • Koussev, T., Weiss, M.P., Reiss, K.: A graphic explanation environment for expert systems. In: Second International Conference on Software Engineering for Real Time Systems, pp. 11–15 (1989)

  • Labreuche, C.: A general framework for explaining the results of a multi-attribute preference model. Artif. Intell. 175(7), 1410–1448 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  • Lacave, C., Díez, F.J.: A review of explanation methods for bayesian networks. Knowl. Eng. Rev. 17(2), 107–127 (2002)

    Article  Google Scholar 

  • Lacave, C., Díez, F.J.: A review of explanation methods for heuristic expert systems. Knowl. Eng. Rev. 19(2), 133–146 (2004)

  • Lacave, C., Oniśko, A., Díez, F.J.: Use of Elvira’s explanation facility for debugging probabilistic expert systems. Knowl. Based Syst. 19(8), 730–738 (2006)

    Article  Google Scholar 

  • Lambert, S.C., Ringland, G.A.: Knowledge representations and interfaces in financial expert systems. In: UK IT 1990 Conference, pp. 434–441 (1990)

  • Langlotz, C.P., Shortliffe, E.H.: Adapting a consultation system to critique user plans. Int. J. Man Mach. Stud. 19(5), 479–496 (1983)

    Article  Google Scholar 

  • Lee, H.M., Hsu, C.C.: Building expert systems by training with automatic neural network generating ability. In: Proceedings Eighth Conference on Artificial Intelligence for Applications, pp. 197–203 (1992)

  • Levy, M., Ferrand, P., Chirat, V.: SESAM-DIABETE, an expert system for insulin-requiring diabetic patient education. Comput. Biomed. Res. 22(5), 442–453 (1989)

    Article  Google Scholar 

  • Li, M., Gregor, S.: Outcomes of effective explanations: empowering citizens through online advice. Decis. Support Syst. 52(1), 119–132 (2011)

    Article  Google Scholar 

  • Libório, A., Furtado, E., Rocha, I., Furtado, V.: Interface design through knowledge-based systems: an approach centered on explanations from problem-solving models. In: Proceedings of the 4th International Workshop on Task Models and Diagrams, TAMODIA ’05, pp. 127–134 (2005)

  • Lieberman, H., van Dyke, N., Vivacqua, A.: Let’s browse: a collaborative browsing agent. Knowl. Based Syst. 12(8), 427–431 (1999)

    Article  Google Scholar 

  • Liu, K.F.R., Lee, J., Chiang, W., Yang, S.J.: Fpnes: fuzzy Petri net based expert system for bridges damage assessment. In: Proceedings Tenth IEEE International Conference on Tools with Artificial Intelligence, pp. 302–309 (1998)

  • Lopez-Suarez, A., Kamel, M.: Dykor: a method for generating the content of explanations in knowledge systems. Knowl. Based Syst. 7(3), 177–188 (1994)

    Article  Google Scholar 

  • Machado, R.J., da Rocha, A.F.: Inference, inquiry and explanation in expert systems by means of fuzzy neural networks. In: Proceedings of the Second IEEE International Conference on Fuzzy Systems, vol. 1, pp. 351–356 (1993)

  • Mahmoud, M., Algadi, N., Ali, A.: Expert system for banking credit decision. In: 2008 International Conference on Computer Science and Information Technology, pp. 813–819 (2008)

  • Malheiro, N., Vale, Z.A., Ramos, C., Santos, J., Marques, A.: Enabling Client-Server Explanation Facilities in a Real-Time Expert System, pp. 333–342. Springer, Berlin (1999)

    Google Scholar 

  • Mao, J.Y., Benbasat, I.: The effects of contextualized access to knowledge on judgement. Int. J. Hum. Comput. Stud. 55(5), 787–814 (2001)

    Article  MATH  Google Scholar 

  • Martincic, C.J.: QUE: an expert system explanation facility that answers “why not” types of questions. J. Comput. Sci. Coll. 19(1), 336–348 (2003)

    Google Scholar 

  • Marx, P., Hennig-Thurau, T., Marchand, A.: Increasing consumers’ understanding of recommender results: a preference-based hybrid algorithm with strong explanatory power. In: Proceedings of the Fourth ACM Conference on Recommender Systems, RecSys ’10, pp. 297–300 (2010)

  • Matelli, J.A., Bazzo, E., da Silva, J.C.: An expert system prototype for designing natural gas cogeneration plants. Expert Syst. Appl. 36(4), 8375–8384 (2009)

    Article  Google Scholar 

  • Matsatsinis, N., Doumpos, M., Zopounidis, C.: Knowledge acquisition and representation for expert systems in the field of financial analysis. Expert Syst. Appl. 12(2), 247–262 (1997)

    Article  Google Scholar 

  • Maybury, M.T.: Enhancing explanation coherence with rhetorical strategies. In: Proceedings of the Fourth Conference on European Chapter of the Association for Computational Linguistics, EACL ’89, pp. 168–173 (1989)

  • McCarthy, K., Reilly, J., McGinty, L., Smyth, B.: Experiments in dynamic critiquing. In: Proceedings of the 10th International Conference on Intelligent User Interfaces, IUI ’05, pp. 175–182. ACM (2005)

  • Mcsherry, D.: Explanation in recommender systems. Artif. Intell. Rev. 24(2), 179–197 (2005)

    Article  MATH  Google Scholar 

  • Mejia-Lavalle, M.: Outlier detection with innovative explanation facility over a very large financial database. In: 2010 IEEE Electronics, Robotics and Automotive Mechanics Conference, pp. 23–27 (2010)

  • Mendes, D., Rodrigues, I.P., Baeta, C.: Ontology based clinical practice justification in natural language. Procedia Technol. 9, 1288–1293 (2013)

    Article  Google Scholar 

  • Metzler, D.P., Martincic, C.J.: QUE: explanation through exploration. Expert Syst. Appl. 15(34), 253–263 (1998)

    Article  Google Scholar 

  • Mitra, S.: Fuzzy mlp based expert system for medical diagnosis. Fuzzy Sets Syst. 65(2), 285–296 (1994)

    Article  Google Scholar 

  • Mitra, S., Pal, S.K.: Fuzzy multi-layer perceptron, inferencing and rule generation. IEEE Trans. Neural Netw. 6(1), 51–63 (1995)

    Article  Google Scholar 

  • Mller-Kolck, U.: Expert system support for the therapeutic management of cerebrovascular disease. Artif. Intell. Med. 2(1), 35–42 (1990)

    Article  Google Scholar 

  • Mocanu, A.: Envisioning a collaborative smart home solution based on argumentative dialogues. In: Proceedings of the 7th Balkan Conference on Informatics Conference, BCI ’15, pp. 23:1–23:6 (2015)

  • Moulin, B., Irandoust, H., Bélanger, M., Desbordes, G.: Explanation and argumentation capabilities: towards the creation of more persuasive agents. Artif. Intell. Rev. 17(3), 169–222 (2002)

    Article  MATH  Google Scholar 

  • Muhammad, K., Lawlor, A., Rafter, R., Smyth, B.: Great Explanations: Opinionated Explanations for Recommendations, pp. 244–258. Springer, Berlin (2015)

    Google Scholar 

  • Muhammad, K.I., Lawlor, A., Smyth, B.: A live-user study of opinionated explanations for recommender systems. In: Proceedings of the 21st International Conference on Intelligent User Interfaces, IUI ’16, pp. 256–260 (2016)

  • Murphy, D.S., Phillips, M.E.: The effects of expert system use on entry-level accounting expertise: an experiment. Expert Syst. Appl. 3(1), 129–134 (1991)

    Article  Google Scholar 

  • Nakatsu, R.T.: Explanatory Power of Intelligent Systems, pp. 123–143. Springer, London (2006)

    Google Scholar 

  • Nakatsu, R.T., Benbasat, I.: Improving the explanatory power of knowledge-based systems: an investigation of content and interface-based enhancements. Trans. Syst. Man Cybern. Part A 33(3), 344–357 (2003)

    Article  Google Scholar 

  • Narayanan, T., McGuinness, D.L.: Towards leveraging inference web to support intuitive explanations in recommender systems for automated career counseling. In: First International Conference on Advances in Computer–Human Interaction, pp. 164–169 (2008)

  • Nart, D.D., Tasso, C.: A personalized concept-driven recommender system for scientific libraries. Procedia Comput. Sci. 38, 84–91 (2014)

    Article  Google Scholar 

  • Ng, G., Ong, K.: Using a qualitative probabilistic network to explain diagnostic reasoning in an expert system for chest pain diagnosis. Comput. Cardiol. 2000(27), 569–572 (2000)

    Google Scholar 

  • Nilashi, M., Jannach, D., bin Ibrahim, O., Esfahani, M.D., Ahmadi, H.: Recommendation quality, transparency, and website quality for trust-building in recommendation agents. Electron. Commer. Res. Appl. 19, 70–84 (2016)

    Article  Google Scholar 

  • Norton, S.W.: An explanation mechanism for bayesian inferencing systems. In: Lemmer, J.F., Kanal, L.N. (eds.) Uncertainty in Artificial Intelligence, Machine Intelligence and Pattern Recognition, vol. 5, pp. 165–173. North-Holland, Amsterdam (1988)

    Google Scholar 

  • Nunes, I., Miles, S., Luck, M., de Lucena, C.J.P.: Investigating explanations to justify choice. In: Proceedings of the 20th International Conference on User Modeling, Adaptation, and Personalization, UMAP’12, pp. 212–224 (2012a)

  • Nunes, I., Chen, Y., Miles, S., Luck, M., Lucena, C.: Transparent Provenance Derivation for User Decisions, pp. 111–125. Springer, Berlin (2012b)

    Google Scholar 

  • Nunes, I., Miles, S., Luck, M., Barbosa, S., Lucena, C.: Pattern-based explanation for automated decisions. In: Proceedings of the Twenty-first European Conference on Artificial Intelligence, ECAI’14, pp. 669–674 (2014)

  • Nuthall, P., Bishop-Hurley, G.: Expert systems for animal feeding management part i: presentation aspects. Comput. Electron. Agric. 14(1), 9–22 (1996)

    Article  Google Scholar 

  • O’Donovan, J., Gretarsson, B., Bostandjiev, S., Hollerer, T., Smyth, B.: A visual interface for social information filtering. In: 2009 International Conference on Computational Science and Engineering, vol. 4, pp. 74–81 (2009)

  • Omran, A.M., Khorshid, M.: Intelligent environmental scanning approach (a case study: the Egyptian wheat crop production). IERI Procedia 7, 28–34 (2014a)

    Article  Google Scholar 

  • Omran, A.M., Khorshid, M.: An intelligent recommender system for long view of Egypt’s livestock production. AASRI Procedia 6, 103–110 (2014b)

    Article  Google Scholar 

  • Oramas, S., Espinosa-Anke, L., Sordo, M., Saggion, H., Serra, X.: Information extraction for knowledge base construction in the music domain. Data Knowl. Eng. 106, 70–83 (2016)

    Article  Google Scholar 

  • Overby, M.A.: Psyxpert: an expert system prototype for aiding psychiatrists in the diagnosis of psychotic disorders. Comput. Biol. Med. 17(6), 383–393 (1987)

    Article  Google Scholar 

  • Pal, K.: An approach to legal reasoning based on a hybrid decision-support system. Expert Syst. Appl. 17(1), 1–12 (1999)

    Article  MathSciNet  Google Scholar 

  • Pal, K., Palmer, O.: A decision-support system for business acquisitions. Decis. Support Syst. 27(4), 411–429 (2000)

    Article  Google Scholar 

  • Papamichail, K., French, S.: Explaining and justifying the advice of a decision support system: a natural language generation approach. Expert Syst. Appl. 24(1), 35–48 (2003)

    Article  Google Scholar 

  • Papadimitriou, A., Symeonidis, P., Manolopoulos, Y.: A generalized taxonomy of explanations styles for traditional and social recommender systems. Data Min. Knowl. Discov. 24(3), 555–583 (2012)

    Article  Google Scholar 

  • Pazzani, M., Iyer, R., See, D., Schroeder, E., Tilles, J.: CTSHIV: a knowledge-based system for the management of HIV-infected patients. In: Intelligent Information Systems, 1997. IIS ’97, pp. 7–13 (1997)

  • Perlin, M., Kanal, E., John, A.: A user interface for visualizing concepts in magnetic resonance imaging. In: Proceedings of the First Conference on Visualization in Biomedical Computing, pp. 260–267 (1990)

  • Popchev, I.P., Zlatareva, N.P., Sinapova, L.J.: EDDY: an expert system in dysmorphology based on truth-maintenance. In: Images of the Twenty-First Century. Proceedings of the Annual International Engineering in Medicine and Biology Society, pp. 1877–1878 (1989)

  • Pu, P., Chen, L.: Trust-inspiring explanation interfaces for recommender systems. Knowl. Based Syst. 20(6), 542–556 (2007)

    Article  Google Scholar 

  • Rahwan, I., Simari, G.R.: Argumentation in Artificial Intelligence, 1st edn. Springer, Berlin (2009)

    Google Scholar 

  • Ramberg, R.: Construing and testing explanations in a complex domain. Comput. Hum. Behav. 12(1), 29–48 (1996)

    Article  Google Scholar 

  • Ray, A.K.: Equipment fault diagnosisa neural network approach. Comput. Ind. 16(2), 169–177 (1991)

    Article  Google Scholar 

  • Reggia, J.A., Perricone, B.T., Nau, D.S., Peng, Y.: Answer justification in diagnostic expert systems—part I: abductive inference and its justification. IEEE Trans. Biomed. Eng. BME–32(4), 263–267 (1985)

    Article  Google Scholar 

  • Reilly, J., McCarthy, K., McGinty, L., Smyth, B.: Explaining compound critiques. Artif. Intell. Rev. 24(2), 199–220 (2005)

    Article  Google Scholar 

  • Reyes, A., Ibarguengoytia, P.H., Elizalde, F., Snchez, L., Nava, A.: ASISTO: an integrated intelligent assistant system for power plant operation and training. In: 16th International Conference on Intelligent System Applications to Power Systems, pp. 1–6 (2011)

  • Richards, D.: The reuse of knowledge: a user-centred approach. Int. J. Hum. Comput. Stud. 52(3), 553–579 (2000)

    Article  Google Scholar 

  • Ringer, M.J., Quinn, T.M., Merolla, A.: Autonomous power system: intelligent diagnosis and control. Telemat. Inform. 8(4), 365–383 (1991)

    Article  Google Scholar 

  • Riordan, D., Carden, K.J.: Explanation in ecological systems. In: Proceedings of the 1990 ACM SIGSMALL/PC Symposium on Small Systems, SIGSMALL ’90, pp. 249–254 (1990)

  • Roitman, H., Messika, Y., Tsimerman, Y., Maman, Y.: Increasing patient safety using explanation-driven personalized content recommendation. In: Proceedings of the 1st ACM International Health Informatics Symposium, IHI ’10, pp. 430–434 (2010)

  • Rook, F.W., Donnell, M.L.: Human cognition and the expert system interface: mental models and inference explanations. IEEE Trans. Syst. Man Cybern. 23(6), 1649–1661 (1993)

    Article  Google Scholar 

  • Samarasinghe, S.: Neural Networks for Applied Sciences and Engineering. Auerbach Publications, Boston (2006)

    Book  MATH  Google Scholar 

  • Santoso, N.I., Darken, C., Povh, G., Erdmann, J.: Nuclear plant fault diagnosis using probabilistic reasoning. In: Proceedings of the 1999 IEEE Power Engineering Society Summer Meeting, vol. 2, pp. 714–719 (1999)

  • Sarkar, A., Bandyopadhyay, S., Jullien, G.A.: Bit-level designer’s assistant-a knowledge based approach to systolic processor design. In: Proceedings of the 33rd Midwest Symposium on Circuits and Systems, pp. 1001–1004 (1990)

  • Saunders, V.M., Dobbs, V.S.: Explanation generation in expert systems. In: IEEE Conference on Aerospace and Electronics, pp. 1101–1106 (1990)

  • Schaffer, J., Giridhar, P., Jones, D., Höllerer, T., Abdelzaher, T., O’Donovan, J.: Getting the message? A study of explanation interfaces for microblog data analysis. In: Proceedings of the 20th International Conference on Intelligent User Interfaces, IUI ’15, pp. 345–356 (2015)

  • Scheel, C., Castellanos, A., Lee, T., De Luca, E.W.: The Reason Why: A Survey of Explanations for Recommender Systems, pp. 67–84. Springer, Berlin (2014)

    Google Scholar 

  • Schröder, O., Möbus, C., Folckers, J., Thole, H.J.: Supporting the construction of explanation models and diagnostic reasoning in probabilistic domains. In: Proceedings of the 1996 International Conference on Learning Sciences, ICLS ’96, pp. 60–67 (1996)

  • Shaalan, K., Rafea, M., Rafea, A.: KROL: a knowledge representation object language on top of Prolog. Expert Syst. Appl. 15(1), 33–46 (1998)

    Article  Google Scholar 

  • Sharma, A., Cosley, D.: Do social explanations work? Studying and modeling the effects of social explanations in recommender systems. In: Proceedings of the 22nd International Conference on World Wide Web, WWW ’13, pp. 1133–1144 (2013)

  • Sherchan, W., Loke, S.W., Krishnaswamy, S.: Explanation-aware service selection: rationale and reputation. Serv. Oriented Comput. Appl. 2(4), 203–218 (2008)

    Article  Google Scholar 

  • Shoval, P.: Principles, procedures and rules in an expert system for information retrieval. Inf. Process. Manag. 21(6), 475–487 (1985)

    Article  Google Scholar 

  • Slagle, J.R.: Applications of a generalized network-based expert system shell-artificial intelligence mini-tutorial. In: Proceedings of the Symposium on the Engineering of Computer-Based Medical, pp. 33–42 (1988)

  • Slotnick, S.A., Moore, J.D.: Explaining quantitative systems to uninitiated users. Expert Syst. Appl. 8(4), 475–490 (1995)

    Article  Google Scholar 

  • Song, W., Shi, H., Li, Q.: Study of an explanation mechanism in expert system based on fault tree for safety risk assessment. In: 2nd International Conference on Future Computer and Communication, vol. 2, pp. V2–479–V2–483 (2010)

  • Sørmo, F., Cassens, J., Aamodt, A.: Explanation in case-based reasoning-perspectives and goals. Artif. Intell. Rev. 24(2), 109–143 (2005)

    Article  MATH  Google Scholar 

  • Srivastava, R.P.: Automating judgmental decisions using neural networks: a model for processing business loan applications. In: Proceedings of the 1992 ACM Annual Conference on Communications, CSC ’92, pp. 351–357 (1992)

  • Strachan, S.M., McArthur, S.D.J., Judd, M.D., McDonald, J.R.: Incremental knowledge-based partial discharge diagnosis in oil-filled power transformers. In: Proceedings of the 13th International Conference on Intelligent Systems Application to Power Systems (2005)

  • Strat, T.M., Lowrance, J.D.: Explaining evidential analyses. Int. J. Approx. Reason. 3(4), 299–353 (1989)

    Article  MATH  Google Scholar 

  • Štrumbelj, E., Kononenko, I., Šikonja, M.R.: Explaining instance classifications with interactions of subsets of feature values. Data Knowl. Eng. 68(10), 886–904 (2009)

    Article  Google Scholar 

  • Suermondt, H.J., Cooper, G.F.: An evaluation of explanations of probabilistic inference. Comput. Biomed. Res. 26(3), 242–254 (1993)

    Article  Google Scholar 

  • Swartout, W.R., Moore, J.D.: Explanation in Second Generation Expert Systems, pp. 543–585. Springer, Berlin (1993)

    Book  Google Scholar 

  • Swinney, L.: The explanation facility and the explanation effect. Expert Syst. Appl. 9(4), 557–567 (1995)

    Article  Google Scholar 

  • Symeonidis, P., Nanopoulos, A., Manolopoulos, Y.: Providing justifications in recommender systems. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 38(6), 1262–1272 (2008)

    Article  Google Scholar 

  • Tan, W.K., Tan, C.H., Teo, H.H.: Consumer-based decision aid that explains which to buy: decision confirmation or overconfidence bias? Decis. Support Syst. 53(1), 127–141 (2012)

    Article  Google Scholar 

  • Tanner, M.C., Keuneke, A.M.: Explanations in knowledge systems: the roles of the task structure and domain functional models. IEEE Expert 6(3), 50–57 (1991)

    Article  Google Scholar 

  • Terano, T., Suzuki, M., Onoda, T., Uenishi, K., Matsuura, T.: CSES: an approach to integrating graphic, music and voice information into a user-friendly interface. In: International Workshop on Industrial Applications of Machine Intelligence and Vision, pp. 349–354 (1989)

  • Thirumuruganathan, S., Huber, M.: Building bayesian network based expert systems from rules. In: 2011 IEEE International Conference on Systems, Man, and Cybernetics, pp. 3002–3008 (2011)

  • Tintarev, N., Masthoff, J.: Effective explanations of recommendations: user-centered design. In: Proceedings of the 2007 ACM Conference on Recommender Systems, RecSys ’07, pp. 153–156 (2007a)

  • Tintarev, N., Masthoff, J.: A survey of explanations in recommender systems. In: Proceedings of the 2007 IEEE 23rd International Conference on Data Engineering Workshop, pp. 801–810 (2007b)

  • Tintarev, N., Masthoff, J.: Designing and evaluating explanations for recommender systems. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 479–510. Springer, Berlin (2011)

    Chapter  Google Scholar 

  • Tintarev, N., Masthoff, J.: Evaluating the effectiveness of explanations for recommender systems. User Model. User Adapt. Interact. 22(4–5), 399–439 (2012)

    Article  Google Scholar 

  • Tjahjadi, T., Bowen, D., Bevan, J.R.: 3M: a user modelling interface of an expert system for x-ray topographic image interpretation. Interact. Comput. 2(3), 259–278 (1990)

    Article  Google Scholar 

  • Tong, L.C.: An explanation facility for a grammar writing system. In: Proceedings of the 13th Conference on Computational Linguistics, COLING ’90, pp. 359–364 (1990)

  • Tong, X., Ang, J.: Explaining control strategies in second generation expert systems. IEEE Trans. Syst. Man Cybern. 25(11), 1483–1490 (1995)

    Article  Google Scholar 

  • Toulmin, S.E.: The Uses of Argument. Cambridge University Press, Cambridge (2003)

    Book  Google Scholar 

  • Tzafestas, S., Konstantinidis, N.: ENGEXP—an integrated environment for the development and application of expert systems in equipment and engine fault diagnosis and repair. Adv. Eng. Softw. 14(1), 3–14 (1992)

    Article  Google Scholar 

  • van Aarle, E., van den Bercken, J.: The development of a knowledge-based system supporting the diagnosis of reading and spelling problems. Comput. Hum. Behav. 8(23), 183–201 (1992)

    Article  Google Scholar 

  • Vashisth, P., Chandoliya, D., Yadav, B.K., Bedi, P.: Trust enabled argumentation based recommender system. In: 12th International Conference on Intelligent Systems Design and Applications (ISDA), pp. 137–142 (2012)

  • Vig, J., Sen, S., Riedl, J.: Tagsplanations: explaining recommendations using tags. In: Proceedings of the 14th International Conference on Intelligent User Interfaces, IUI ’09, pp. 47–56 (2009)

  • Vogiatzis, D., Karkaletsis, V.: A cognitive framework for robot guides in art collections. Univers. Access Inf. Soc. 10(2), 179–193 (2011)

    Article  Google Scholar 

  • Wall, R., Cunningham, P., Walsh, P., Byrne, S.: Explaining the output of ensembles in medical decision support on a case by case basis. Artif. Intell. Med. 28(2), 191–206 (2003)

    Article  Google Scholar 

  • Wang, L., Libert, G., Liu, B.: An expert system for forecasting model selection. In: Proceedings of the First IEEE Conference on Control Applications, pp. 704–709 (1992)

  • Wang, N., Pynadath, D.V., Hill, S.G.: The impact of pomdp-generated explanations on trust and performance in human-robot teams. In: Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, AAMAS ’16, pp. 997–1005 (2016a)

  • Wang, N., Pynadath, D.V., Hill, S.G.: Trust calibration within a human-robot team: comparing automatically generated explanations. In: Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 109–116 (2016b)

  • Wang, W., Qiu, L., Kim, D., Benbasat, I.: Effects of rational and social appeals of online recommendation agents on cognition- and affect-based trust. Decis. Support Syst. 86(C), 48–60 (2016c)

    Article  Google Scholar 

  • Washington, E.S., Ali, M.: PISCES: an expert system for coal fired power plant monitoring and diagnostics. In: Proceedings of the 1st International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems, IEA/AIE ’88, pp. 87–93 (1988)

  • Wick, M.R., Slagle, J.R.: An explanation facility for today’s expert systems. IEEE Expert 4(1), 26–36 (1989a)

    Article  Google Scholar 

  • Wick, M.R., Slagle, J.R.: The partitioned support network for expert system justification. IEEE Trans. Syst. Man Cybern. 19(3), 528–535 (1989b)

    Article  Google Scholar 

  • Widyantoro, D.H., Baizal, Z.K.A.: A framework of conversational recommender system based on user functional requirements. In: 2nd International Conference on Information and Communication Technology (ICoICT), pp. 160–165 (2014)

  • Wong, K.P., Cheung, H.N.: Expert system for protection current transformer design specification preparation. IEE Proc. C Gener. Transm. Distrib. 136(6), 391–400 (1989)

    Article  Google Scholar 

  • Yasdi, R.: Design of the exis’s explanation component. Comput. Ind. 13(1), 15–21 (1989)

    Article  Google Scholar 

  • Ye, L.R.: The value of explanation in expert systems for auditing: an experimental investigation. Expert Syst. Appl. 9(4), 543–556 (1995)

    Article  Google Scholar 

  • Ye, L.R., Johnson, P.E.: The impact of explanation facilities on user acceptance of expert systems advice. MIS Q. 19, 157–172 (1995)

    Article  Google Scholar 

  • Yen, J.: Gertis: a dempster-shafer approach to diagnosing hierarchical hypotheses. Commun. ACM 32(5), 573–585 (1989)

    Article  Google Scholar 

  • Yoon, Y., Guimaraes, T., Swales, G.: Integrating artificial neural networks with rule-based expert systems. Decis. Support Syst. 11(5), 497–507 (1994)

    Article  Google Scholar 

  • Yu, C., Lakshmanan, L., Amer-Yahia, S.: It takes variety to make a world: diversification in recommender systems. In: Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology, EDBT ’09, pp. 368–378 (2009)

  • Zain, M.F.M., Islam, M.N., Basri, I.H.: An expert system for mix design of high performance concrete. Adv. Eng. Softw. 36(5), 325–337 (2005)

    Article  MATH  Google Scholar 

  • Zanker, M.: The influence of knowledgeable explanations on users’ perception of a recommender system. In: Proceedings of the Sixth ACM Conference on Recommender Systems, RecSys ’12, pp. 269–272 (2012)

  • Zanker, M., Ninaus, D.: Knowledgeable explanations for recommender systems. In: 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, vol. 1, pp. 657–660 (2010)

  • Zeleznikow, J., Stranieri, A., Gawler, M.: Project report: split-up—a legal expert system which determines property division upon divorce. Artif. Intell. Law 3(4), 267–275 (1995)

    Article  Google Scholar 

  • Zhang, Y., Lai, G., Zhang, M., Zhang, Y., Liu, Y., Ma, S.: Explicit factor models for explainable recommendation based on phrase-level sentiment analysis. In: Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR ’14, pp. 83–92 (2014)

Download references

Acknowledgements

The authors would like to thank Michael Jugovac for carefully proofreading this paper. Ingrid Nunes also would like to thank for research grants CNPq ref. 303232/2015-3, CAPES ref. 7619-15-4, and Alexander von Humboldt, ref. BRA 1184533 HFSTCAPES-P.

Author information

Authors and Affiliations

  1. Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil

    Ingrid Nunes

  2. TU Dortmund, Dortmund, Germany

    Ingrid Nunes & Dietmar Jannach

Authors
  1. Ingrid Nunes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dietmar Jannach
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ingrid Nunes.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nunes, I., Jannach, D. A systematic review and taxonomy of explanations in decision support and recommender systems. User Model User-Adap Inter 27, 393–444 (2017). https://doi.org/10.1007/s11257-017-9195-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-017-9195-0

Keywords

Search

Navigation