The decoy effect in relative performance evaluation and the debiasing role of DEA

https://doi.org/10.1016/j.ejor.2015.07.045Get rights and content

Highlights

  • We investigate the decoy effect in the context of Data Envelopment Analysis (DEA).

  • It is shown that relative performance evaluation is prone to this kind of bias.

  • Adding DEA scores to the choice set proves to be an effective debiasing procedure.

  • Adding information about slacks also reduces the decoy effect in the experiment.

Abstract

There is overwhelming evidence that performance ratings and evaluations are context dependent. A special case of such context effects is the decoy effect, which implies that the inclusion of a dominated alternative can influence the preference for non-dominated alternatives. Adapting the well-known experimental setting from the area of consumer behavior to the performance evaluation context of Data Envelopment Analysis (DEA), an experiment was conducted. The results show that adding a dominated decision making unit (DMU) to the set of DMUs augments the attractiveness of certain dominating DMUs and that DEA efficiency scores discriminating between efficient and inefficient DMUs serve as an appropriate debiasing procedure. The mention of the existence of slacks for distinguishing between strong and weak efficient DMUs also contributes to reducing the decoy effect, but it is also associated to other unexpected effects.

Introduction

Despite being described as a discipline aiming at facilitating decision making, operations research (OR) has largely omitted behavioral issues. Most recently, this deficit has been addressed by Hämäläinen, Luoma, and Saarinen (2013). They comprehensively discuss the high potential of behavioral OR for advancing the practical benefit of normative OR methods and propose nine topics for a respective research agenda. The present paper refers particularly to the topic of behavioral/cognitive aspects, since it stresses performance evaluation biases related to the so-called decoy effect.

The decoy effect is a special kind of context effect, which refers to the influence that contextual variables have on decision making. Overwhelming evidence exists demonstrating that context is likely to influence performance evaluation (see, e.g., Damisch et al., 2006, Page and Page, 2010). For example, evaluators’ preferences are biased not only by the past performance of the decision making unit (DMU), but also by the performance of other DMUs under analysis. With regard to the latter aspect, the decoy effect implies that the inclusion of a dominated alternative – the decoy – can influence the choice between the non-dominated alternatives. Concretely, the probability of preferring the target alternative, which is the non-dominated option that is most similar to the decoy, may increase. The existence of this effect has repeatedly been confirmed by research on consumer behavior, showing that customers tend to prefer the target alternative (Ariely, 2009, Huber et al., 1982, Wedell and Pettibone, 1996).

The research settings used for studying consumer behavior strongly resemble relative performance evaluation cases where alternatives are compared on a utility function level. Furthermore, these research settings are constructed in a way that they differentiate between non-dominated (i.e., efficient) and dominated (i.e., inefficient) units like in the context of Data Envelopment Analysis (DEA). Against this background, two questions arise: (i) does the decoy effect also occur in cases where the relative performance of alternatives is evaluated, and (ii) to what extent can the application of DEA – namely the incorporation of respective efficiency scores and the mention of existing slacks – act as a debiasing tool?

To shed light on these questions, we conducted a vignette-based experiment with bachelor students taking management control and business accounting courses. For analyzing whether relative performance evaluation may also be affected by a decoy effect, the well-known experimental setting from the area of consumer behavior was adapted to the performance context. The results show that adding a dominated DMU to the set of DMUs augments the attractiveness of the target DMU. Therefore, a decoy effect can be observed when DMUs are compared against each other.

In a second step, the role of reporting DEA results as a debiasing procedure for the identified decoy effect was considered. Participants in the corresponding treatments were provided with additional information about DEA efficiency scores and slacks. The results indicate that the scores discriminating between efficient and inefficient DMUs can significantly reduce the decoy effect in a relative performance evaluation context. The mention of the existence of slacks for distinguishing between strong and weak efficient DMUs also contributes to avoiding the decoy effect, but it is associated to unexpected effects, such as the increment of the proportion of participants choosing the dominated alternative as the best performing one.

An additional experiment was conducted to evaluate whether this unexpected effect was due to a negative formulation of the meaning of the slacks. The results show that a positive explanation of the slacks does not affect the percentage of participants considering the dominated DMU as the best performing one. This suggests that further research for understanding the interpretation and use of the slacks information should be conducted.

The rest of the paper is structured as follows: Section 2 depicts the theoretical background of our research, outlining the decoy effect in choice tasks as well as the basic aspects of relative performance evaluation with DEA. The hypotheses are derived in Section 3, and the method to investigate these hypotheses is described in Section 4. Section 5 presents the results of the study, addressing the decoy effect on the one side and the potential of DEA as a debiasing mechanism on the other side. Section 6 concludes the paper.

Section snippets

Managerial issues related to performance evaluation and the role of DEA

Performance has been defined as a social construct that acquires its meaning within a decision-making context. It is a complex concept that refers not only to the actions performed, but also to their results in comparison to benchmarks. Without such a comparison, it is almost impossible to qualify performance as good or bad. The inherent complexity of the performance concept is augmented by the fact that each decision maker may interpret performance data differently (Lebas & Euske, 2007). To

Hypotheses and predictions

The present study investigates (i) the existence of the decoy effect in a hypothetical setting of relative performance evaluation and (ii) the potential of the DEA approach to reduce this effect. Thanks to the comment of a reviewer, it should be clarified here that both scenarios (i) and (ii) address utility determination, while scenario (ii) also deals with (DEA) efficiency determination. Utility determination and efficiency determination can be understood as two different evaluation levels

Participants and design

Bachelor students (N = 482) taking introductory management control and business accounting courses at a German university received a performance report during a lecture. Eight participants did not complete the task and their questionnaires were therefore eliminated from further analysis. The basic design variables were: (i) decoy type (R or RF), (ii) decoy target (a or b), and (iii) DEA analysis (with or without DEA results). Students were randomly assigned to the different treatments and

Results

Consistent with previous research, the results show that the inclusion of a decoy in a performance evaluation context increments the proportion of participants preferring the target alternative. As shown in Fig. 2, approximately 60 percent of the participants in the two control conditions chose DMU a as the best performing DMU. This proportion increased when an inferior alternative similar to a was included in the report and decreased when the inferior option was similar to b. The decoy effect

Discussion

The present study offers an analysis of the decoy effect in a performance evaluation context, focusing on the debiasing characteristics of DEA results. The traditional experimental setting commonly used in consumer behavior was slightly modified with the aim of adapting it to a relative performance evaluation case. To the best of our knowledge, it constitutes the first attempt to link the DEA approach to the decoy effect. The results of the experiments can be summarized as follows:

  • The first

Acknowledgments

The authors would like to acknowledge Joseph Paradi for his valuable advice at the 10th International Conference on DEA, which contributed to the development of the main research question of this paper.

The authors would also like to thank two anonymous reviewers for their very helpful comments.

This work was supported by the Deutsche Forschungsgemeinschaft (DFG) in the context of the research project “Advanced Data Envelopment Analysis” under Grant DY 10/5-2 and AH 90/3-2.

References (55)

  • KaupB. et al.

    Processing negated sentences with contradictory predicates: Is a door that is not open mentally closed?

    Journal of Pragmatics

    (2006)
  • LevinI.P. et al.

    All frames are not created equal: A typology and critical analysis of framing effects

    Organizational Behavior and Human Decision Processes

    (1998)
  • MortensenT. et al.

    Students as surrogates for practicing accountants: Further evidence

    Accounting Forum

    (2012)
  • MortensonM.J. et al.

    Operational research from Taylorism to Terabytes: A research agenda for the analytics age

    European Journal of Operational Research

    (2015)
  • PageL. et al.

    Last shall be first: A field study of biases in sequential performance evaluation on the Idol series

    Journal of Economic Behavior & Organization

    (2010)
  • PettiboneJ.C. et al.

    Examining models of nondominated decoy effects across judgment and choice

    Organizational Behavior and Human Decision Processes

    (2000)
  • SlaughterJ.E. et al.

    The decoy effect as a covert influence tactic

    Journal of Behavioral Decision Making

    (2011)
  • Alia-KleinN. et al.

    What is in a word? No versus yes differentially engage the lateral orbitofrontal cortex

    Emotion

    (2007)
  • ArielyD.

    Predictably Irrational: The Hidden Forces that Shape our Decisions

    (2009)
  • BankerR.D. et al.

    An empirical investigation of an incentive plan that includes nonfinancial performance measures

    The Accounting Review

    (2000)
  • BankerR.D. et al.

    The balanced scorecard: Judgmental effects of performance measures linked to strategy

    The Accounting Review

    (2004)
  • ChengM.M. et al.

    The differential improvement effects of the strategy map on scorecard perspectives on managers’ strategic judgments

    The Accounting Review

    (2012)
  • ColettiA.L. et al.

    The effect of control systems on trust and cooperation in collaborative environments

    The Accounting Review

    (2005)
  • ConnollyT. et al.

    Regret salience and accountability in the decoy effect

    Judgment and Decision Making

    (2013)
  • CooperW.W. et al.

    Data envelopment analysis: A comprehensive text with models, applications, references and DEA-solver software

    (2007)
  • DamischL. et al.

    Olympic medals as fruits of comparison? Assimilation and contrast in sequential performance judgments

    Journal of Experimental Psychology: Applied

    (2006)
  • DyckhoffH.

    Produktionstheorie: Grundzüge industrieller Produktionswirtschaft (Production theory: Basics of industrial production management)

    (2006)
  • Cited by (8)

    • Enhancing supply chain decisions with consumers’ behavioral factors: An illustration of decoy effect

      2020, Transportation Research Part E: Logistics and Transportation Review
      Citation Excerpt :

      Studies of the decoy effect have shown on how consumer decisions are affected by a decoy in many experiments and real-world applications, both in physical and virtual products. For example, Ahn and Vazquez Novoa (2016) evaluated how the decoy effect affected performance evaluation. Some recent studies have turned to intertemporal choices, probability choices, and choices under risks.

    • Decoy effect in food appearance, traceability, and price: Case of consumer preference for pork hindquarters

      2020, Journal of Behavioral and Experimental Economics
      Citation Excerpt :

      Pettibone and Wedell (2000) and Kim, Kim, Lee, Kim and Hyde (2019) found that a decoy changes the range and sequence of product attributes and increases the preference order of the inferior attribute of the target product from the worst to the second worst, so that the inferior attribute of the target product becomes relatively superior. Wedell and Pettibone (1996) and Ahn and Novoa (2016) suggested that a decoy added to a choice set provides consumers with a purchasing reference point and changes consumer judgment on the relative values of various product options in the choice set. Hamilton, Hong and Chernev (2007) and Trueblood and Pettibone (2017) found that a decoy changes the rate of substitution among product attributes and increases the weights of superior attributes of the target product.

    View all citing articles on Scopus
    View full text