The decoy effect in relative performance evaluation and the debiasing role of DEA
Introduction
Despite being described as a discipline aiming at facilitating decision making, operations research (OR) has largely omitted behavioral issues. Most recently, this deficit has been addressed by Hämäläinen, Luoma, and Saarinen (2013). They comprehensively discuss the high potential of behavioral OR for advancing the practical benefit of normative OR methods and propose nine topics for a respective research agenda. The present paper refers particularly to the topic of behavioral/cognitive aspects, since it stresses performance evaluation biases related to the so-called decoy effect.
The decoy effect is a special kind of context effect, which refers to the influence that contextual variables have on decision making. Overwhelming evidence exists demonstrating that context is likely to influence performance evaluation (see, e.g., Damisch et al., 2006, Page and Page, 2010). For example, evaluators’ preferences are biased not only by the past performance of the decision making unit (DMU), but also by the performance of other DMUs under analysis. With regard to the latter aspect, the decoy effect implies that the inclusion of a dominated alternative – the decoy – can influence the choice between the non-dominated alternatives. Concretely, the probability of preferring the target alternative, which is the non-dominated option that is most similar to the decoy, may increase. The existence of this effect has repeatedly been confirmed by research on consumer behavior, showing that customers tend to prefer the target alternative (Ariely, 2009, Huber et al., 1982, Wedell and Pettibone, 1996).
The research settings used for studying consumer behavior strongly resemble relative performance evaluation cases where alternatives are compared on a utility function level. Furthermore, these research settings are constructed in a way that they differentiate between non-dominated (i.e., efficient) and dominated (i.e., inefficient) units like in the context of Data Envelopment Analysis (DEA). Against this background, two questions arise: (i) does the decoy effect also occur in cases where the relative performance of alternatives is evaluated, and (ii) to what extent can the application of DEA – namely the incorporation of respective efficiency scores and the mention of existing slacks – act as a debiasing tool?
To shed light on these questions, we conducted a vignette-based experiment with bachelor students taking management control and business accounting courses. For analyzing whether relative performance evaluation may also be affected by a decoy effect, the well-known experimental setting from the area of consumer behavior was adapted to the performance context. The results show that adding a dominated DMU to the set of DMUs augments the attractiveness of the target DMU. Therefore, a decoy effect can be observed when DMUs are compared against each other.
In a second step, the role of reporting DEA results as a debiasing procedure for the identified decoy effect was considered. Participants in the corresponding treatments were provided with additional information about DEA efficiency scores and slacks. The results indicate that the scores discriminating between efficient and inefficient DMUs can significantly reduce the decoy effect in a relative performance evaluation context. The mention of the existence of slacks for distinguishing between strong and weak efficient DMUs also contributes to avoiding the decoy effect, but it is associated to unexpected effects, such as the increment of the proportion of participants choosing the dominated alternative as the best performing one.
An additional experiment was conducted to evaluate whether this unexpected effect was due to a negative formulation of the meaning of the slacks. The results show that a positive explanation of the slacks does not affect the percentage of participants considering the dominated DMU as the best performing one. This suggests that further research for understanding the interpretation and use of the slacks information should be conducted.
The rest of the paper is structured as follows: Section 2 depicts the theoretical background of our research, outlining the decoy effect in choice tasks as well as the basic aspects of relative performance evaluation with DEA. The hypotheses are derived in Section 3, and the method to investigate these hypotheses is described in Section 4. Section 5 presents the results of the study, addressing the decoy effect on the one side and the potential of DEA as a debiasing mechanism on the other side. Section 6 concludes the paper.
Section snippets
Managerial issues related to performance evaluation and the role of DEA
Performance has been defined as a social construct that acquires its meaning within a decision-making context. It is a complex concept that refers not only to the actions performed, but also to their results in comparison to benchmarks. Without such a comparison, it is almost impossible to qualify performance as good or bad. The inherent complexity of the performance concept is augmented by the fact that each decision maker may interpret performance data differently (Lebas & Euske, 2007). To
Hypotheses and predictions
The present study investigates (i) the existence of the decoy effect in a hypothetical setting of relative performance evaluation and (ii) the potential of the DEA approach to reduce this effect. Thanks to the comment of a reviewer, it should be clarified here that both scenarios (i) and (ii) address utility determination, while scenario (ii) also deals with (DEA) efficiency determination. Utility determination and efficiency determination can be understood as two different evaluation levels
Participants and design
Bachelor students (N = 482) taking introductory management control and business accounting courses at a German university received a performance report during a lecture. Eight participants did not complete the task and their questionnaires were therefore eliminated from further analysis. The basic design variables were: (i) decoy type (R or RF), (ii) decoy target (a or b), and (iii) DEA analysis (with or without DEA results). Students were randomly assigned to the different treatments and
Results
Consistent with previous research, the results show that the inclusion of a decoy in a performance evaluation context increments the proportion of participants preferring the target alternative. As shown in Fig. 2, approximately 60 percent of the participants in the two control conditions chose DMU a as the best performing DMU. This proportion increased when an inferior alternative similar to a was included in the report and decreased when the inferior option was similar to b. The decoy effect
Discussion
The present study offers an analysis of the decoy effect in a performance evaluation context, focusing on the debiasing characteristics of DEA results. The traditional experimental setting commonly used in consumer behavior was slightly modified with the aim of adapting it to a relative performance evaluation case. To the best of our knowledge, it constitutes the first attempt to link the DEA approach to the decoy effect. The results of the experiments can be summarized as follows:
- •
The first
Acknowledgments
The authors would like to acknowledge Joseph Paradi for his valuable advice at the 10th International Conference on DEA, which contributed to the development of the main research question of this paper.
The authors would also like to thank two anonymous reviewers for their very helpful comments.
This work was supported by the Deutsche Forschungsgemeinschaft (DFG) in the context of the research project “Advanced Data Envelopment Analysis” under Grant DY 10/5-2 and AH 90/3-2.
References (55)
- et al.
Measuring the relative balance of DMUs
European Journal of Operational Research
(2012) - et al.
Seeking subjective dominance in multidimensional space: An explanation of the asymmetric dominance effect
Organizational Behavior and Human Decision Processes
(1995) - et al.
A path model examining the relations among strategic performance measurement system characteristics, organizational justice, and extra- and in-role performance
Accounting, Organizations and Society
(2009) - et al.
The role of management control systems in planned organizational change: An analysis of two organizations
Accounting, Organizations and Society
(2007) - et al.
Data Envelopment Analysis (DEA): Thirty years on
European Journal of Operational Research
(2009) - et al.
Pitfalls and protocols in DEA
European Journal of Operational Research
(2001) The effect of comprehensive performance measurement systems on role clarity, psychological empowerment and managerial performance
Accounting, Organizations and Society
(2008)- et al.
Does understanding negation entail affirmation? An examination of negated metaphors
Journal of Pragmatics
(2006) Context-dependent selection: The effects of decoy and phantom job candidates
Organizational Behavior and Human Decision Processes
(1996)- et al.
Assessing empirical research in managerial accounting: A value-based management perspective
Journal of Accounting and Economics
(2001)
Processing negated sentences with contradictory predicates: Is a door that is not open mentally closed?
Journal of Pragmatics
All frames are not created equal: A typology and critical analysis of framing effects
Organizational Behavior and Human Decision Processes
Students as surrogates for practicing accountants: Further evidence
Accounting Forum
Operational research from Taylorism to Terabytes: A research agenda for the analytics age
European Journal of Operational Research
Last shall be first: A field study of biases in sequential performance evaluation on the Idol series
Journal of Economic Behavior & Organization
Examining models of nondominated decoy effects across judgment and choice
Organizational Behavior and Human Decision Processes
The decoy effect as a covert influence tactic
Journal of Behavioral Decision Making
What is in a word? No versus yes differentially engage the lateral orbitofrontal cortex
Emotion
Predictably Irrational: The Hidden Forces that Shape our Decisions
An empirical investigation of an incentive plan that includes nonfinancial performance measures
The Accounting Review
The balanced scorecard: Judgmental effects of performance measures linked to strategy
The Accounting Review
The differential improvement effects of the strategy map on scorecard perspectives on managers’ strategic judgments
The Accounting Review
The effect of control systems on trust and cooperation in collaborative environments
The Accounting Review
Regret salience and accountability in the decoy effect
Judgment and Decision Making
Data envelopment analysis: A comprehensive text with models, applications, references and DEA-solver software
Olympic medals as fruits of comparison? Assimilation and contrast in sequential performance judgments
Journal of Experimental Psychology: Applied
Produktionstheorie: Grundzüge industrieller Produktionswirtschaft (Production theory: Basics of industrial production management)
Cited by (8)
Enhancing supply chain decisions with consumers’ behavioral factors: An illustration of decoy effect
2020, Transportation Research Part E: Logistics and Transportation ReviewCitation Excerpt :Studies of the decoy effect have shown on how consumer decisions are affected by a decoy in many experiments and real-world applications, both in physical and virtual products. For example, Ahn and Vazquez Novoa (2016) evaluated how the decoy effect affected performance evaluation. Some recent studies have turned to intertemporal choices, probability choices, and choices under risks.
Decoy effect in food appearance, traceability, and price: Case of consumer preference for pork hindquarters
2020, Journal of Behavioral and Experimental EconomicsCitation Excerpt :Pettibone and Wedell (2000) and Kim, Kim, Lee, Kim and Hyde (2019) found that a decoy changes the range and sequence of product attributes and increases the preference order of the inferior attribute of the target product from the worst to the second worst, so that the inferior attribute of the target product becomes relatively superior. Wedell and Pettibone (1996) and Ahn and Novoa (2016) suggested that a decoy added to a choice set provides consumers with a purchasing reference point and changes consumer judgment on the relative values of various product options in the choice set. Hamilton, Hong and Chernev (2007) and Trueblood and Pettibone (2017) found that a decoy changes the rate of substitution among product attributes and increases the weights of superior attributes of the target product.
A new approach for ranking efficient DMUs with data envelopment analysis
2020, World Journal of EngineeringCompromise Effect in Food Consumer Choices in China: An Analysis on Pork Products
2020, Frontiers in Psychology