Skip to main content
SpringerLink
Log in

Reactions to Different Types of Forced Distribution Performance Evaluation Systems

  • Published:
Journal of Business and Psychology Aims and scope Submit manuscript

Abstract

Purpose

We isolate and describe four key elements that distinguish different forms of forced distribution systems (FDS). These key elements are the consequences for low performers, differentiation of rewards for top performers, frequency of feedback, and comparison group size. We examine how these elements influence respondents’ attraction to FDS.

Design/methodology/approach

Undergraduate students (n = 163) completed a policy capturing study designed to determine how these four FDS elements influence their attraction to FDS. We examine the relative importance of these elements that most influence attraction to different FDS, as well as individual attributes (i.e., cognitive ability, gender, and major) that may affect those preferences.

Findings

Respondents were most attracted to systems with less stringent treatment of low performers, high differentiation of rewards, frequent feedback and large comparison groups. Consequences for low performers were nearly twice as influential as any other element. Respondents with higher cognitive ability favored high reward differentiation and males were less affected by stringent consequences for low performers.

Implications

Before practitioners implement FDS, it would be prudent to consider all four elements examined in this study—with the treatment of low performers being the most salient issue. Future accounts of FDS should clarify the nature of these elements when reporting on FDS. Such precision will be useful in generating a knowledge base on FDS.

Originality/value

We add precision to the discussion of FDS by identifying four key elements. This is one of the first studies to examine perceptions of FDS from a ratee perspective.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Policy capturing uses regression techniques to capture the cognitive processes underlying judgments. The method has been used to study variety of decision-making processes within organizations (see Karren and Barringer 2002 for a listing of policy capturing studies appearing in top-tier journals), including organizational attraction, job search, and job termination decisions (e.g. Aiman-Smith et al. 2001; Cable and Judge 1994; Rousseau and Anton 1988). For more information on policy capturing, see Karren and Barringer (2002) or Aiman-Smith et al. (2002).

  2. We did not hypothesize interactions, and post hoc analyses revealed no statistically significant interactions were present.

References

  • Adams, J. S. (1965). Inequity in social exchange. In L. Berkowitz (Ed.), Advances in experimental psychology (Vol. 2, pp. 267–299). New York: Academic Press.

    Google Scholar 

  • Aiman-Smith, L., Bauer, T. N., & Cable, D. M. (2001). Are you attracted? Do you intend to pursue? A recruiting policy capturing study. Journal of Business and Psychology, 16, 219–237. doi:10.1023/A:1011157116322.

    Article  Google Scholar 

  • Aiman-Smith, L., Scullen, S. E., & Barr, S. H. (2002). Conducting studies of decision making in organizational contexts: A tutorial for policy-capturing and other regression-based techniques. Organizational Research Methods, 5, 388–414. doi:10.1177/109442802237117.

    Article  Google Scholar 

  • Arvey, R. D., & Murphy, K. R. (1998). Performance evaluation in work settings. Annual Review of Psychology, 49, 141–168. doi:10.1146/annurev.psych.49.1.141.

    Article  PubMed  Google Scholar 

  • Axelrod, B., Handfield-Jones, H., & Michaels, E. (2002). A new game plan for C players. Harvard Business Review, 83, 80–88.

    Google Scholar 

  • Bates, S. (2003). Forced ranking. HRMagazine, 48, 62–68. (June).

    Google Scholar 

  • Bloom, M. (1999). The performance effects of pay dispersion on individuals and organizations. Academy of Management Journal, 42, 25–40. doi:10.2307/256872.

    Article  Google Scholar 

  • Blume, B. D., Baldwin, T. T., & Rubin, R. S. (2005). Forced ranking: Who is attracted to it? A study of performance management system preferences. Paper presented at the annual meeting of the Academy of Management, Honolulu.

  • Bossidy, L., & Charan, R. (2002). Execution: The discipline of getting things done. New York: Random House.

    Google Scholar 

  • Breaugh, J. A., & Starke, M. (2000). Research on employee recruitment: So many studies, so many remaining questions. Journal of Management, 26, 405–434. doi:10.1177/014920630002600303.

    Article  Google Scholar 

  • Bretz, R. D., & Judge, T. A. (1994). The role of human resource systems in the job applicant decision processes. Journal of Management, 20, 531–551. doi:10.1016/0149-2063(94)90001-9.

    Article  Google Scholar 

  • Bretz, R. D., Jr., Milkovich, G. T., & Read, W. (1992). The current state of performance appraisal research and practice: Concerns, directions, and implications. Journal of Management, 18, 321–352.

    Google Scholar 

  • Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage.

    Google Scholar 

  • Cable, D. M., & Judge, T. A. (1994). Pay preferences and job search decisions: A person-organization fit perspective. Personnel Psychology, 47, 317–348. doi:10.1111/j.1744-6570.1994.tb01727.x.

    Article  Google Scholar 

  • Cable, D. M., & Judge, T. A. (1996). Person-organization fit, job choice decisions, and organizational entry. Organizational Behavior and Human Decision Processes, 67, 294–311. doi:10.1006/obhd.1996.0081.

    Article  Google Scholar 

  • Cawley, B. D., Keeping, L. M., & Levy, P. E. (1998). Participation in the performance appraisal process and employee reactions: A meta-analytic review of field investigations. The Journal of Applied Psychology, 83, 615–633. doi:10.1037/0021-9010.83.4.615.

    Article  Google Scholar 

  • Colquitt, J. A., Conlon, D. E., Wesson, M. J., Porter, C. O. L. H., & Ng, K. Y. (2001). Justice at the millennium: A meta-analytic review of 25 years of organizational justice research. The Journal of Applied Psychology, 86, 425–445. doi:10.1037/0021-9010.86.3.425.

    Article  PubMed  Google Scholar 

  • Colvin, G. (2001). We can’t all be above average. Fortune, 144, 3.

    Google Scholar 

  • Davison, B. (2003). Management span of control: How wide is too wide? The Journal of Business Strategy, 24, 22–29. doi:10.1108/02756660310494854.

    Article  Google Scholar 

  • Dodrill, C. B. (1981). An economical method for the evaluation of general intelligence in adults. Journal of Consulting and Clinical Psychology, 4, 668–673. doi:10.1037/0022-006X.49.5.668.

    Article  Google Scholar 

  • Dodrill, C. B. (1983). Long term reliability of the Wonderlic Personnel Test. Journal of Consulting and Clinical Psychology, 51, 316–317. doi:10.1037/0022-006X.51.2.316.

    Article  Google Scholar 

  • Dodrill, C. B., & Warner, M. H. (1988). Further studies of the Wonderlic Personnel Test as a brief measure of intelligence. Journal of Consulting and Clinical Psychology, 56, 145–147. doi:10.1037/0022-006X.56.1.145.

    Article  PubMed  Google Scholar 

  • Dreher, G., & Dougherty, T. W. (2002). Human resource strategy: A behavioral perspective for the general manager. Irwin/McGraw-Hill.

  • Duffy, K. E., & Webber, R. E. (1974). On “relative” rating systems. Personnel Psychology, 27, 307–311. doi:10.1111/j.1744-6570.1974.tb01536.x.

    Article  Google Scholar 

  • Folger, R., & Konovsky, M. A. (1989). Effects of procedural and distributive justice on reactions to pay raise decisions. Academy of Management Journal, 32, 115–130. doi:10.2307/256422.

    Article  Google Scholar 

  • Gary, L. (2001). The controversial practice of forced ranking. Harvard Management Update, October, 1–2.

  • Gerhart, B., & Milkovich, G. T. (1990). Organizational differences in managerial compensation and financial performance. Academy of Management Journal, 33, 663–691. doi:10.2307/256286.

    Article  Google Scholar 

  • Gerhart, B., & Milkovich, G. T. (1992). Employee compensation: Research and practice. In M. Dunnette & L. Hough (Eds.), Handbook of industrial and organizational psychology (Vol. 3, pp. 481–569). Palo Alto, CA: Consulting Psychologists Press.

  • Gladwell, M. (2002). The talent myth. Are smart people overrated? New Yorker (New York, N.Y.), (July), 22.

  • Goffin, R. D., Gellatly, I. R., Paunonen, S. V., Jackson, D. N., & Meyer, J. P. (1996). Criterion validation of two approaches to performance appraisal: The behavioral observation scale and the relative percentile method. Journal of Business and Psychology, 11, 23–34. doi:10.1007/BF02278252.

    Article  Google Scholar 

  • Grote, D. (2002). Forced ranking: Behind the scenes. Across the Board, 40–45 (Nov./Dec).

  • Grote, D. (2005). Forced ranking: Making performance management work. Boston: Harvard Business School.

    Google Scholar 

  • Guralnik, O., Rozmarin, E., & So, A. (2004). Forced distribution: Is it right for you? Human Resource Development Quarterly, 15, 339–345. doi:10.1002/hrdq.1107.

    Article  Google Scholar 

  • Hedge, J. W., & Teachout, M. S. (2000). Exploring the concept of acceptability as a criterion for evaluating performance measures. Group & Organization Management, 25, 22–44. doi:10.1177/1059601100251003.

    Article  Google Scholar 

  • Heneman, R. L. (1986). The relationship between supervisory ratings and results-oriented measures of performance: A meta-analysis. Personnel Psychology, 39, 811–826. doi:10.1111/j.1744-6570.1986.tb00596.x.

    Article  Google Scholar 

  • Hofmann, D. A. (1997). An overview of the logic and rationale of hierarchical linear models. Journal of Management, 23, 723–744. doi:10.1177/014920639702300602.

    Article  Google Scholar 

  • Huselid, M. A., Beatty, R. W., & Becker, B. E. (2005). A players or a positions? Harvard Business Review, 83, 110–117.

    PubMed  Google Scholar 

  • Jawahar, I. M., & Williams, C. R. (1997). Where all the children are above average: The performance appraisal purpose effect. Personnel Psychology, 50, 905–925. doi:10.1111/j.1744-6570.1997.tb01487.x.

    Article  Google Scholar 

  • Judge, T. A., & Bretz, R. D. (1992). Effects of work values on job choice decisions. The Journal of Applied Psychology, 77, 261–271. doi:10.1037/0021-9010.77.3.261.

    Article  Google Scholar 

  • Judge, T. A., & Ferris, G. R. (1993). Social context of performance evaluation decisions. Academy of Management Journal, 36, 80–105. doi:10.2307/256513.

    Article  Google Scholar 

  • Jurgensen, C. E. (1978). Job preferences (what makes a job good or bad?). The Journal of Applied Psychology, 63, 267–276. doi:10.1037/0021-9010.63.3.267.

    Article  Google Scholar 

  • Kanfer, R. (1990). Motivational theory and industrial and organizational psychology. In M. D. Dunnette & L. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., pp. 75–105). Palo Alto, CA: Consulting Psychologist Press.

    Google Scholar 

  • Kanfer, R. M., & Heggestad, E. D. (1997). Motivational traits and skills: A person-centered approach to work motivation. In B. Staw & L. L. Cummings (Eds.), Research in organizational behavior (Vol. 19, pp. 1–56). Greenwich, CT: JAI Press.

    Google Scholar 

  • Karren, R. J., & Barringer, M. W. (2002). A review of the policy-capturing methodology in organizational research: Guidelines for research and practice. Organizational Research Methods, 5, 337–361. doi:10.1177/109442802237115.

    Article  Google Scholar 

  • Keeping, L. M., & Levy, P. E. (2000). Performance appraisal reactions: Measurement, modeling, and method bias. The Journal of Applied Psychology, 85, 708–723. doi:10.1037/0021-9010.85.5.708.

    Article  PubMed  Google Scholar 

  • Korsgaard, M. A., & Roberson, L. (1995). Procedural justice in performance evaluation: The role of instrumental and non-instrumental voice in performance appraisal discussions. Journal of Management, 21, 657–669. doi:10.1177/014920639502100404.

    Article  Google Scholar 

  • Kristof-Brown, A. L., Jansen, K. J., & Colbert, A. E. (2002). A policy capturing study of simultaneous effects of fit with jobs, groups, and organizations. The Journal of Applied Psychology, 87, 985–993. doi:10.1037/0021-9010.87.5.985.

    Article  PubMed  Google Scholar 

  • Kruger, A. N., & DeNisi, A. S. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254–284. doi:10.1037/0033-2909.119.2.254.

    Article  Google Scholar 

  • Landy, F. J., Barnes, J. L., & Murphy, K. R. (1978). Correlates of perceived performance and accuracy of performance evaluation. The Journal of Applied Psychology, 63, 751–754. doi:10.1037/0021-9010.63.6.751.

    Article  Google Scholar 

  • Lawler, E. E. (2002). The folly of forced ranking. Strategy + Business, 28, 28–32.

    Google Scholar 

  • Lawler, E. E. (2003). Reward practices and performance management effectiveness. Organizational Dynamics, 32, 396–404. doi:10.1016/j.orgdyn.2003.08.007.

    Article  Google Scholar 

  • Levinson, M. (2003). One tough job: How to find, fix or fire your poor performers; bad employees drain your IT organization and the company. Forced ranking can help you get tough—but at what cost? CIO, 17(3), 1–4.

    Google Scholar 

  • Levy, P. E., & Williams, J. R. (2004). The social context of performance appraisal: A review and framework for the future. Journal of Management, 30, 881–905. doi:10.1016/j.jm.2004.06.005.

    Google Scholar 

  • London, M. (2003). Job feedback. Upper Saddle River, NJ: Earlbaum.

    Google Scholar 

  • McBriarty, M. A. (1988). Performance appraisal: Some unintended consequences. Public Personnel Management, 17, 421–434.

    Google Scholar 

  • McKelvie, S. J. (1989). The Wonderlic Personnel Test: Reliability and validity in an academic setting. Psychological Reports, 65, 161–162.

    Google Scholar 

  • Mellor, S., Paley, M. J., & Holzworth, R. J. (1999). Fans’ judgments about the 1994–95 major league baseball players’ strike. Multivariate Behavioral Research, 34, 59–87. doi:10.1207/s15327906mbr3401_3.

    Article  Google Scholar 

  • Milne, J. L. (2002). Checking in: Survey shows frequence of employee performance reviews. Canadian Manager, 27, 2.

    Google Scholar 

  • Morrison, E. W., & Vancouver, J. B. (2000). Within-person analysis of information seeking: The effects of perceived costs and benefits. Journal of Management, 26, 119–137. doi:10.1016/S0149-2063(99)00040-9.

    Article  Google Scholar 

  • Mount, M. K. (1984). Satisfaction with a performance appraisal system and appraisal discussion. Journal of Occupational Behavior, 5, 271–279. doi:10.1002/job.4030050404.

    Article  Google Scholar 

  • Murphy, K. R., & Cleveland, J. (1995). Understanding performance appraisal: Social, organizational, and goal-based perspectives. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Nathan, B. R., & Alexander, R. A. (1988). A comparison of criteria for test validation: A metal-analytic investigation. Personnel Psychology, 41, 517–535. doi:10.1111/j.1744-6570.1988.tb00642.x.

    Article  Google Scholar 

  • Olson C. A., & Davis G. M. (2003). Pros and cons of forced ranking and other relative performance ranking systems. Society for Human Resource Management Legal Report, March.

  • Organ, D. W. (1990). The subtle significance of job satisfaction. Clinical Laboratory Management Review, 4, 94–98.

    Google Scholar 

  • Pfeffer, J. (1998). The human equation: Building profits by putting people first. Boston: Harvard Business School.

    Google Scholar 

  • Pfeffer, J. (2001). Fighting the war for talent is hazardous to your organization’s health. Organizational Dynamics, 29, 248–259. doi:10.1016/S0090-2616(01)00031-6.

    Article  Google Scholar 

  • Pfeffer, J., & Sutton, R. I. (2006). Evidence-based management. Harvard Business Review, 84, 62–74.

    PubMed  Google Scholar 

  • Robinson, J. L., & Lipman-Blumen, J. (2003). Leadership behavior of male and female managers, 1984–2002. Journal of Education for Business, 79, 28–33.

    Article  Google Scholar 

  • Roch, S. G., Sternburgh, A. M., & Caputo, P. M. (2007). Absolute vs. relative performance rating formats: Implications for fairness and organizational justice. International Journal of Selection and Assessment, 15, 302–316. doi:10.1111/j.1468-2389.2007.00390.x.

    Article  Google Scholar 

  • Rousseau, D. M., & Anton, R. J. (1988). Fairness and implied contract obligations in job terminations: A policy capturing study. Human Performance, 1, 273–289. doi:10.1207/s15327043hup0104_4.

    Article  Google Scholar 

  • Rynes, S. L., Brown, K. G., & Colbert, A. E. (2002). Seven common misconceptions about human resource practices: Research findings versus practitioner beliefs. Academy of Management Executive, 16, 92–103.

    Google Scholar 

  • Schleicher, D. J., Bull, R. A., & Green, S. G. (2009). Rater reactions to forced distribution rating systems. Journal of Management, (in press).

  • Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274. doi:10.1037/0033-2909.124.2.262.

    Article  Google Scholar 

  • Schwab, D. P., Rynes, S. L., & Aldag, R. J. (1987). Theories and research on job search and choice. In K. M. Rowland & G. R. Ferris (Eds.), Research in personnel and human resource management (Vol. 5, pp. 126–166). Greenwich, CT: JAI Press.

    Google Scholar 

  • Scullen, S. E., Bergey, P. K., & Aiman-Smith, L. (2005). Forced distribution rating systems and the improvement of workforce potential: A baseline simulation. Personnel Psychology, 58, 1–32. doi:10.1111/j.1744-6570.2005.00361.x.

    Article  Google Scholar 

  • Shirouzu, N. (2001). Ford stops using letter rankings to rate workers. Wall Street Journal, B.1 (July 11).

  • Taylor, M. S., Masterson, S. S., Renard, M. K., & Tracy, K. B. (1998). Managers reactions to procedurally just performance management systems. Academy of Management Journal, 41, 568–579. doi:10.2307/256943.

    Article  Google Scholar 

  • Tichy, N. M., & Sherman, S. (2001). Control your destiny or someone else will: Lessons in mastering change-from the principles jack welch is using to revolutionize GE. New York: HarperCollins.

    Google Scholar 

  • Trank, C. Q., Rynes, S. L., & Bretz, R. D. (2002). Attracting applicants in the war for talent: differences in work preferences among high achievers. Journal of Business and Psychology, 16, 331–345. doi:10.1023/A:1012887605708.

    Article  Google Scholar 

  • Wagner, S. H., & Goffin, R. D. (1997). Differences in accuracy of absolute and comparative performance appraisal methods. Organizational Behavior and Human Decision Processes, 70, 95–103. doi:10.1006/obhd.1997.2698.

    Article  Google Scholar 

  • Walster, E., Walster, G. W., & Scott, W. G. (1978). Equity: Theory and research. Boston: Allyn & Bacon.

    Google Scholar 

  • Welch, J. F. (2001). Jack: Straight from the gut. New York: Warner Books, Inc.

    Google Scholar 

  • Wonderlic Personnel Test Manual. (1983). Northfield, IL: E.F. Wonderlic & Associates.

  • Wright, R. P. (2002). Perceptual dimensions of performance management systems in the eyes of different sample categories. International Journal of Management, 19, 184–193.

    Google Scholar 

  • Zetlin, M. (1994). Up for review. Sales & Marketing Management, 146, 82–86.

    Google Scholar 

Download references

Acknowledgments

Previous versions of this paper were presented at the 2006 Academy of Management Conference in Atlanta, GA and the 2007 Society for Industrial and Organizational Psychology in Philadelphia, PA. We gratefully acknowledge the helpful comments of George Dreher.

Author information

Authors and Affiliations

  1. School of Management, University of Michigan, 3119 WSW Bldg., Flint, MI, 48502, USA

    Brian D. Blume

  2. Kelley School of Business, Indiana University, Bloomington, IN, USA

    Timothy T. Baldwin

  3. Kellstadt Graduate School of Business, DePaul University, Chicago, IL, USA

    Robert S. Rubin

Authors
  1. Brian D. Blume
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Timothy T. Baldwin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert S. Rubin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brian D. Blume.

Additional information

Received and reviewed by former editor, George Neuman.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Blume, B.D., Baldwin, T.T. & Rubin, R.S. Reactions to Different Types of Forced Distribution Performance Evaluation Systems. J Bus Psychol 24, 77–91 (2009). https://doi.org/10.1007/s10869-009-9093-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10869-009-9093-5

Keywords

Search

Navigation