Abstract
The Brief Fear of Negative Evaluation Scale (BFNE; Leary Personality and Social Psychology Bulletin, 9, 371–375, 1983) assesses fear and worry about receiving negative evaluation from others. Rodebaugh et al. Psychological Assessment, 16, 169–181, (2004) found that the BFNE is composed of a reverse-worded factor (BFNE-R) and straightforwardly-worded factor (BFNE-S). Further, they found the BFNE-S to have better psychometric properties and provide more information than the BFNE-R. Currently there is a lack of research regarding the measurement invariance of the BFNE-S across gender and ethnicity with respect to item thresholds. The present study uses item response theory (IRT) to test the BFNE-S for differential item functioning (DIF) related to gender and ethnicity (White, Asian, and Black). Six data sets consisting of clinical, community, and undergraduate participants were utilized (N = 2,109). The factor structure of the BFNE-S was confirmed using categorical confirmatory factor analysis, IRT model assumptions were tested, and the BFNE-S was evaluated for DIF. Item nine demonstrated significant non-uniform DIF between White and Black participants. No other items showed significant uniform or non-uniform DIF across gender or ethnicity. Results suggest the BFNE-S can be used reliably with men and women and Asian and White participants. More research is needed to understand the implications of using the BFNE-S with Black participants.
Similar content being viewed by others
Notes
To test whether the results of the DIF analyses were dependent on anchor items were-ran the analyses for both gender and ethnicity using items three and eight which did not exhibit LD. The results from the DIF analyses for using items three and eight versus the items described above were identical.
References
Adams, C. E., Myers, V. H., Barbera, B. L., & Brantley, P. J. (2011). The role of fear of negative evaluation in predicting depression and quality of life 4 years after bariatric surgery in women. Psychology, 2, 150–154. doi:10.4236/psych.2011.23024.
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B, 57, 289–300. doi:10.2307/2346101.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246. doi:10.1037/0033-2909.107.2.238.
Blanchard, J. J., Mueser, K. T., & Bellack, A. S. (1998). Anhedonia, positive and negative affect, and social functioning in schizophrenia. Schizophrenia Bulletin, 24, 413–424. http://schizophreniabulletin.oxfordjournals.org/.
Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: application of an EM algorithm. Psychometrika, 46, 443–459. doi:10.1007/BF02293801.
Breslau, J., Aguilar-Gaxiola, S., Kendler, K. S., Su, M., Williams, D., & Kessler, R. C. (2006). Specifying race-ethnic differences in risk for psychiatric disorder in a USA national sample. Psychological Medicine, 36, 57–68. doi:10.1017/S0033291705006161.
Brown, P. J., & Roose, S. P. (2011). Age and anxiety and depressive symptoms: the effect on domains of quality of life. International Journal of Geriatric Psychiatry, 26, 1260–1266.
Cai, L. (2008). SEM of another flavor: Two new applications of the supplemented EM algorithm. British Journal of Mathematical and Statistical Psychology, 61, 309–329. doi:10.1348/000711007X249603.
Cai, L. (2012). flexMIRT™ version 1.88: A numerical engine for multilevel item factor analysis and test scoring. [Computer Software]. Seattle: Vector Psychometric Group.
Cai, L., Thissen, D., & du Toit, S. H. C. (2013). IRTPRO:Flexible, multidimensional, multiple categorical IRT modeling [Computer software]. Chicago: Scientific Software International.
Carleton, N. R., Collimore, K. C., McCabe, R. E., & Antony, M. M. (2011). Addressing revisions to the brief fear of negative evaluation scale: measuring fear of negative evaluation across anxiety and mood disorders. Journal of Anxiety Disorders, 25, 822–828. doi:10.1016/j.janxdis.2011.04.002.
Carleton, N. R., McCreary, D. R., Norton, P. J., & Asmundson, G. G. (2006). Brief fear of negative evaluation scale-revised. Depression and Anxiety, 23, 297–303. doi:10.1002/da.20142.
Chen, W.-H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265–289. doi:10.2307/1165285.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233–255. doi:10.1207/S15328007SEM0902_5.
Collins, K. A., Westra, H. A., Dozois, D. A., & Stewart, S. H. (2005). The validity of the brief version of the fear of negative evaluation scale. Journal of Anxiety Disorders, 19, 345–359. doi:10.1016/j.janxdis.2004.02.003.
De Ayala, R. J. (2009). The theory and practice of item response theory. New York, NY: The Guilford Press. http://www.guilford.com/.
DiNardo, P. A., Brown, T. A., & Barlow, D. H. (1994). Anxiety disorders interview schedule for DSM-IV: Lifetime version (ADIS-IV-L). San Antonio: The Psychological Corporation.
Elosua, P. (2011). Assessing measurement equivalence in ordered-categorical data. Psicológica, 32, 403–421. http://www.uv.es/revispsi/paraARCHIVES/2011.html.
First, M.B., Spitzer, R.L., Gibbon, M., et al. Structured Clinical Interview for DSM-IV Axis I Disorders (SCID), Clinician Version: Administration Booklet. Washington, DC: American Psychiatric Press; 1996. http://www.appi.org/Home
Gilbert, N., & Meyer, C. (2005). Fear of negative evaluation and the development of eating psychopathology: a longitudinal study among nonclinical women. International Journal of Eating Disorders, 37, 307–312. doi:10.1002/eat.20105.
Haikal, M., & Hong, R. Y. (2010). The effects of social evaluation and looming threat on self-attentional biases and social anxiety. Journal of Anxiety Disorders, 24, 345–352. doi:10.1016/j.janxdis.2010.01.007.
Hambrick, J. P., Rodebaugh, T. L., Balsis, S., Woods, C. M., Mendez, J. L., & Heimberg, R. G. (2010). Cross-ethnic measurement equivalence of measures of depression, social anxiety, and worry. Assessment, 17, 155–171. doi:10.1177/1073191109350158.
Harpole, J. K., & Woods, C. M. (2013). The impact of local dependence on differential item functioning. Paper presented at the annual meeting of the Psychometric Society, Arnhem, Netherlands.
Harpole, J. K., & Woods, C. M. (2014). The impact of local dependence on differential item functioning. Manuscript submitted for publication.
Heimberg, R. G., Brozovich, F. A., & Rapee, R. M. (2010). A cognitive-behavioral model of social anxiety disorder: update and extension. In S. G. Hofmann & P. M. DiBartolo (Eds.), Social anxiety: Clinical, developmental, and social perspectives. New York: Elsevier. doi:10.1016/B978-0-12-375096-9.00015-8.
Honaker, J., King, G., & Blackwell, M. (2011). Amelia II: A program for missing data. Journal of Statistical Software, 45, 1–47. http://www.jstatsoft.org
Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424–453. doi:10.1037/1082-989X.3.4.424.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. doi:10.1080/10705519909540118.
Langer, M. (2008). A reexamination of Lord’s Wald test for differential item functioning using item response theory and modern error estimation, Unpublished doctoral dissertation. University of North Carolina at Chapel Hill. http://dc.lib.unc.edu/cdm/singleitem/collection/etd/id/2084.
Leary, M. R. (1983). A brief version of the fear of negative evaluation scale. Personality and Social Psychology Bulletin, 9, 371–375. doi:10.1177/0146167283093007.
Levinson, C. A., & Rodebaugh, T. L. (2012). Social anxiety and eating disorders: the role of negative social evaluation fears. Eating Behaviors, 13, 27–35. doi:10.1016/j.eatbeh.2011.11.006.
Levinson, C. A., & Rodebaugh, T. L. (2011). Validation of the social appearance anxiety scale: factor, convergent, and divergent validity. Assessment, 18, 350–356. doi:10.1177/1073191111404808.
Levinson, C. A., Rodebaugh, T. L., Menatti, A., & Weeks, J. W. (2013). Development and validation of the Social Exercise and Anxiety Measure (SEAM): assessing fears, avoidance, and importance of social exercise. Journal of Psychopathology and Behavioral Assessment, 35, 244–253. doi:10.1007/s10862-012-9326-1.
Lewis, B. A., & O’Neill, H. (2000). Alcohol expectancies and social deficits relating to problem drinking among college students. Addictive Behaviors, 25, 295–299. doi:10.1016/S0306-4603(99)00063-5.
Liebowitz, M.R. (1987). Social phobia. Modern Problems of Pharmacopsychiatry, 22, 141–173. http://www.karger.com/BookSeries/Home/223929.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale: Lawrence Erlbaum.
Lubke, G. H., & Muthén, B. O. (2004). Applying multigroup confirmatory factor models for continuous outcomes to likert scale data complicates meaningful group comparisons. Structural Equation Modeling, 11, 514–534. doi:10.1207/s15328007sem1104_2.
MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130–149. doi:10.1037/1082-989X.1.2.130.
Mattick, R. P., & Clarke, J. C. (1998). Development and validation of measures of social phobia, scrutiny, fear, and social interaction anxiety. Behaviour Research and Therapy, 36, 455–470. doi:10.1016/S0005-7967(97)10031-6.
Maydeu-Olivares, A., & Joe, H. (2005). Limited-and full-information estimation and goodness-of-fit testing in 2n contingency tables. Journal of the American Statistical Association, 100, 1009–1020. doi:10.1198/016214504000002069.
Maydeu-Olivares, A., & Joe, H. (2006). Limited information goodness-of-fit testing in multidimensional contingency tables. Psychometrika, 71, 713–732. doi:10.1007/s11336-005-1295-9.
Meyer, T. J., Miller, M. L., Metzger, R. L., & Borkovec, T. D. (1990). Development and validation of the Penn State Worry Questionnaire. Behaviour Research and Therapy, 28, 487–495. doi:10.1016/0005-7967(90)90135-6.
Millsap, R. E. (2011). Statistical approaches to measurement invariance. New York: Routledge. http://www.routledge.com.
Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus user’s guide. Seventh Edition. Los Angeles, CA: Muthén & Muthén. http://www.statmodel.com.
Norton, P. J., & Weeks, J. W. (2009). A multi-ethnic examination of socioevaluative fears. Journal of Anxiety Disorders, 23, 904–908. doi:10.1016/j.janxdis.2009.05.008.
O’Connor, L. E., Berry, J. W., Weiss, J., & Gilbert, P. (2002). Guilt, fear, submission, and empathy in depression. Journal of Affective Disorders, 71, 19–27. doi:10.1016/S0165-0327(01)00408-6.
Okazaki, S. (1997). Sources of ethnic differences between Asian American and White American college students on measures of depression and social anxiety. Journal of Abnormal Psychology, 106, 52–60. doi:10.1037/0021-843X.106.1.52.
Orlando, M., & Thissen, D. (2003). Further investigation of the performance of S-χ 2: an item-fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27, 289–298. doi:10.1177/0146621603027004004.
Orlando, M., & Thissen, D. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50–64. doi:10.1177/01466216000241003.
Core Team, R. (2013). R: A Language and Environment for Statistical Computing [Computer Software]. Vienna: R Foundation for Statistical Computing.
Rodebaugh, T. L., Shumaker, E. A., Levinson, C. A., Fernandez, K. C., Langer, J. K., Lim, M. H., & Yarkoni, T. (2013). Interpersonal constraint conferred by generalized social anxiety disorder is evident on a behavioral economics task. Journal of Abnormal Psychology, 22, 39–44. doi:10.1037/a0030975.
Rodebaugh, T. L., Woods, C. M., Thissen, D. M., Heimberg, R. G., Chambless, D. L., & Rapee, R. M. (2004). More information from fewer questions: the factor structure and item properties of the original and brief fear of negative evaluation scale. Psychological Assessment, 16, 169–181. doi:10.1037/1040-3590.16.2.169.
Rodebaugh, T. L., Heimberg, R. G., Brown, P. J., Fernandez, K. C., Blanco, C., Schneier, F. R., & Liebowitz, M. R. (2011). More reasons to be straightforward: Findings and norms for two scales relevant to social anxiety. Journal of Anxiety Disorders, 25, 623–630. doi:10.1016/j.janxdis.2011.02.002.
Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York: Wiley. doi:10.1002/9780470316696.indsub.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monograph No. 17). Richmond, VA: Psychometric Society. http://www.psychometrika.org.
Schaefer, J. L., & Olsen, M. K. (1998). Multiple imputation for multivariate missing-data problems: a data analysts perspective. Multivariate Behavioral Research, 33, 545–571. doi:10.1207/s15327906mbr3304_5.
Smith, S. M., Stinson, F. S., Dawson, D. A., Goldstein, R., Huang, B., & Grant, B. F. (2006). Race/ethnic differences in the prevalence and co-occurrence of substance use disorders and independent mood and anxiety disorders: results from the national epidemiologic survey on alcohol and related conditions. Psychological Medicine, 36, 987–998. doi:10.1017/S0033291706007690.
Stark, S. (2008). MODFIT: Plot theoretical item response functions and examine the fit of dichotomous and polytomous IRT models to response data [Computer program]. Department of Psychology, University of South Florida at Tampa.
Steiger, J. H., & Lind, J. (1980). Statistically-based tests for the number of common factors. Paper presented at the Annual Spring Meeting of the Psychometric Society, Iowa City, IA. http://www.statpower.net.
Steinberg, L., & Thissen, D. (2006). Using effect sizes for research reporting: examples using item response theory to analyze differential item functioning. Psychological Methods, 11, 402–415. doi:10.1037/1082-989X.11.4.402.
Thissen, D., Steinberg, L., & Gerrard, M. (1986). Beyond group-mean differences: the concept of item bias. Psychological Bulletin, 99, 118–128. doi:10.1177/014662169501900105.
Thissen, D., Steinberg, L., & Kuang, D. (2002). Quick and easy implementation of the Benjamini-Hochberg procedure for controlling the false positive rate in multiple comparisons. Journal of Educational and Behavioral Statistics, 27, 77–83. doi:10.3102/10769986027001077.
Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 67–111). Hillsdale: Lawrence Erlbaum.
Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1–10. doi:10.1007/BF02291170.
Wang, W.-C., Shih, C.-L., & Yang, C.-C. (2009). The MIMIC method with scale purification for detecting differential item functioning. Educational and Psychological Measurement, 69, 713–731. doi:10.1177/0013164409332228.
Watson, D., & Friend, R. (1969). Measurement of social-evaluative anxiety. Journal of Consulting and Clinical Psychology, 33, 448–457. doi:10.1037/h0027806.
Weeks, J. W., Heimberg, R. G., Fresco, D. M., Hart, T. A., Turk, C. L., Schneier, F. R., et al. (2005). Empirical validation and psychometric evaluation of the brief fear of negative evaluation scale in patients with social anxiety disorder. Psychological Assessment, 17, 179–190. doi:10.1037/1040-3590.17.2.179.
Weeks, J. W., Heimberg, R. G., & Rodebaugh, T. L. (2008). The fear of positive evaluation scale: assessing a proposed cognitive component of social anxiety. Journal of Anxiety Disorders, 22, 44–55. doi:10.1016/j.janxdis.2007.08.002.
Weeks, J. W., Heimberg, R. G., Rodebaugh, T. L., Goldin, P. R., & Gross, J. J. (2012). Psychometric evaluation of the fear of positive evaluation scale in patients with social anxiety disorder. Psychological Assessment, 24, 301–312. doi:10.1037/a0025723.
Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: current approaches and future directions. Psychological Methods, 12, 58–79. doi:10.1037/1082-989X.12.1.58.
Woods, C. M. (2009a). Empirical selection of anchors for tests of differential item functioning. Applied Psychological Measurement, 33, 42–57. doi:10.1177/0146621607314044.
Woods, C. M. (2009b). Evaluation of MIMIC-model methods for DIF testing with comparison to two-group analysis. Multivariate Behavioral Research, 44, 1–27. doi:10.1080/00273170802620121.
Woods, C. M., Cai, L., & Wang, M. (2013). The Langer-improved Wald test for DIF testing with multiple groups: evaluation and comparison to two-group IRT. Educational and Psychological Measurement, 73, 532–547. doi:10.1177/0013164412464875.
Zimmerman, M., & Mattia, J. I. (1998). Body dysmorphic disorder in psychiatric outpatients: recognition, prevalence, comorbidity, demographic, and clinical correlates. Comprehensive Psychiatry, 39, 265–270. doi:10.1016/S0010-440X(98)90034-7.
Acknowledgments
This research was supported in part by the National Institute of Mental Health (NIMH) grant F31-MH096433-01 to Cheri A. Levinson; National Institute of Health (NIH)/NIMH grant MH090308 to Thomas L. Rodebaugh; NIH grant UL1 RR024992 to Washington University St. Louis; National Institute of Aging (NIA) grant T32 AG0030 and NIMH grant T32 MH20004 to Patrick J. Brown; NIMH grant R01 MH064481-01A1 and GlaxoSmithKline Pharmaceuticals grant 101618 to Richard G. Heimberg; NIH grant K02 DA023200 to Carlos Blanco; NIH grant R01 MH064726 to Michael Liebowitz. Carlos Blanco, Franklin Schneier, and Michael Liebowitz were also supported by the New York Psychiatric Institute. The authors would like to thank the Center for Research Methods and Data Analysis at the University of Kansas for assistance in preparing this manuscript.
Conflict of Interest
Jared K. Harpole, Cheri A. Levinson, Carol M. Woods, Thomas L. Rodebaugh, Justin W. Weeks, Patrick J. Brown, Richard G. Heimberg, Andrew R. Menatti, Carlos Blanco, Franklin Schneier, and Michael Liebowitz all declare that there were no conflicts of interest.
Experiment Participants
All data sets that were collected in this study had Institutional Review Board approval and all participants gave informed consent before entering the studies.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Harpole, J.K., Levinson, C.A., Woods, C.M. et al. Assessing the Straightforwardly-Worded Brief Fear of Negative Evaluation Scale for Differential Item Functioning Across Gender and Ethnicity. J Psychopathol Behav Assess 37, 306–317 (2015). https://doi.org/10.1007/s10862-014-9455-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10862-014-9455-9