Next Article in Journal
Mitigation of CyanoHABs Using Phoslock® to Reduce Water Column Phosphorus and Nutrient Release from Sediment
Next Article in Special Issue
HIV Knowledge and Stigmatizing Attitude towards People Living with HIV/AIDS among Medical Students in Jordan
Previous Article in Journal
Adolescents’ Attachment to Parents and Reactive–Proactive Aggression: The Mediating Role of Alexithymia
Previous Article in Special Issue
Self-Disclosure and Non-Communication: Stigma Management in Third-Sector Transitional Employment
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 18
Issue 24
10.3390/ijerph182413352
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Measure of Internalized Sexual Stigma for Lesbians and Gay Men (MISS-LG) in Taiwan: Psychometric Evidence from Rasch and Confirmatory Factor Analysis

by
Cheng-Fang Yen
1,2,3,
Yu-Te Huang
4,
Marc N. Potenza
5,6,7,8,
Tzu-Tung Tsai
9,*,
Chung-Ying Lin
10,11,12,13,* and
Hector W. H. Tsang
14
1
Department of Psychiatry, School of Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung 802, Taiwan
2
Department of Psychiatry, Kaohsiung Medical University Hospital, Kaohsiung 802, Taiwan
3
College of Professional Studies, National Pingtung University of Science and Technology, Pingtung 91201, Taiwan
4
Department of Social Work and Social Administration, The University of Hong Kong, Hong Kong RM543, China
5
Departments of Psychiatry and Neuroscience and the Child Study Center, School of Medicine, Yale University, New Haven, CT 06511, USA
6
Connecticut Council on Problem Gambling, Wethersfield, CT 06109, USA
7
Connecticut Mental Health Center, New Haven, CT 06519, USA
8
Wu Tsai Institute, Yale University, New Haven, CT 06510, USA
9
Department of Neurology, E-Da Hospital, I-Shou University, Kaohsiung City 82445, Taiwan
10
Institute of Allied Health Sciences, College of Medicine, National Cheng Kung University, Tainan 701, Taiwan
11
Department of Occupational Therapy, College of Medicine, National Cheng Kung University, Tainan 701, Taiwan
12
Department of Public Health, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan 701, Taiwan
13
Biostatistics Consulting Center, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan 701, Taiwan
14
Department of Rehabilitation Sciences, Faculty of Health and Social Sciences, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong 999077, China
 
add Show full affiliation list
 
remove Hide full affiliation list
*
Authors to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2021, 18(24), 13352; https://doi.org/10.3390/ijerph182413352
Submission received: 30 September 2021 / Revised: 11 December 2021 / Accepted: 13 December 2021 / Published: 18 December 2021
(This article belongs to the Special Issue Second Edition of Stigma, Health and Wellbeing)
Versions Notes

Abstract

:
Internalized sexual stigma may hamper the development of self-identity, social interaction, and intimate relationship in lesbian, gay, and bisexual (LGB) individuals. The Measure of Internalized Sexual Stigma for Lesbians and Gay Men (MISS-LG) is an important instrument assessing internalized sexual stigma for LGB individuals. However, its psychometric properties have not been examined in LGB populations outside the Italian community. The present study used advanced psychometric testing to evaluate the traditional Chinese version (the MISS-LG) among LGB individuals in Taiwan. LGB individuals (500 male, 500 female) participated in this study and completed the MISS-LG, HIV and Homosexuality Related Stigma (HHRS), and Acceptance and Action Questionnaire-II (AAQ). Confirmatory factor analysis results confirmed the three-factor structure of the MISS-LG; however, two items for males and one item for females had somewhat low factor loadings on the sexuality factor. Each MISS-LG factor was found to be unidimensional in Rasch results, except for the slight misfit in Item 3 and concurrent validity of the MISS-LG was supported by the positive correlations with HHRS and the AAQ. The Chinese MISS-LG has relatively satisfactory psychometric properties. However, further research is needed to investigate the reasons for problematic fitting of several items.

1. Introduction

A wide range of social stigma derived from heterosexism toward lesbian, gay, and bisexual (LGB) individuals prevails worldwide [1]. Structural stigma, including bans/restrictions on same-sex relationships and health care disparities among LGB individuals [2,3,4,5,6], may promote other stigma such as bullying and hate crimes against LGB individuals [7], and disapprobation of LGB cultures [8] may impact the identities, behaviors, health, and relationships of LGB individuals profoundly. Internalized sex stigma, or internalized homonegativity, refers to the process whereby LGB individuals perceive and internalize societal messages toward gender and sex—often unconsciously, as part of their self-image [9]. Negative feelings toward themselves may threaten their psychological well-being [10] and increase risks of mental health problems [11,12] among LGB individuals. LGB individuals with high internalized sexual stigma may avoid disclosing their sexual orientation to others [12] to protect themselves from harm [13]; however, it may damage LGB individuals’ relationship well-being [14] and reduce social support. LGB individuals may use alcohol and addictive substance use to cope with the stress-related to internalized sexual stigma [15,16]. Research has found that internalized sexual stigma was associated with practicing HIV at-risk sexual behaviors, such as condomless sex, multiple sex partners, and unprotected anal sex among gay men [12,15,17]. Internalized sexual stigma may also deter the LGB individuals from seeking medical services. For example, a previous study on Chinese men who have sex with men indicated that internalized homophobia was a major barrier to accessing human immunodeficiency virus (HIV) prevention and care services [18].
Young adulthood is a phase of the life span from adolescence to full-fledged adulthood where the individuals become more independent and explore various life possibilities [19]. LGB individuals may experience multiple dimensions of social stigma during their adolescence and internalize sexual stigma; internalized sexual stigma may be intensified in early adulthood when they become more independent and expand aspects of life. Internalized sexual stigma may hamper the development of self-identity, social interaction, and intimate relationships in LGB individuals during early adulthood. There is an urgent need to assess the internalized sexual stigma for LGB individuals during early adulthood. Therefore, a standardized instrument assessing internalized sexual stigma with good psychometric properties should be developed.
A review study identified six scales that were developed for measuring internalized sexual stigma and had the reports of psychometric properties [20]. Among them, the Reactions to Homosexuality Scale [21], Internalized Homonegativity Inventory [22], and Internalized Homophobia Scale [23] measured internalized sexual stigma in gay men; the Lesbian Internalized Homophobia Scale [24] measured internalized sexual stigma in lesbians; and the subscale of the Lesbian, Gay, and Bisexual Identity Scale [25] and Measure of Internalized Sexual Stigma for Lesbians and Gay men (MISS-LG) [26] measured internalized sexual stigma in lesbians and gay men.
The MISS-LG was developed using robust procedures, including (i) construct (i.e., internalized sexual stigma) definition; (ii) scale design; (iii) pilot testing; (iv) measure purification and scale administration; and (v) construct validation verification. Several rounds of psychometric testing with different methods (including content validity checking, cognitive interviewing, exploratory factor analysis, and confirmatory factor analysis) were applied to the MISS-LG; finally, the MISS-LG was verified to be a three-factor structure (i.e., sexuality, identity, and social discomfort) with 17 items [26]. The feature of the MISS-LG is that this scale has two versions, one for males and another for females with the same three first-order factors, the same number of items, and Likert-type scale with some different item contents [26]. For example, the sexuality factor of the MISS-LG contains the item “If you are gay, it is better to have an active sexual role” in the version for gay and bisexual men only and the item “All lesbian women end up isolated and alone” in the version for lesbian and bisexual women only. As Szymanski et al. [27] noted, there are significant differences in the ways that internalized sexual stigma is experienced for sexual minority men and women, and it is important to identify manifestations that have some common features and specificities for lesbian women and gay men. Therefore, the MISS-LG will help capture gender-specific characteristics of internalized sex stigma.
Although the MISS-LG was developed using a robust procedure and has been cited more than 120 times since its release, the current literature provides little information on its psychometric properties. To the best of our knowledge, the psychometric properties of the MISS-LG have only been reported in one study [26], in the manuscript reporting the development of the MISS-LG. In the manuscript [26], study participants were Westerns, and therefore, psychometric properties of the MISS-LG should be evaluated in other populations (e.g., Eastern ones like in Taiwan) to evaluate its generalizability beyond WEIRD (White, educated, industrialized, rich, and democratic) groups [28]. Indeed, prior research has suggested the psychometric properties of an instrument should be accumulated for scientific reasons [29].
Tolerance to sexual minorities in Taiwan has outpaced that in China, Japan, and South Korea over the past two decades [30]. In May 2017, Taiwan’s Council of Grand Justices announced that the current Civil Code that barred same-sex relationships was a violation of human rights to equality and was unconstitutional. This announcement encouraged many LGB individuals in Taiwan. However, the results of the vote for the Same-Sex Marriage Referendums in Taiwan released on November 24, 2018, indicated that over 70% of voters opposed same-sex marriage defined in the Civil Code [31]. Although the Taiwanese government legalized same-sex relationships outside the Civil Code in May 2019 [32], unfavorable attitudes and unfriendly behaviors appear to remain rooted in many Taiwanese people’s mindset and difficult to change, especially for those with strong traditional Chinese ethics [33] or Christian faith [30]. Many LGB individuals are likely to experience and be impacted by these unfavorable attitudes and unfriendly behaviors and gradually develop internalized sexual stigma.
In order to supplement the psychometric evidence of the MISS-LG in non-Western societies, the present study aimed to use two types of psychometric testing (i.e., classical test theory and item response theory) [34,35] to evaluate the MISS-LG among LGB individuals in Taiwan. Specifically, the classical test theory in the present study adopts the confirmatory factor analysis (CFA) to verify whether the three-factor structure of the MISS-LG can be replicated among LGB individuals in Taiwan. Classical test theory was also used to examine the concurrent validity of the MISS-LG after its three-factor structure was confirmed in the present sample. For item response theory, Rasch modeling [36,37] was applied to examine whether the each of three factors had a unidimensional feature.
We also examined the concurrent validity of the MISS-LG by testing the correlations between internalized sex stigma and perceived social stigma attitudes toward homosexuality and psychological flexibility. We hypothesized that the MISS-LG factors would have significant correlations with perceived social stigma attitudes toward homosexuality and psychological flexibility because of the following reasons: (1) as internalized sex stigma is the result whereby LGB individuals perceive and internalize social stigma toward sexual minority, internalized sex stigma and perceived social stigma attitudes toward homosexuality should be positively associated; (2) individuals with a higher level of psychological flexibility may cope better with unfriendly environments that may prevent generation of internalized stigma [38].

2. Materials and Methods

2.1. Participants and Procedure

We recruited participants by posting online advertisements on social media including Facebook, Twitter, and LINE messaging app (Line Corporation, Tokyo, Japan), the Bulletin Board System, and the home pages of three health promotion and counseling centers for LGB individuals from August 2018 to July 2019. The recruitment criteria were individuals who identified their sexual orientation as homosexual or bisexual, aged between 20 and 30 years, and lived in Taiwan. Anyone interested in participating in this study was to phone the research assistants. Research assistants assessed the eligibility of potential participants based on study criteria, explained the study aims and procedures to them, and scheduled times for completing the study questionnaires individually in the study room. The research assistants evaluated participants face-to-face in the study room to determine whether they had impaired intellect or evidence of alcohol and substance use that might interfere with understanding the study’s purpose or method of completing the questionnaire; if they had, they were excluded from the study. In total, 1000 participants (500 males and 500 females) participated. No participants were excluded. Informed consent was obtained from all participants. This study was approved by the Institutional Review Board of Kaohsiung Medical University Hospital—KMUHIRB-F(II)-20180018.

2.2. Measures

2.2.1. The MISS-LG

The MISS-LG contains 17 items regarding internalized sexual stigma for lesbian women and gay men [26]. The MISS-LG has two versions, one for lesbian women and another for gay men, with the same factor structure (i.e., three factors of sexuality, identity, and social discomfort). The identity factor (5 items) indicates an enduring propensity to have a negative self-attitude as homosexual and to consider sexual stigma as a part of self-identity (e.g., “If it were possible, I would do anything to change my sexual orientation”). The social discomfort factor (7 items) reflects fear of public identification as a lesbian woman or gay man in the social context and disclosure in private and professional life (e.g., “When I feel attracted to another lesbian/gay man, I hope no one realizes it”). The sexuality factor (5 items) indicates the pessimistic attitudes toward the quality and duration of intimate relationships in lesbians and gay men (e.g., “Lesbians/gay men can only have flings/one-night stands”) and the negative conception of gay or lesbian sexual behaviors (e.g., “When I have sex with a woman/man, I feel awkward”). All MISS-LG items are rated on a five-point Likert type scale with the descriptors of strongly disagree (score 1), disagree (score 2), neither disagree nor agree (score 3), agree (score 4), and strongly agree (score 5); therefore, a higher MISS-LG score indicates higher levels of internalized sexual stigma. The MISS-LG has demonstrated satisfactory psychometric properties [26]. The internal consistency of MISS-LG was acceptable to excellent in the present sample (McDonald’s omega = 0.67 for sexuality in male participants; = 0.87 for identity in male participants; = 0.90 for social discomfort in male participants; = 0.64 for sexuality in female participants; = 0.87 for identity in female participants; and = 0.91 for social discomfort in female participants).
The MISS-LG has been translated into a Chinese version for Taiwanese LGB individuals using standard forward-, backward-, and pretest-step methods [39]. First, the original version was translated into a traditional Chinese version by one bilingual translator. Next, the traditional Chinese version was back-translated into English by another bilingual translator. Finally, the original version was compared with the back-translation. If discrepancies arose in the back-translation, translators worked cooperatively to make corrections in the final traditional Chinese version. We further invited three experts in the field of sexuality study to examine the adequacy of the questionnaire.

2.2.2. Measures Used for Concurrent Validity of the MISS-LG

We used two measures for examining the concurrent validity of the MISS-LG. The first measure was the HIV and Homosexuality Related Stigma (HHRS) [40]. We adopted the 12 items on the HHRS-Homosexuality subscale measuring the stigma attitudes toward homosexuality that LGB individuals perceive from their families. The items are rated on a four-point Likert type scale with the descriptors of strongly disagree (score 1), disagree (score 2), agree (score 3), and strongly agree (score 4); therefore, a higher HHRS-Homosexuality score indicates a higher level of perceived stigma related to homosexuality from families. The HHRS-Homosexuality had satisfactory psychometric properties in prior research [39] and in the present sample (e.g., McDonald’s omega = 0.95 in male participants; and = 0.94 in female participants).
The second measure is the Acceptance and Action Questionnaire-II (AAQ) [41]. The AAQ contains 7 items asking an individual’s psychological flexibility. All the AAQ items are rated on a seven-point Likert type scale with the descriptors of completely disagree (score 1), almost always disagree (score 2), rarely agree (score 3), sometimes agree (score 4), usually agree (score 5), almost always agree (score 6), and completely agree (score 7); therefore, a higher AAQ score indicates a lower level of psychological flexibility. The AAQ had satisfactory psychometric properties in prior research [41,42] and in the present sample (e.g., McDonald’s omega = 0.94 in male participants; =0.95 in female participants).

2.3. Sociodemographic and Sexual Orientation Factors

Data were collected regarding the participants’ gender (male vs. female), age, education level (high school or below vs. college or above), sexual orientation (“Do you identify yourself as a gay/lesbian or bisexual?”), paternal and maternal education, and sexual orientation known by family, friends, and online friends (no or few vs. many or a great quantity).

2.4. Data Analysis

Participants’ characteristics and MISS-LG item scores were first analyzed using descriptive statistics. Then, the distributions of the MISS-LG items were checked (where absolute skewness < 3 and absolute kurtosis < 10 indicate normal distribution [43]) before psychometric analyses were conducted. The psychometric testing of the MISS-LG involved three parts. First, CFA with a diagonally weighted least-squares estimator was used to examine the three-factor structure of the MISS-LG. Accordingly, factor loadings derived from the CFA were calculated with a value >0.3 indicating acceptability [44]. In addition, the CFA was examined using several fit indices to determine whether the three-factor structure was supported. The fit indices were the comparative fit index (CFI) > 0.95, Tucker–Lewis index (TLI) > 0.95, root mean square error of approximation (RMSEA) < 0.06, and standardized root mean square residual (SRMR) < 0.08 [45]. Second, Rasch analysis with a partial credit model was used to examine whether the items embedded in the MISS-LGB factors belonged to their constructs. Specifically, infit mean square (MnSq) and outfit MnSq were used to decide whether an item fit with its construct. When both infit and outfit MnSq of an item was between 0.5 and 1.5, this item was considered to be in its embedded construct [46]. Item difficulty was also calculated from the Rasch analysis. Third, concurrent validity of the traditional Chinese version of the MISS-LG was examined using the correlations between MISS-LG factors and other measures (i.e., HHRS-Homosexuality and AAQ).
All psychometric testing for the MISS-LG was conducted separately for male and female participants because of the different gender versions. All statistical analyses were conducted using R software (Boston, MA, USA) (the CFA was done with the use of lavaan package version 0.6–9; https://lavaan.ugent.be/index.html, accessed on 12 December 2021), except for the Rasch analyses. The Rasch analyses were done using the WINSTEPS (WINSTEPS, Chicago, IL, USA) (https://www.winsteps.com, accessed on 12 December 2021).

3. Results

The participants were 500 males (mean age = 24.80 years; SD age = 2.91; age range = 20–30) and 500 females (mean age = 24.45 years; SD age = 3.06; age range = 20–30). Most participants had obtained a college diploma or college degree (87.2% in male participants and 91.0% in female participants). Nearly three-fourths of male participants (73.0%) and slightly less than half of the female participants (41.0%) were homosexual. Other characteristics, including parental educational levels and whether participants’ sexual orientation was known by others, are presented in Table 1.
Table 2 reports the item properties and CFA results for the MISS-LG. All items were normally distributed or nearly normally distributed (skewness = −02 to 1.97, kurtosis = −1.25 to 3.87 for males; skewness = −0.11 to 3.13, kurtosis = −0.96 to 9.77 for females). Most items had an acceptable factor loading (i.e., >0.3) with the exceptions of items 8 (“I do not believe in love between homosexual individuals”; factor loading = 0.270 for males) and 11 (“If you are gay, it is better to have an “active” sexual role”; factor loading = 0.167 for males and “Lesbians can only have flings/one-night stands”; factor loading = 0.299 for females). Regarding the entire factor structure, the MISS-LG demonstrated a good fit with the three-factor structure by the support of excellent fit indices. For males, the fit indices were chi-square (df) = 232.47 (116), p-value < 0.001, CFI = 0.985, TLI = 0.983, RMSEA (90% CI) = 0.045 (0.036, 0.053), and SRMR = 0.062 or females, the fit indices were chi-square (df) = 186.62 (116), p-value < 0.001, CFI = 0.987, TLI = 0.985, RMSEA (90% CI) = 0.035 (0.025, 0.044), and SRMR = 0.056.
Results from Rasch analyses (Table 3) indicate that the difficulty range of the MISS-LG sexuality factor was between 0.97 and −1.03 for males and between 0.60 and −0.66 for females. The difficulty range of the MISS-LG identity factor was between 1.12 and −1.01 for males and between −1.12 and 0.85 for females. The difficulty range of the MISS-LG social discomfort was between −0.45 and 0.36 for males and between −1.29 and 0.93 for females. Moreover, almost all the items had good fit statistics in both infit and outfit MnSq with the exceptions of items 3 (“I’m worried to understand whether I like women”; outfit MnSq = 1.53) and 17 (“Effeminate gay men annoy me”; outfit MnSq = 1.51) for males.
Finally, the concurrent validity of the MISS-LG is shown in Table 4 with both HHRS-Homosexuality and AAQ measures. The correlations between the three factors of MISS-LG were in strong magnitude (r = 0.752 to 0.937 for males; = 0.640 to 0.784 for females; all p-values < 0.001). Moreover, the three MISS-LG factors were positively associated with HHRS-Homosexuality scores (r = 0.220 to 0.256 for males; = 0.190 to 0.384 for females; all p-values < 0.001) and AAQ scores (r = 0.323 to 0.427 for males; = 0.135 to 0.424 for females; all p-values < 0.001). Finally, the concurrent validity of the MISS-LG is shown in Table 4 with both HHRS-Homosexuality and AAQ measures. The correlations between the three factors of MISS-LG were strong in magnitude (r = 0.752 to 0.937 for males; = 0.640 to 0.784 for females; all p-values < 0.001). Moreover, the three MISS-LG factors were positively associated with HHRS-Homosexuality scores (r = 0.220 to 0.256 for males; = 0.190 to 0.384 for females; all p-values < 0.001) and AAQ scores (r = 0.323 to 0.427 for males; = 0.135 to 0.424 for females; all p-values < 0.001).

4. Discussion

This study used various statistical methods to examine the psychometric properties of a Chinese version of the MISS-LG. The results derived from CFA and Rasch analysis all suggest that the traditional Chinese version of the MISS-LG had satisfactory construct validity across groups. Specifically, CFA results indicated that the factorial structure of the MISS-LG included three factors in both male and female sexual-minority individuals in Taiwan. Therefore, the sexuality, identity, and social discomfort factors are different constructs in the MISS-LG, which echoes findings from Lingiardi et al. [26]. Rasch analysis additionally suggested that each factor was unidimensional: the items under the same factor of the MISS-LG were grouped together to demonstrate the same constructs. The concurrent validity of the MISS-LG was also supported by the significant correlations of the three MISS-LG factor scores with HHRS-Homosexuality and AAQ scores.
The CFA results indicated that most items had an acceptable factor loading. However, we found that item 11 for males (“If you are gay, it is better to have an “active” sexual role”) had a low factor loading (factor loading = 0.167). Therefore, item 11 might not well adhere to its embedded sexuality factor. The sexuality factor indicates the negative attitudes toward sexual behaviors and intimate relationships in lesbian women, gay men, or bisexual individuals. However, item 11 describes a value judgment on a specific sexual behaviors or positions rather than sexual behaviors or intimate relationships in gay men. This may be a possible reason to explain the low factor loadings for item 11 among males. It is also possible that cultural differences relating to individualism versus collectivism in Western and Eastern cultures may in part underlie apparent differences. Another item for males was found to have slightly low factor loadings; specifically, item 8 for males (“I do not believe in love between homosexual individuals”; factor loading = 0.270). Item 8 for males does not explicitly describe attitudes toward sexual behaviors or intimate relationships. For example, whether a person “believes in love” could be influenced by past experiences of dating. Therefore, when a person reads this sentence, the focus may not be on the intimate relationship of the individual per se. One more item, item 11 for females, also had a factor loading lower than 0.3 (factor loading = 0.299). However, this item (i.e., “Lesbians can only have flings/one-night stands”) conceptually fits well in the sexuality factor and its factor loadings is extremely close to 0.3. Nevertheless, future studies are needed to further explore the factor structure of MISS-LG for both the male and female versions.
The three factors of the MISS-LG were positively associated with HHRS-Homosexuality and AAQ scores. A high HHRS-Homosexuality score indicates a high level of perceived stigma regarding homosexuality from families. Family is the microsystem in which individuals have typically lived since childhood [47]. LGB individuals who perceive stigmatizing attitudes toward homosexuality may experience long-lasting impacts on their self-identity and internalize perceived stigma. A high AAQ score indicates a low level of psychological flexibility. Individuals with low psychological flexibility lack adaptable perspectives to observe and think without attachment to particular experiences, and therefore they may become trapped by past experiences and find it difficult to move forward [48]. LGB individuals who have low psychological flexibility may lack the effort to challenge social sexual stigma and internalize it as self-stigma. The significant correlations between MISS-LG and HHRS-Homosexuality and AAQ scores support the concurrent validity of the MISS-LG.
There are several limitations in this study. First, participants were a group of LGB individuals in early adulthood; thus, our results may not generalize to other age groups. Second, biases inherent to questionnaires (e.g., relating to social desirability) should be considered. Although we assured participants that all questionnaires were anonymous and individual data would be kept confidential, we cannot guarantee that every participant had no other concerns when they completed the questionnaires. Third, we did not conduct test–retest reliability and responsiveness for the MISS-LG. Therefore, it is unclear whether the traditional Chinese version of the MISS-LG has good reproducibility over time and whether it is sensitive to detecting changes in self-stigma. Fourth, the study included only lesbian, gay, and bisexual individuals. How sexual stigma may impact other sexual minority groups (e.g., transsexual, asexual) should be considered in future studies. Fifth, this study inquired participants’ gender identities by the binary of male and female but did not include the options of transgender, gender nonbinary, or genderqueer. Research has found that sexual and gender minority identities have intersectional impacts on health [49] and behaviors [50]; both sexual and gender minority identities should be considered in public health practice [51].

5. Conclusions

In conclusion, the MISS-LG has relatively satisfactory psychometric properties in a sample of LGB individuals in Taiwan. The three factors of the MISS-LG were confirmed in the CFA results; however, two items (Item 8 in male participants; Items 8 and 11 in both males and females) were found to have relatively low factor loadings in the sexuality factor. Each factor of the three MISS-LG factors were found to be unidimensional and most items fit satisfactorily in their embedded constructs, except for slight misfitting of Item 3 and concurrent validity of the MISS-LG was supported by the positive correlations with HHRS-Homosexuality family subscale and AAQ scores. However, more investigation is needed to understand reasons for the slight misfitting of several MISS-LG items.

Author Contributions

Conceptualization, C.-F.Y., T.-T.T. and C.-Y.L.; methodology, C.-F.Y., C.-Y.L., M.N.P. and H.W.H.T.; software, C.-Y.L., H.W.H.T. and C.-F.Y.; validation, Y.-T.H., M.N.P. and C.-F.Y.; formal analysis, C.-Y.L.; investigation, C.-F.Y.; resources, C.-F.Y.; data curation, C.-F.Y.; writing, original draft preparation, T.-T.T., C.-Y.L. and C.-F.Y.; writing, review and editing, Y.-T.H., M.N.P., C.-F.Y., T.-T.T., C.-Y.L. and H.W.H.T.; visualization, C.-Y.L.; supervision, C.-F.Y. and C.-Y.L.; project administration, C.-F.Y.; funding acquisition, C.-F.Y. All authors have read and agreed to the published version of the manuscript.

Funding

This study was supported by a research grant from the Ministry of Science and Technology, Taiwan (107–2314-B-037–102-MY3) and Kaohsiung Medical University Hospital (grants KMUH107–7R69, KMUH108–8R65 and KMUH109–9R77).

Institutional Review Board Statement

The institutional review boards of Kaohsiung Medical University Hospital (approval number: KMUHIRB-F(II)-20180018; date of approval: 27 March 2018) approved this study.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data will be available upon reasonable request to the corresponding authors.

Conflicts of Interest

The authors declare no conflict of interest. M.N.P. has the following disclosures. M.N.P. has consulted for and advised Opiant Pharmaceuticals, Idorsia Pharmaceuticals, AXA, Game Day Data, and the Addiction Policy Forum; has been involved in a patent application with Yale University and Novartis; has received research support from the Mohegan Sun Casino, the Connecticut Council on Problem Gambling, and the National Center for Responsible Gaming; has participated in surveys, mailings or telephone consultations related to addictive behaviors and other health topics; and has consulted for law offices and gambling entities on issues related to impulse control or addictive disorders.

References

  1. Bryant, K. In defense of gay children? `Progay’ homophobia and the production of homonormativity. Sexualities 2008, 11, 455–475. [Google Scholar] [CrossRef]
  2. Hatzenbuehler, M.L. Structural stigma and the health of lesbian, gay, and bisexual populations. Curr. Dir. Psychol. Sci. 2014, 23, 127–132. [Google Scholar] [CrossRef]
  3. Link, B.G.; Phelan, J.C. Conceptualizing stigma. Annu. Rev. Sociol. 2001, 27, 363–385. [Google Scholar] [CrossRef]
  4. Rostosky, S.S.; Riggle, E.D.; Horne, S.G.; Denton, F.N.; Huellemeier, J.D. Lesbian, gay, and bisexual individuals’ psychological reactions to amendments denying access to civil marriage. Am. J. Orthopsychiat. 2010, 80, 302. [Google Scholar] [CrossRef] [PubMed]
  5. King, M.; Bartlett, A. What same sex civil partnerships may mean for health. J. Epidemiol. Community Health 2006, 60, 188–191. [Google Scholar] [CrossRef]
  6. Hafeez, H.; Zeshan, M.; A Tahir, M.; Jahan, N.; Naveed, S. Health care disparities among lesbian, gay, bisexual, and transgender youth: A literature review. Cureus 2017, 9, e1184. [Google Scholar] [CrossRef] [Green Version]
  7. O’Shaughnessy, M.; Russell, S.; Heck, K.; Calhoun, C.; Laub, C. Safe Place to Lean; A Report of the California Safe Schools Coalition and the 4-H Center for Youth Development University of California; California Safe Schools Coalition: Los Angeles, CA, USA, 2004. [Google Scholar]
  8. Callender, K.A. Understanding antigay bias from a cognitive-affective-behavioral perspective. J. Homosex. 2015, 62, 782–803. [Google Scholar] [CrossRef] [PubMed]
  9. Meyer, I. Minority Stress and Mental Health in Gay Men, 2nd ed.; Columbia University Press: New York, NY, USA, 2003. [Google Scholar] [CrossRef]
  10. Herek, G.M.; Gillis, J.R.; Cogan, J.C. Internalized stigma among sexual minority adults: Insights from a social psychological perspective. J. Couns. Psychol. 2009, 56, 32–43. [Google Scholar] [CrossRef] [Green Version]
  11. Su, X.; Zhou, A.N.; Li, J.; Shi, L.-E.; Huan, X.; Yan, H.; Wei, C. Depression, loneliness, and sexual risk-taking among HIV-negative/Unknown men who have sex with men in China. Arch. Sex. Behav. 2017, 47, 1959–1968. [Google Scholar] [CrossRef] [PubMed]
  12. Xu, W.; Zheng, L.; Xu, Y.; Zheng, Y. Internalized homophobia, mental health, sexual behaviors, and outness of gay/bisexual men from Southwest China. Int. J. Equity Health 2017, 16, 1–10. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Lehavot, K.; Simoni, J.M. The impact of minority stress on mental health and substance use among sexual minority women. J. Consult. Clin. Psychol. 2011, 79, 159–170. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Cao, H.; Zhou, N.; Fine, M.; Liang, Y.; Li, J.; Mills-Koonce, W.R. sexual minority stress and same-sex relationship well-being: A meta-analysis of research prior to the U.S. nationwide legalization of same-sex marriage. J. Marriage Fam. 2017, 79, 1258–1277. [Google Scholar] [CrossRef] [PubMed]
  15. Puckett, J.A.; Newcomb, M.E.; Garofalo, R.; Mustanski, B. Examining the conditions under which internalized homophobia is associated with substance use and condomless sex in young MSM: The moderating role of impulsivity. Ann. Behav. Med. 2017, 51, 567–577. [Google Scholar] [CrossRef]
  16. Jeffries, W.L.; Johnson, O.D. Internalized homonegativity and substance use among U.S. Men who have sex with men only (MSMO) and men who have sex with men and women (MSMW). Subst. Use Misuse 2017, 53, 559–564. [Google Scholar] [CrossRef]
  17. Crosby, R.A.; Salazar, L.F.; Mena, L.; Geter, A. Associations between internalized homophobia and sexual risk behaviors among young black men who have sex with men. Sex. Transm. Dis. 2016, 43, 656–660. [Google Scholar] [CrossRef] [PubMed]
  18. Pyun, T.; Santos, G.-M.; Arreola, S.; Do, T.; Hebert, P.; Beck, J.; Makofane, K.; Wilson, P.A.; Ayala, G. Internalized homophobia and reduced hiv testing among men who have sex with men in China. Asia Pac. J. Public Health 2014, 26, 118–125. [Google Scholar] [CrossRef] [PubMed]
  19. Arnett, J.J. Emerging adulthood: A theory of development from the late teens through the twenties. Am. Psychol. 2000, 55, 469–480. [Google Scholar] [CrossRef] [PubMed]
  20. Berg, R.C.; Munthe-Kaas, H.M.; Ross, M. Internalized homonegativity: A systematic mapping review of empirical research. J. Homosex. 2015, 63, 541–558. [Google Scholar] [CrossRef] [Green Version]
  21. Ross, M.W.; Rosser, B.R. Measurement and correlates of internalized homophobia: A factor analytic study. J. Clin. Psychol. 1996, 52, 15–21. [Google Scholar] [CrossRef]
  22. Mayfield, W. The Development of an internalized homonegativity inventory for gay men. J. Homosex. 2001, 41, 53–76. [Google Scholar] [CrossRef]
  23. Gençöz, T.; Yuksel, M. Psychometric properties of the Turkish version of the internalized homophobia scale. Arch. Sex. Behav. 2006, 35, 597–602. [Google Scholar] [CrossRef]
  24. Szymanski, D.M.; Chung, Y.B. The Lesbian internalized homophobia scale: A rational/theoretical approach. J. Homosex. 2001, 41, 37–52. [Google Scholar] [CrossRef] [PubMed]
  25. Mohr, J.J.; Kendra, M.S. Revision and extension of a multidimensional measure of sexual minority identity: The Lesbian, Gay, and Bisexual Identity Scale. J. Couns. Psychol. 2011, 58, 234–245. [Google Scholar] [CrossRef] [PubMed]
  26. Lingiardi, V.; Baiocco, R.; Nardelli, N. Measure of internalized sexual stigma for lesbians and gay men: A New scale. J. Homosex. 2012, 59, 1191–1210. [Google Scholar] [CrossRef] [PubMed]
  27. Szymanski, D.M.; Kashubeck-West, S.; Meyer, J. Internalized heterosexism: Measurement, psychosocial correlates, and research directions. Couns. Psychol. 2008, 36, 525–574. [Google Scholar] [CrossRef]
  28. Azar, B. Are your findings ‘WEIRD’? Monitor Psychol. 2010, 41, 11. [Google Scholar]
  29. Lin, C.-Y.; Hwang, J.-S.; Wang, W.-C.; Lai, W.-W.; Su, W.-C.; Wu, T.-Y.; Yao, G.; Wang, J.-D. Psychometric evaluation of the WHOQOL-BREF, Taiwan version, across five kinds of Taiwanese cancer survivors: Rasch analysis and confirmatory factor analysis. J. Formos. Med Assoc. 2018, 118, 215–222. [Google Scholar] [CrossRef]
  30. Cheng, Y.-H.A.; Wu, F.-C.F.; Adamczyk, A. Changing attitudes toward homosexuality in Taiwan. Chin. Sociol. Rev. 2016, 48, 317–345. [Google Scholar] [CrossRef]
  31. Ko, N.-Y.; Lin, I.-H.; Huang, Y.-T.; Chen, M.-H.; Lu, W.-H.; Yen, C.-F. Associations of perceived socially unfavorable attitudes toward homosexuality and same-sex marriage with suicidal ideation in Taiwanese people before and after same-sex marriage referendums. Int. J. Environ. Res. Public Health 2020, 17, 1047. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Ministry of Justice, Taiwan. Act for implementation of J.Y. Interpretation. Available online: https://law.moj.gov.tw/ENG/LawClass/LawAll.aspx?pcode=B0000008/ (accessed on 11 September 2021).
  33. Jeffreys, E.; Wang, P. Pathways to legalizing same-sex marriage in China and Taiwan: Globalization and “Chinese values”. In Global Perspectives on Same-Sex Marriage: A Neo-Institutional Approach; Winter, B., Forest, M., Sénac, R., Eds.; Springer Nature: Cham, Switzerland, 2018; pp. 197–219. [Google Scholar]
  34. Cappelleri, J.C.; Lundy, J.J.; Hays, R.D. Overview of classical test theory and item response theory for the quantitative assessment of items in developing patient-reported outcomes measures. Clin. Ther. 2014, 36, 648–662. [Google Scholar] [CrossRef] [Green Version]
  35. Raykov, T.; Marcoulides, G.A. On the relationship between classical test theory and item response theory: From one to the other and back. Educ. Psychol. Meas. 2016, 76, 325–338. [Google Scholar] [CrossRef] [Green Version]
  36. Lin, C.-Y.; Broström, A.; Griffiths, M.D.; Pakpour, A.H. Psychometric evaluation of the persian ehealth literacy scale (eHEALS) among elder iranians with heart failure. Eval. Health Prof. 2019, 43, 222–229. [Google Scholar] [CrossRef] [Green Version]
  37. Nejati, B.; Fan, C.-W.; Boone, W.J.; Griffiths, M.D.; Lin, C.-Y.; Pakpour, A.H. Validating the Persian intuitive eating scale-2 among breast cancer survivors who are overweight/obese. Eval. Health Prof. 2020, 44, 385–394. [Google Scholar] [CrossRef] [PubMed]
  38. Hayes, S.C.; Strosahl, K.D.; Wilson, K.G.; Bissett, R.T.; Pistorello, J.; Toarmino, D.; Polusny, M.A.; Dykstra, T.A.; Batten, S.V.; Bergan, J.; et al. Measuring experiential avoidance: A preliminary test of a working model. Psychol. Record 2004, 54, 553–578. [Google Scholar] [CrossRef] [Green Version]
  39. Brislin, R.W. Back-translation for cross-cultural research. J. Cross Cult. Psychol. 1970, 1, 185–216. [Google Scholar] [CrossRef]
  40. Liu, H.; Feng, T.; Rhodes, A.G. Assessment of the Chinese version of HIV and homosexuality related stigma scales. Sex. Transm. Infect. 2009, 85, 65–69. [Google Scholar] [CrossRef] [PubMed]
  41. Bond, F.W.; Hayes, S.; Baer, R.A.; Carpenter, K.M.; Guenole, N.; Orcutt, H.; Waltz, T.; Zettle, R.D. Preliminary psychometric properties of the acceptance and action questionnaire–II: A revised measure of psychological inflexibility and experiential avoidance. Behav. Ther. 2011, 42, 676–688. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  42. Chen, Y.; Xu, X.; Liu, X.; Ou, M.; Xie, C. Psychological flexibility of nurses in a cancer hospital: Preliminary validation of a chinese version of the work-related acceptance and action questionnaire. Asia-Pacific J. Oncol. Nurs. 2017, 5, 83–90. [Google Scholar] [CrossRef]
  43. Kline, R.B. Principles and Practice of Structural Equation Modeling, 2nd ed.; Guilford Press: New York, NY, USA, 2004. [Google Scholar]
  44. Tabachnick, B.G.; Fidell, L.S. SAS for Windows Workbook for Tabachnick and Fidell Using Multivariate Statistics; Allyn and Bacon: Boston, MA, USA, 2001. [Google Scholar]
  45. Hu, L.T.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equ. Model. Multidiscip. J. 1999, 6, 1–55. [Google Scholar] [CrossRef]
  46. Souza, M.A.P.; Coster, W.J.; Mancini, M.C.; Dutra, F.C.M.S.; Kramer, J.; Sampaio, R.F. Rasch analysis of the participation scale (P-scale): Usefulness of the P-scale to a rehabilitation services network. BMC Public Health 2017, 17, 934. [Google Scholar] [CrossRef] [Green Version]
  47. Bronfenbrenner, U. The Ecology of Human Development: Experiments by Nature and Design; Harvard University Press: Boston, MA, USA, 1979. [Google Scholar]
  48. Hayes, S.C.; Strosahl, K.D.; Wilson, K.G. Acceptance and Commitment Therapy: The Process and Practice of Mindful Change; Guilford Press: New York, NY, USA, 2011. [Google Scholar]
  49. Hsieh, N.; Ruther, M. Sexual minority health and health risk factors: Intersection effects of gender, race, and sexual identity. Am. J. Prev. Med. 2016, 50, 746–755. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  50. Lewis, B.J.; Hesse, C.L.; Cook, B.C.; Pedersen, C.L. Sexistential crisis: An intersectional analysis of gender expression and sexual orientation in masculine overcompensation. J. Homosex. 2018, 67, 58–78. [Google Scholar] [CrossRef] [PubMed]
  51. Sell, R.L.; Krims, E.I. Structural transphobia, homophobia, and biphobia in public health practice: The Example of COVID-19 surveillance. Am. J. Public Health 2021, 111, 1620–1626. [Google Scholar] [CrossRef] [PubMed]
Table
Table 1. Participants’ characteristics (N = 1000).
Table 1. Participants’ characteristics (N = 1000).
VariablesMale (N = 500)Female (N = 500)
Age in years; M (SD)24.80 (2.91)24.45 (3.06)
Educational level; N (%)
High school or below64 (12.8)45 (9.0)
College or above436 (87.2)455 (91.0)
Sexual orientation; N (%)
Homosexual365 (73.0)205 (41.0)
Bisexual135 (27.0)295 (59.0)
Paternal education; N (%)
High school or below319 (63.8)272 (54.4)
College or above181 (36.2)228 (45.6)
Maternal education; N (%)
High school or below355 (71.0)305 (61.0)
College or above145 (29.0)195 (39.0)
Sexual orientation known by family
No or few383 (76.6)398 (79.6)
Many or a great quantity117 (23.4)102 (20.4)
Sexual orientation known by friends
No or few195 (39.0)159 (31.8)
Many or a great quantity305 (61.0)341 (68.2)
Sexual orientation known by online friends
No or few187 (37.4)289 (57.8)
Many or a great quantity313 (62.6)211 (42.2)
Table
Table 2. Confirmatory factor analysis results and item properties for the Measure of Internalized Sexual Stigma for Lesbians and Gay Men (MISS-LG).
Table 2. Confirmatory factor analysis results and item properties for the Measure of Internalized Sexual Stigma for Lesbians and Gay Men (MISS-LG).
Items Male Female
Factor LoadingMean (SD)SkewnessKurtosisFactor LoadingMean (SD)SkewnessKurtosis
Sexuality
I20.7512.89 (1.16)0.07−0.940.5421.74 (0.90)1.241.16
I50.4661.85 (1.03)1.110.510.4791.42 (0.83)2.123.79
I80.2701.45 (0.79)1.973.870.4991.12 (0.36)3.139.77
I110.1672.99 (1.01)−0.01−0.230.2991.24 (0.57)2.879.46
I140.3901.76 (1.00)1.250.850.3801.33 (0.64)2.154.61
Identity
I30.6042.06 (1.16)0.86−0.290.7251.65 (0.84)1.271.25
I60.7411.81 (1.08)1.170.480.5681.47 (0.76)1.622.34
I90.7582.41 (1.21)0.34−0.870.6352.24 (1.21)0.52-0.96
I120.7271.80 (0.94)0.89−0.220.7132.02 (1.02)0.63-0.43
I150.7482.65 (1.37)0.20−1.250.8281.65 (0.86)1.311.17
Social discomfort
I10.7102.50 (1.16)0.37−0.770.6092.82 (1.02)-0.11−0.64
I40.7232.52 (1.17)0.31−0.850.7131.80 (0.95)1.080.40
I70.6112.63 (1.18)0.22−1.000.7532.50 (1.19)0.40−0.91
I100.7782.54 (1.16)0.31−0.870.7982.10 (1.04)0.72−0.26
I130.7572.90 (1.29)-0.02−1.160.5662.03 (1.01)0.71−0.27
I160.6702.76 (1.25)0.06−1.110.6971.96 (1.09)0.86−0.35
I170.4282.39 (1.13)0.35−0.780.7211.67 (0.87)1.341.39
Table
Table 3. Rasch analysis results for the Measure of Internalized Sexual Stigma for Lesbians and Gay Men (MISS-LG).
Table 3. Rasch analysis results for the Measure of Internalized Sexual Stigma for Lesbians and Gay Men (MISS-LG).
Items Male Female
DifficultyInfitOutfitDifficultyInfitOutfit
Sexuality
I2−0.860.910.91−0.661.111.03
I50.401.001.00−0.040.850.75
I80.970.920.800.600.800.72
I11−1.031.361.360.371.051.07
I140.530.760.73−0.261.201.18
Identity
I30.001.441.530.370.991.06
I60.430.810.800.851.161.24
I9−0.540.850.84−1.121.011.01
I121.120.880.87−0.540.910.91
I15−1.011.010.980.440.930.93
Social discomfort
I10.110.920.96−1.291.071.06
I40.120.840.850.720.980.98
I7−0.021.101.14−0.820.940.93
I100.080.880.880.010.990.96
I13−0.450.820.790.221.361.47
I16−0.210.960.960.230.880.86
I170.361.451.510.930.820.68
Infit = infit mean square; outfit = outfit mean square; values in bold indicate misfit (i.e., >1.5).
Table
Table 4. Concurrent validity results for the Measure of Internalized Sexual Stigma for Lesbians and Gay Men (MISS-LG).
Table 4. Concurrent validity results for the Measure of Internalized Sexual Stigma for Lesbians and Gay Men (MISS-LG).
VariablesMISS-LG SexualityMISS-LG IdentityMISS-LG Social DiscomfortHHRS-HomosexualityAAQ
MISS-LG Sexuality(0.67/0.64)0.752 ***0.937 ***0.220 ***0.427 ***
MISS-LG Identity0.784 ***(0.87/0.87)0.784 ***0.231 ***0.339 ***
MISS-LG Social discomfort0.700 ***0.640 ***(0.90/0.91)0.256 ***0.323 ***
HHRS-Homosexuality family0.190 ***0.270 ***0.384 ***(0.95/0.94)-
AAQ0.287 ***0.282 ***0.424 ***-(0.94/0.95)
Abbreviations: Measure of Internalized Sexual Stigma for Lesbians and Gay Men (MISS-LG); HIV and Homosexuality Related Stigma (HHRS); Acceptance and Action Questionnaire-II (AAQ). Upper triangular matrix reports correlation coefficients from male participants; lower triangular matrix reports correlation coefficients from female participants; diagonal values are McDonald’s omega, where the first values derived from male participants and the second values from female participants. *** p < 0.001.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Yen, C.-F.; Huang, Y.-T.; Potenza, M.N.; Tsai, T.-T.; Lin, C.-Y.; Tsang, H.W.H. Measure of Internalized Sexual Stigma for Lesbians and Gay Men (MISS-LG) in Taiwan: Psychometric Evidence from Rasch and Confirmatory Factor Analysis. Int. J. Environ. Res. Public Health 2021, 18, 13352. https://doi.org/10.3390/ijerph182413352

AMA Style

Yen C-F, Huang Y-T, Potenza MN, Tsai T-T, Lin C-Y, Tsang HWH. Measure of Internalized Sexual Stigma for Lesbians and Gay Men (MISS-LG) in Taiwan: Psychometric Evidence from Rasch and Confirmatory Factor Analysis. International Journal of Environmental Research and Public Health. 2021; 18(24):13352. https://doi.org/10.3390/ijerph182413352

Chicago/Turabian Style

Yen, Cheng-Fang, Yu-Te Huang, Marc N. Potenza, Tzu-Tung Tsai, Chung-Ying Lin, and Hector W. H. Tsang. 2021. "Measure of Internalized Sexual Stigma for Lesbians and Gay Men (MISS-LG) in Taiwan: Psychometric Evidence from Rasch and Confirmatory Factor Analysis" International Journal of Environmental Research and Public Health 18, no. 24: 13352. https://doi.org/10.3390/ijerph182413352

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop