Psychometric properties comparison among diverse versions of the invalidation childhood environmental scale (ICES) in Peruvian adults

Introduction: The current tendency regarding psychometric assessment is to use brief versions of measurement tools. Objective: The present study sought to compare the psychometric properties of various versions in the length of the Invalidating Children Environment Scale (ICES). Method: Three hundred and twelve Peruvian university students participated as a sample. Confirmatory factor analysis was performed

Invalidating environments (IE) offer minimization, trivialization, and extreme responses regarding an individual's emotional, cognitive, and behavioral responses (Linehan, 1993).IE is characterized by (a) expressive rejection of private experiences through extinction (i.e., non-reinforcement) and blocking principles (i.e., punishment, criticism), (b) intermittent reinforcement of more high-intensity emotional expressions and patterns' perpetuation consequently, and (c) problems' oversimplification through supposedly easy solutions (Boggiano & Gagliesi, 2020).IE between childhood and adolescent stages generates a series of repercussions on mental health and associates.Various psychopathologies are developed consequently, usually underlying a key factor denominated as emotional dysregulation (Linehan et al., 2007).In that sense, depression (Boring et al., 2021), anxiety (Ramírez & Espinoza, 2021), borderline personality disorder (Keng & Soh, 2018), low self-esteem and self-compassion (Keng & Wong, 2017;Bontempo, 2022) and lower quality and satisfaction with life (Elzy, 2013;Stefanatou et al., 2022) are examples of this dynamic, being highly identified in the current scene.On the contrary, validating environments promote support and a more disposition toward one's emotional repertoires, promoting healthier individual development (Stoewsand, 2021).These contexts are beneficial in promoting cognitive-emotional expression, freedom, and acceptance, as well as situationally flexible and functional repertoires (Hopko et al., 2003;Koerner, 2012).Certain variables that may contribute to invalidating environments have been identified in the Peruvian context.In that sense, different stressors can be noticed, usually associated with interpersonal relationships, economic issues, and public safety difficulties (Cassaretto et al., 2021).These stressors affect college students, especially when they become responsible for their own expenses.The academic demands, schedules, and pre-professional practices also increase discomfort.Since this population could be considered a vulnerable community, lacking adequate emotion regulation and distress tolerance skills is highly prevalent, increasing their emotional difficulties and promoting the invalidation of other people's emotional responses (Boggiano & Gagliesi, 2020).To analyze such prevalent constructs, the Invalidating Childhood Environment Scale (ICES) was initially developed by Mountford et al. (2007) to evaluate the retrospective interaction perceptions that the adults had with both parents during childhood and adolescent stages, considering Linehan's (1993) conceptualization of IE.Evidence of convergent and discriminant validity was preliminarily obtained, but other relevant psychometric properties were not identified to determine the internal structure of the test (AERA et al., 2014;Mountford et al., 2007).Various studies were developed to determine the psychometric properties of the ICES.In this sense, Alpay et al. (2018) developed a Turkish version, in which a unifactorial structure was proposed through confirmatory factor analysis with acceptable fit indices for both parental figures and internal consistency co-efficients within expected.Regarding this adaptation, although statistical results were obtained according to current standards, not discerning patterns of invalidation-validation would not allow the detailed diagnosis of the interactions between the child and their parents.Along the same lines, Robertson et al. (2013) opted to eliminate five items (all belonging to the validating environments factor) to improve the fit of the instrument.The adjustment indices significantly enhanced for both parental dimensions based on such conditions.Secondly, the same dynamic is visualized in Alpay et al.'s adaptation. (2018) by not considering the components of validating responses from both parents.Finally, in a recent adaptation, Holden et al. ( 2021) stipulated an internal three-factor structure, considering maternal invalidation (items 1, 10, 11, and 13), paternal active invalidation (items 4, 7, 10, and 11), and paternal passive invalidation, which considers the items with reversed scoring (items 5, 8, 12 and 14).Certain aspects of this study need improvement, mainly in conceptualizing passive invalidating responses.When considering validity evidence based on content from that dimension, the items primarily focus on the assessment of supportive environments (validating responses) from both parents, according to Mountford et al. (2007).In other words, considering passive invalidating responses as validating repertoires does not have much coherence at a conceptual level.As a complement, recent Latin American proposals have been developed to avoid these previously mentioned methodological aspects (Puddington et al., 2017;Puddington et al., 2022;Okumura-Clark et al., 2023), determining a clear two-factor structure for each parent (validating and invalidating responses).Although successful results have been identified, current trends emphasize the relevance of constructing much shorter tests for assessment and research aims (Sleep et al., 2021).Under these premises, the adaptation of the ICES is highly relevant, in addition to identifying possible brief versions of this exact instrument, considering that this test responds to needs at the clinical assessment level.Therefore, this research aims to determine the psychometric performance of various reduced versions of the Invalidating Childhood Environment Scale (ICES) in Peruvian samples.

Design
Our study was cross-sectional.
(86.2%).Most participants were from Lima, the capital of Peru (96.8%).The prevalence of anxiety and depression symptoms was 25.6% and 26.0%, respectively, based on established cutoffs of our self-report measures.
To determine the sample size, the calculator proposed by Kim (2005) for structural equation models was used.In the extensive version of the test, considering a CFI of .95,two factors composed of 10 and 4 items, respectively, an estimated average factor loading of .65, a latent compression of .60,and a power of 80%, it was estimated a minimum sample size of 183 participants.The exact estimators were considered for its brief version, except for the items per factor (5 and 4, respectively) and the estimated factor loading (.70), with the recommended minimum number of 181 participants.

Measurements
Sociodemographic record.Data such as biological sex, gender, age, sexual orientation, and province of origin were requested.Mountford et al. (2007) to assess the retrospective adult's perception of validating and invalidating responses from both parents (mother and father) during the formative years (0-18 years of age).It was later Spanish adapted by Puddintong et al. (2017), and years later, a Peruvian adaptation was developed (Okumura-Clark et al., 2023).It comprises 14 items for each parent and 5 Likert scale alternative responses (1=Never, 5=All the time).Several studies have identified the internal structure of the test composed of two dimensions (parental's validating and invalidating responses).Adequate internal consistency coefficients were identified in the Peruvian adaptation (ω = .85-.87) (Okumura-Clark et al., 2023).Patient Health Questionnaire-2 item (PHQ-2).This test is a reduced version of PHQ-9, composed of two items that measure vital components of depressive symptomatology experienced over the last two weeks.Each item can be answered with four alternative responses (0=Not all, 3=Nearly every day).An adequate internal consistency coefficient has been identified in the derived scores obtained in Peruvian college samples (α = .80)(Caycho-Rodriguez et al., 2020).In systematic review studies, the sensitivity and specificity of the test have been identified as exceeding 60%, leading to its recommendation for determining depressive symptoms (Varela Chávez & Guayusamin Tipanta, 2023).Meta-analytic data suggest that a cutoff score of ≥ 3 yields acceptable sensitivity (approximately .72-.76) and specificity (approximately .85-.87) (Levis et al., 2020;Manea et al., 2016).Generalized Anxiety Disorder Scale-2 (GAD-2).This test is a reduced version of GAD-7, composed of two items that assess the main anxiety symptoms over the last two weeks.A 4-option Likert response has been considered (0=Not at all, 3=Nearly every day).An acceptable internal consistency coefficient has been obtained in Peruvian samples (ω = .81)(Baños-Chaparro, 2022).In its extended version, high levels of sensitivity (73.3%) and specificity (67.3%) have been identified (Zhong et al., 2015).According to an international meta-analysis, the GAD-2 with a cutoff score of ≥ 3 has a sensitivity of .76 and a specificity of .81(Plummer et al., 2016).

Procedure
Data collection was facilitated using a digital format (Google Forms), conducted within academic settings, and in collaboration with a private Peruvian university and its faculty members.The application process required 10 to 15 minutes per participant.All the data was collected in September 2023.It should be noted that participants in this study were undergraduate students from a university located in the southern zone of Lima, Peru.This county is known for its working-class population and has a history of community-driven development despite economic challenges.

Data analysis
First, a series of confirmatory factor analyses were conducted using the WLSMV method to test the following models: (a) the original unidimensional 14-item model, (b) the 14-item twofactor model proposed by Puddington et al. (2017), and (c) the reduced two-factor model proposed by Okumura-Clark et al. (2023).The fit was assessed through the following approximate indices: Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), Root-Mean-Square Error of Approximation (RMSEA) and Standardized Root-Mean-Squared Error (SRMR).Following the usual guidelines, values of CFI > .95,TLI > .95,RMSEA < .06,and SRMR < .08 would indicate a good fit (Hu & Bentler, 1999).Consistency reliability was examined using the categorical omega coefficient (Green & Yang, 2009).Likewise, the possibility of proposing a brief version based on inspecting the factor structure was examined.Finally, evidence of associative validity was examined through Spearman correlations with anxiety and depression scores.

Ethical aspects
At first, a macro-research proposal was developed to analyze the associations between early experiences related to a person's development, emotion regulation transdiagnostic variables, and mental health components, which considered the psychometric analysis of the measurements as a previous research step.This proposal was revised and later approved by the research ethics commission of the Dirección de Investigación de la Escuela de Psicología de la Universidad Autónoma del Perú in June 2023.During the preliminary testing phase, informed consent was employed, outlining the aim of the study and ethical principles, including confidentiality, anonymity, and voluntary participation.All participants accepted and signed the informed consent to participate in the study.

Confirmatory Factor Analyses
The unidimensional model was first tested on the Mother Scale.As presented in Table 1, the model fit was unacceptable.Puddington et al.'s (2017) two-factor model performed slightly better, but the fit was still mediocre.On the other hand, Okumura-Clark et al.'s (2023) reduced model had a notably better fit.Examination of the modification indices revealed that allowing the errors of items 1 ("My mother would become angry if I disagreed with her") and 11 ("My mother would explode with anger if I made decisions without asking her first") to correlate would increase model fit.Since this modification made conceptual sense (both items referred to reactions of anger), it was added to the model.The final model had an excellent fit (Table 1).As to the Father Scale, the unidimensional and Puddington et al.'s (2017) model showed a mediocre fit, as expected.Similarly, Okumura-Clark et al.'s (2023) model also had a sub-optimal fit for this scale.Modification indices revealed that item 2 ("When I was anxious, my father ignored this") was problematic since it required a cross-loading to be added.After dropping this item, the fit improved.Modification indices showed that allowing items 1 and 11 to correlate would also improve model fit for the Father Scale, so we decided to add it to the model (Table 1).As can be seen in Figures 1A and 2A, the final models were identical for both scales.

Development of a Short Version
To achieve a more economical version of the ICES, we iteratively dropped items so that no correlated errors were included and only the most considerable factor loadings remained.We also kept the same items in both scales: 4, 6, 7, 10, and 13 (invalidating behaviors) and 5, 8, 12, and 14 (validating behaviors).Both short versions had an excellent fit (Table 1).They are visually presented in Figures 1B and 2B.

Internal Consistency Reliability
In the Mother Scale's final long model (Figure 1A), internal consistency reliability was estimated to be adequate both for invalidating (ω = .88)and validating behaviors (ω = .86).Reliability was nearly identical for the short version of the Mother Scale (ω invalidating = .88,ω validating = .86).Similarly, the Father Scale's extended version (Figure 2A) had excellent reliability for the invalidating subscale (ω = .92),and excellent reliability for the validating dimension (ω = .87).The short version of the Father Scale also had excellent internal consistency reliability (ω invalidating = .93,ω validating = .87).

Association with Anxiety and Depression
Table 2 displays the latent correlations between the ICES scales (as well as their short versions) and two variables relevant to psychopathology: anxiety and depression.It can be seen that invalidating behaviors (both from the mother and the father) are associated with higher psychopathology.Also, to a lesser extent, validating behaviors are related to less psychopathology.It can also be seen that short versions perform similarly to the lengthier ones, even though some evidence of attenuation is observed.

DISCUSSION
In recent decades, invalidating environments have been highly studied due to their psychological impact on the individual and are conceptualized as one of the key concepts in various contemporary therapies (Boggiano & Gagliesi, 2020).Although these elements have been considered in intervention proposals, psychometric assessment has yet to be developed.ICES is highly regarded among the main current instruments in this field (Mountford et al., 2007), translated into Spanish, and adapted to the Peruvian context (Puddington et al., 2017;Okumura-Clark et al., 2023).Associated with this, current perspectives describe the relevance of using shorter instruments due to their impact on evaluation and research domains (Sleep et al., 2021).Under these premises, this research aimed to determine the psychometric functioning of various ICES versions in Peruvian samples.

Main findings
Firstly, we sought to determine the internal structure of the test through the confirmatory factor analysis method.Various proposals were tested to identify the best-fit indices.The first proposal consisted of a unidimensional approach; however, the fit indices could have been better, which would be supported by the theoretical foundations of the initial ICES version (Mountford et al., 2007).Secondly, TLI and RMSEA were poorly fit on the Mother Scale based on Puddington et al.'s proposal (2022).Thirdly, it was sought to replicate the Peruvian version of the   the test results in its reduced version (Kline, 2020).On the other hand, reliability evidence was determined in the derived scores from the tests.Although Cronbach's alpha coefficient is the most used to assess reliability, it has been identified that for its correct estimation, it requires compliance with a series of assumptions that are difficult to obtain in the psychological field, such as tau-equivalence, unidimensionality and that the variables (items) must be continuous and with a normal distribution (McNeish, 2018).Due to this, the omega coefficient was chosen, which does not require compliance with such rigid assumptions, in addition to working with ordinal variables (McNeish, 2018).In that sense, the omega internal consistency coefficients were similar in both the long and short versions.Due to these results, the scores derived from the short version are just as precise and consistent as those from the extended version.

Comparison with other studies
Psychometric comparisons were performed between various versions of the ICES.Among the main findings, it was determined in the first instance that the unidimensional model in both scales and Puddington et al.'s proposal (2017) on the mother scale did not meet the minimum expected threshold at the level of adjustment indices.Another critical point was that better-fit indices were identified in proposals that used correlations between errors and item elimination; however, these processes would not be the most recommended in the sense that maintaining the original factorial structure of the test is preferred.Finally, Okumura-Clark et al. 's proposal (2023) and the 9-item version demonstrated excellent fit indices; however, several authors emphasize the relevance of the principles of parsimony in short versions of the test (Sleep et al., 2021), which is the reason the short version is recommended for assessment and research aims.

Strength and limitations
Despite its strengths, the present research has several limitations worth mentioning.The non-probabilistic sampling method limits the generalizability of the results to other Peruvian contexts.Additionally, we did not obtain psychometric properties associated with equity, considered necessary in this field of study.We could not examine measurement invariance regarding gender or age in our data, as doing so would have resulted in some groups being too small (fewer than 100 participants), thus potentially leading to biased or unstable results (Dimitrov, 2010).Finally, other associated factors, such as different types of invalidation and elements of therapeutic models of psychotherapy, were not considered.

Implications in research
This study is a relevant precedent at a research level, in the sense that this is the first reduced ICES proposal, having obtained a series of psychometric evidence of validity and reliability according to current standards for the adaptation and construction of psychological tests (AERA et al., 2014;ITC, 2017).Research proposals may be made about this field of study.Among them, we highlight and identify other psychometric properties of this new version, such as the test's incremental validity and equity studies.

Conclusion
This study has allowed the identification of a brief version of 9 items, which presents the same two-dimensional factor structure and better-fit indices compared to other previously developed versions.Contrastingly, evidence of convergent and divergent validity and acceptable internal consistency coefficients were identified.

Figure 1 .
Figure 1.Long and Short Version of the ICES Mother Scale.Note.Standardized coefficients are presented.For simplicity, error terms are not displayed.

Figure 2 .
Figure 2. Long and Short Version of the ICES Father Scale.Note.Standardized coefficients are presented.For simplicity, error terms are not displayed.

Table 1 .
Confirmatory Factor Analysis Fit Indices of the Models Under Study Note.All the analyses were conducted using the WLSMV estimator.

Table 2 .
Correlations of Long and Short Versions of the ICES with Anxiety and Depression.