Empirical Construct Validation of the Portuguese Version of the Restraint Scale

Aim: The main purpose of this study was to adapt the Restraint Scale (RS) to Portuguese and examine its psychometric properties, specifically its construct validity. Method: In this study, 238 normal-weight adults (82% women; Mean age = 36.6, SD = 15.0) participated in an online survey containing measures of Restraint Scale, Three Factor Eating Questionnaire, Dutch Eating Behaviour Questionnaire, and Body Dissatisfaction and Drive for Thinness scales. Results: Exploratory factor analyses corroborated the two-factors structure found in previous studies, in particular when three items without clear factorial assignment and low correlation were excluded. A final two-factors version of the RS containing seven items presented a very good fit to the measurement model and good internal consistency. Confirmatory factor analysis of the 7-items RS in relation to a three-factor model of overeating, dieting and body dissatisfaction measures revealed that the RS was the only restraint measure loading in all three factors. Conclusion: This suggests that the 7-items Portuguese version of the RS has good psychometric properties and unique features that lend it appropriate to identify and study unsuccessful chronic dieters. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


Predictive Validity
The predictive validity of the RS in determining overeating was initially investigated through a 'preloading paradigm' whereby individuals with varying scores consume a high caloric and palatable food (the preload) before a bogus taste test (Herman & Mack, 1975). Many such studies have found that while individuals scoring low on the RS (unrestrained eaters) tend to reduce their intake in the taste test as a result of the preload, those individuals scoring high (restrained eaters) usually consume more calories after the preload (e.g., Rotenberg & Flood, 2000;Westenhoefer, Broeckmann, Münch, & Pudel, 1994).
Notwithstanding the usefulness of the RS to assess the factors influencing overeating, some researchers have criticised the RS for 'criterion confounding' due to the inclusion of items (e.g., CD3) that refer to overeating behaviour (Stice, Ozer, & Kees, 1997;Stunkard & Messick, 1985;Wardle & Beales, 1987). According to this perspective, the often found relationship between high scores on the RS and disinhibited eating might not be the result of eating restraint only, but instead a confound due to the measurement of overeating and disinhibition in the RS.
This led to a conceptual clarification by Heatherton et al. (1988) about the type of eating restraint that the RS is supposed to measure. According to the authors, what the critics call a confounding can be justified as 'construct validity' if one acknowledges that the RS intends to measure both restraint and overeating behaviours that characterize chronic unsuccessful restrained eaters.

(Un)Successful Restraint Eating
The notion that the RS is suited to assess chronic unsuccessful eaters has received strong support when comparing it with other measures of eating restraint. The Cognitive Restraint Scale of the Three Factor Eating Questionnaire (TFEQ-R; Stunkard & Messick, 1985) and the Restraint subscale of the Dutch Eating Behaviour Questionnaire (DEBQ-R;Van Strien, Frijters, Bergers, & Defares, 1986) are both one-dimensional measures of restraint eating that have been construed to be independent from any tendency to overeat. Both subscales were developed in large part to avoid the ambiguities surrounding the RS two-factors structure and its supposed 'criterion confounding'.
However, accumulating evidence suggests that by excluding this confounding, these measures assess a different type of restrained eaters.

Construct Validity of the Portuguese Version of the Restraint Scale
Given the crucial role of the RS in understanding dieters' eating behaviours, the main purpose of the study was to adapt and validate the RS to the Portuguese population. Although the RS has been previously adapted to Brazilian Portuguese by Scagliusi et al. (2005), this study focused on test-retest reliability and discriminant validity in a sample of women with eating disorders. In the present study we examined key conceptual and psychometric issues associated with the RS based on reliability and construct validity investigation in a normal-weight sample of an adapted Portuguese version of this scale. We focused on validating the Portuguese RS factorial structure using exploratory factor analysis.
Furthermore, we examined the multidimensional construct validity of the Portuguese RS in comparison with other measures of restraint to test to what extent the RS is especially suited to identify unsuccessful chronic dieters.
We used confirmatory factor analysis to test the hypothesis that the Portuguese RS shows a unique multidimensional relationship with dieting, overeating, and body dissatisfaction.

Method Participants
Faculty and students of several Portuguese universities and academic institutes were requested by e-mail to participate in this online study as well as to divulge it through their mailing lists. As an incentive to participate, those who accessed the study were informed that two 25 Euro vouchers would be raffled among participants who completed the entire survey. The study followed the ethics guidelines of the hosting institution -Centro de Investigação e Intervenção Social, Lisbon, Portugal (CIS-IUL, 2013). A total of 451 subjects accessed the online study.
Among these, 120 participants (26,6%) did not finish the survey and were excluded from the analysis. One more participant was excluded for not providing us with explicit consent to use the data.
To allow for comparability with previous validity studies which have mostly focused on normal-weight samples (e.g., Van Strien et al., 2002, we used a Body Mass Index (BMI) interval of 18 ≤ BMI ≤ 25 as inclusion criterion. The BMI was calculated (weight/height 2 ) based on self-reported weight and height. Following this criterion, eight participants with a BMI under 18 and 79 participants with a BMI above 25 were excluded from data analysis.
Five other participants were also excluded because they did not report the necessary values to calculate their BMI. The final sample used for data analysis included a total of 238 participants (196 women and 42 men), with a mean age of 36.58 years (SD = 15.00) and BMI of 21.77 (SD = 1.82).

Measures Portuguese Version of the Restraint Scale
The RS is a self-report questionnaire developed to measure eating restraint (Herman & Mack, 1975). Originally, the RS was a 5-item questionnaire (Herman & Mack, 1975), later expanded to 11-items (Herman & Polivy, 1975).
In a revised version, the RS scoring system was simplified and the questionnaire reduced to ten items (see Table   1), which can be aggregated into a single score of eating restraint (Heatherton et al., 1988;Polivy, Herman, & Warsh, 1978). However, most studies examining the factorial structure of the RS have found a two-factors model: 1) Concern for Dieting (CD); and 2) Weight Fluctuation (WF) (Blanchard & Frost, 1983;Lowe, 1984;Overduin & Jansen, 1996;Van Strien et al., 2002). The CD subscale includes six items assessing preoccupations with food intake, while the WF subscale includes four items assessing typical fluctuations in body weight (see Table 1 for the original English version). CD items are answered in a 4-point scale, except for item CD1 (5-point scale). All WF items are answered in a 5-point scale.

CD1
Qual é a quantidade máxima de peso (em quilos) que alguma vez perdeu em 1 mês?  What is the maximum amount of weight (in kilos) you have ever lost within 1 month?

WF1
Qual é a quantidade máxima de peso ganho (em quilos) numa semana? 0-0.5; 0.5-1; 1-1.5; 1.  What is the maximum amount of weight gain (in kilos) within a week? The RS was translated to Portuguese from the English version by the authors. The Brazilian Portuguese version (Scagliusi et al., 2005) was used for semantic comparison. A bilingual translator unacquainted with the original version retroverted all items to English. Discrepancies between the original and the retroverted versions were resolved by consensus. When necessary, the Portuguese items were revised in order to accomplish semantic and lexical equivalency between the original and Portuguese versions. The final RS Portuguese version is presented in Table 1.

Portuguese Version of the Dutch Eating Behaviour Questionnaire
The Portuguese version of the DEBQ (Viana & Sinde, 2003;original version developed by Van Strien, Frijters, Bergers, et al., 1986;Van Strien, Frijters, Van Staveren, Defares, & Deurenberg 1986) has 33 items divided into three separate scales: 1) Emotional eating (13 items); 2) External eating (10 items); and 3) Restraint eating (10 items). All DEBQ scales from the Portuguese version showed good internal consistency and factorial validity in a previous validation study (Viana & Sinde, 2003). In the current study these scales presented excellent internal consistency: Cronbach's alpha was .954 for emotional eating, .863 for external eating and .910 for restraint eating.

Portuguese Version of the Three Factor Eating Questionnaire
The Portuguese version of the TFEQ (Moreira, Almeida, Sampaio, & Almeida, 1998 original version developed by Stunkard & Messick, 1985) has 26 items divided into three separate scales: 1) Cognitive restraint (thirteen items); 2) Disinhibition (seven items); and 3) Hunger (six items). In the present study we only included the disinhibition and hunger scales and replaced the cognitive restraint scale with the rigid control and flexible control subscales proposed by Westenhoefer, Stunkard, and Pudel (1999). These subscales incorporate many items from the TFEQ cognitive restraint scale, but are assumed to represent distinct components of eating restraint. Previous studies suggest that these subscales have different relations with unhealthy eating behaviors and successful weight control (e.g., Westenhoefer et al., 1999;Karhunen et al., 2012). Hence, by including these subscales we can better understand if distinct components of cognitive restraint show distinct relationships with overeating and body dissatisfaction.
A recent validation study of a Portuguese version of the rigid and flexible control subscales showed that both have good internal consistency and factorial validity (Poínhos et al., 2013). In the present study, we found good internal consistency in all scales: Cronbach's alpha was .826 for the disinhibition scale; .778 for the hunger scale; and .742 and .804 for the rigid and flexible control subscales, respectively.

Portuguese Version of the Body Dissatisfaction and Drive for Thinness Subscales
Following similar studies assessing the RS multidimensional structure, we included measures of motivational constructs that can lead one to adopt a restraint eating style (Laessle et al., 1989;Van Strien et al., 2002. Given that body image seems to be an important motive for dieting, we included the body dissatisfaction (nine items) and drive for thinness (eight items) subscales from the Eating Disorder Inventory (EDI-I; Garner, Olmstead, & Polivy, 1983) due to their relation with issues of body weight control and regulation, such as one's discontent with body shape and one's desire of being thin. The Portuguese version of the EDI-I, including both subscales, showed good internal consistency and factorial validity (Machado, Gonçalves, Martins, & Soares, 2001). In the present study, we found good internal consistency in both subscales. Cronbach's alpha was .888 for the body dissatisfaction subscale and.874 for the drive for thinness subscale.

Procedure
Participants were invited by email to participate in an online study about eating behaviours that would take 15 minutes to completion. Those who accessed the study received more information about its aim and were assured that their data would remain anonymous and confidential. After providing informed consent, participants responded to the different scales. The order of presentation of the scales was randomized. Participants also answered to some socio-demographic questions (e.g., age, gender, height and weight).

Data Analysis
The Portuguese RS factorial structure was examined through exploratory factor analysis to validate the latent structure and to assess possible inconsistencies in the factorial assignment of items to the expected factors.
Given the categorical nature of its response scales, we used the recommended weighted least squares means and variance adjusted (WLSMV) estimator (Muthén, du Toit, & Spisic, 1997). For exploratory factor analysis we used an oblique rotation to estimate factor loadings, assuming that the factors are conceptually correlated (Heatherton et al., 1988;Van Strien et al., 2002).
For the RS multidimensional construct validation, we first analyzed bivariate Pearson correlations between all measures. In addition, we assessed the RS convergent validity through confirmatory factor analyses using a Maximum Likelihood (ML) estimator. We assumed that the RS would be related with a three-factors structure including overeating, dieting and body dissatisfaction. For convergent validity, results of the confirmatory factor analysis with the RS loading on all three factors were compared to the results of the RS loading only on the dieting factor. This procedure was repeated for the two subscales CD and WF. We followed the same procedure for the other three measures of restraint eating to assess the unique multifactorial structure of the RS compared to these measures. We expected the other restraint measures to yield loadings only on the dieting factor and not on the overeating factor. Model fit indices, the significance of the chi-square difference test between models, and the scale's loadings in the three factors were used to assess the multifactorial hypothesis.
The adequacy of all the exploratory and confirmatory factorial models was determined based on the Chi-square statistic (χ 2 /df), Root-Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI) and Tucker-Lewis Fit Index (TLI). A RMSEA ≤ .050, CFI > .900 and TLI > .900 and a non-significant Chi-square (p ≥ .050), were considered acceptable indices of fit for the model (Byrne, 2012). Multivariate normality of distributions was examined by observed values of skewness and kurtosis. All distributions were within the range of values (± 2.0) expected in normal distributions (Muthén & Kaplan, 1985).
Chronbach's alphas were computed to assess internal consistency.
All analyses were conducted using SPSS 22 and Mplus 7. Given that Mplus does not provide parallel analyses for exploratory factor analyses with categorical data, this analysis was conducted on sample polychoric correlations as implemented in R package psych (Revelle, 2016 Table 2 shows the Cronbach's alpha of the RS (α = .780), CD (α = .699) and WF (α = .711) subscales. It also shows the correlations between each item and its respective subscale as well as the Cronbach's alpha if the item is deleted. Only item CD6 deletion would increase the Cronbach's alpha in both the RS (α = .787) and CD subscale (α = .729). Given the inconsistent factorial assignment of some items on previous studies, we decided to conduct an exploratory factor analysis to assess possible psychometric issues with the Portuguese version in this regard. good fit (see Table 3). A forced one-factor solution presented poor fit, χ 2 (35) = 183.74, p < .001, RMSEA = .134, 90% CI [.115, .153], CFI = .855, TLI = .813, but a forced two-factors solution obtained adequate goodness of fit (see Table 3). Table 3 shows each item's factors loading on both the two-and three-factors solutions after oblique rotation. For the two-factors solution (explaining 54.5% of total variance), four items from the CD subscale (items CD1, CD2, CD4, CD5) had the highest loadings on the first factor (≥ .596). Three items from the WF subscale (items WF1, WF2, WF3) had the highest loadings on the second factor (≥ .679). Three items -two from the CD subscale (CD3, CD6) and one from the WF subscale (WF4) -presented similar small loadings in both factors (≤ .389).

Restraint Scale's Reliability and Factorial Validity
For the three-factors solution (explaining 65.6% of total variance), the same four items from the CD subscale (items CD1, CD2, CD4, CD5) had also the highest loadings on the first factor (≥ .583). However, on the second factor only two items from the WF subscale (WF2, WF3) had the highest loadings (≥ .696). Compared with the two-factors solution, item WF1 had a lower loading on the second factor of the three-factors solution (.409) and item CD3 had the third highest loading (.499). However, both items had significant loadings on other factors (see Table 3). Similar to the two-factors solution, item CD6 had very small and not significant loadings on all three factors (≤ .162).
These results suggest that, although a three-factors solution presents a better goodness of fit, a two-factors model also has satisfactory fit. It seems also clear that some items do not present a consistent factor loading.
Item CD6 appears to be inadequately represented in both factorial solutions and had the lowest correlation with other items from the CD subscale (see Table 2). Thus, item CD6 was excluded from further analysis using the

RS.
Items CD3 and WF4 also did not present a consistent factorial representation, showing statistically significant, medium to small loads, on more than one factor. This inconsistency has been found in previous studies (e.g., Blanchard & Frost, 1983;Overduin & Jansen, 1996;Van Strien et al., 2002). Moreover, these items have been criticized on conceptual grounds: item CD3 for possible criterion confounding (e.g., Stice et al., 1997;Wardle & Beales, 1987; but see Heatherton et al., 1988); and item WF4 for referring to history of overweight that does not necessarily equate with weight fluctuations (Lowe, 1984). Given these conceptual and psychometric issues, we decided to exclude these two items from further analysis.
To examine the validity of the new proposed 7-item RS, we repeated the exploratory factor analysis. This new RS version resulted in a two-factors solution (explaining 66.3% of total variance) based on the eigenvalue > 1 criterion and the parallel analysis. The eigenvalues were 3.178 for the first factor and 1.461 for the second factor. This two-factors solution presented a better goodness of fit (see Table 3) than the two-and three-factors solutions of the 10-items RS version. It also presented a much better fit than a one-factorial model of the 7-item RS version,  Table 3 shows each of the 7-items' factor loading for the two-factors solution after oblique rotation. All four CD items had higher loads on the first factor (≥ .589) and all three WF items had higher loads on the second factor (≥ .614). Moreover, the Cronbach's alpha was .739 for the 7-item RS, .737 for the 4-item CD, and .694 for the 3item WF (see Table 2).   Table 4 shows the zero order Pearson correlation coefficients between all measures. The highest correlation of the 7-items RS was obtained with one measure of body dissatisfaction (EDI drive for thinness: r = .721). The next two highest correlations were with other dieting measures (TFEQ rigid control and DEBQ restrained eating: r ≥ .620). Albeit smaller, the 7-items RS also showed significant correlations with measures of overeating (r > .230), the highest being with the TFEQ disinhibited (r = .443) and DEBQ emotional eating (r = .483). The 4-items CD subscale showed very similar results, with only slight variations on correlation values. However, the 3-items WF subscale obtained smaller correlations with most measures, particularly with other dieting measures (.219 ≤ r ≥ .360) and body dissatisfaction (r = .342). Overall, these results lend some support to the RS and its subscales multidimensionality.

Construct Validity of the RS
We conducted confirmatory factor analyses to further test the multidimensional construct of the RS and its subscales compared with other restrained eating measures. Following the three-factor structure proposed by Laessle et al. (1989) and Van Strien et al. (2007), we tested a model of three latent variables corresponding to: 1) overeating and disinhibited eating, 2) dieting and restrained eating, and 3) body dissatisfaction. Four measures were assumed to load on the overeating factor (TFEQ disinhibition, TFEQ Hunger, DEBQ emotional eating, DEBQ external eating), three on the dieting factor (TFEQ rigid control, TFEQ flexible control, DEBQ restrained eating), and two on the body dissatisfaction factor (EDI drive for thinness and body dissatisfaction).

Discussion
The present study aimed to adapt the revised RS to Portuguese and assess its reliability, factorial and construct validity. The results from the factorial analyses confirm the adequacy of the RS two-factors structure and its multidimensionality. However, in line with previous studies, the psychometric properties of some items are questionable and unclear.
Item CD6 ("How conscious are you of what you are eating?") appears to be uncorrelated with the RS and it is poorly represented in the scale's factorial structure. This is inconsistent with previous validation studies that have found consistent significant loadings on the CD factor. A possible reason for this inconsistency could be the ambiguous meaning of its content in Portuguese language as it relates with restraint. The item can be interpreted as one being aware of food intake without implying an increased concern with dieting. While 'being conscious' in decide to include it they should assess how this impacts the reliability of the scale and how the item relates with the RS subscales.
Items CD3 and WF4 also showed inconsistencies in their factorial assignment, in line with previous studies (e.g., Blanchard & Frost, 1983;Overduin & Jansen, 1996;Van Strien et al., 2002), suggesting that they could be measuring other dimensions. Moreover, these items have also been criticised by possible 'criterion confounding' (Lowe, 1984;Stice et al., 1997;Wardle & Beales, 1987; but see Heatherton et al., 1988). Hence, their inclusion in future studies should be justified by a rigorous psychometric assessment and validation.
It is important to note that although item CD3 may be an important factor in the relationship between the RS and overeating (Stice et al., 1997), in this study, after excluding it, the RS still obtained a positive significant correlation with different overeating constructs (e.g., Disinhibition, Emotional Eating and External Eating) and showed a significant load on the overeating factor. Van Strien et al. (2007) found a similar result, suggesting that other RS items might contribute to its relationship with overeating despite no explicit references to disinhibited eating. Research has helped to determine that the RS seems to assess restraint of food intake and overeating and, thus, the construct of restraint identified by the RS should incorporate both dieting and overeating behaviours expected to cause greater weight fluctuations as a result of a chronic cycle of restraint and overindulgence (Heatherton et al., 1988).
Notwithstanding the psychometric and conceptual issues of the three items, the Portuguese RS corroborates the adequacy of the two-factors structure, particularly when these items are excluded. In fact, the exploratory factor analysis including only the remaining 7-items presented a much better fit when two factors were extracted. In this two-factors solution the remaining items presented high loadings on the expected subscales and good internal consistency.
The Portuguese RS also supports the multidimensional structure found in previous construct validation studies (Laessle et al., 1989;Van Strien et al., 2007). The RS and its CD subscale were the only restraint measures that had significant associations with dieting, overeating and body dissatisfaction. The WF subscale was only associated with dieting and overeating. In contrast, the restraint subscales of the TFEQ and the DEBQ were not associated with the overeating factor, although the TFEQ Flexible Control subscale and the DEBQ restraint scale were associated with both dieting and body dissatisfaction. Overall, these results corroborate the findings of Van Strien et al. (2007) who found a similar pattern except for the association between the DEBQ restraint scale and body dissatisfaction.
We expected the TFEQ Rigid Control subscale to show a stronger relation with overeating and/or body dissatisfaction compared with the Flexible Control subscale based on previous studies that suggest that rigid cognitive control is associated with unhealthy eating behaviours and unsuccessful weight regulation (Karhunen et al., 2012;Westenhoefer et al., 1999). However, that was not the case, which suggests that the RS could be more adequate than the Rigid Control subscale to identify dieters that tend to overeat.
In sum, the present study appears to validate the RS multidimensional relationship with dieting, overeating and body dissatisfaction. More relevant to future studies on eating restraint, the results from the comparison between the different restraint scales lend support to the hypothesis that the RS is especially suited to identify unsuccessful chronic dieters, whose eating behaviour is characterized by periods of restraint and overeating. Hence, when using the RS one should keep in mind the inherent bias of this measure in selecting dieters with a tendency towards disinhibition (Van Strien, 1999). This makes the RS inappropriate to disentangle disinhibition from restraint eating and, consequently, inadequate to differentiate between successful and unsuccessful dietary restraint (Van Strien, 1999). In contrast, uni-dimensional restraint scales from the TFEQ and DEBQ, as "pure" measures of eating restraint, can play an important role in the study of (un)successful dietary restraint by allowing us to separate restraint eating from other eating behaviours. Although high scores on these measures can be ambiguous in terms of how (un)successful an individual is in dietary restraint (Ogden, 1993), by complementing scores on pure measures of restraint with scores on overeating and disinhibition subscales it is possible to unconfound successful and unsuccessful dieters with high restraint scores (Van Strien, 1997, 1999Westenhoefer et al., 1994 These differences between restraint measures should be taken into account when designing and interpreting future studies. For instance, researchers interested in understanding self-regulation failures that lead to overconsumption should benefit from using the RS as it identifies individuals prone to be unsuccessful in eating restraint, with tendency to overeat. In contrast, uni-dimensional restraint measures, when complemented with other eating behaviour measures, should be considered if the goal is to differentiate dieters with low versus high susceptibility towards failure of dietary restraint (Van Strien, 1999). Such distinction is crucial to understand when, how and to whom dietary restraint can be a successful strategy to achieve and maintain a healthy weight.
Future validation studies should test the invariance of the multifactorial structure of the Portuguese RS across different weight status groups -normal-weight vs. overweight/obese populations. Likewise, analyzing its discriminative validity by comparing RS scores between these two population groups is warranted, as in previous validation studies the RS was considered to be more appropriate for assessing restraint in normal-weight than in overweight samples (e.g. Van Strien et al., 2007).
The adaptation and validation of the RS is an important first step to investigate its predictive validity among the Portuguese population. The RS can be quite valuable in identifying motivational and cognitive factors of chronic dieters that may contribute to impulsive food consumption (e.g., Houben et al., 2012;Papies et al., 2008;Veenstra & de Jong, 2010), as well as in identifying vulnerable individuals in the population. We hope the Portuguese RS can contribute to increase our knowledge about eating behaviours and disorders, particularly among chronic dieters.

Funding
This research was supported by a PhD grant from the Portuguese Foundation for Science and Technology (PD/BD/111222/2015) awarded to the first author.