Public Perceptions on Elite Sport’s Societal Outcomes: A Validated Scale in a European Context

Studies have demonstrated the potential positive (e.g., inclusion) and negative (e.g., pollution) outcomes that elite sport has on society. Over the years, the interest in measuring these societal outcomes has grown. This research extends previous studies that emphasize the need for a validated scale to accurately measure public perceptions of elite sport’s outcomes on society. The purpose of this study was to validate the Mapping Elite Sport’s potential Societal Impact (MESSI) scale in a European context. Using a quantitative research design, the MESSI scale was administered to 10,400 European citizens from seven countries (i.e., Belgium, Czech Republic, Finland, France, The Netherlands, Poland, and Portugal). Confirmatory factor analysis was conducted to assess the validity and reliability of the measurement instrument. Results indicated a psychometrically acceptable 68-item 10-dimension MESSI scale. The European population perceived more positive than negative potential societal outcomes of elite sport. The study confirms that the MESSI scale is a valid, reliable, and robust instrument for measuring the perceptions of the potential positive and negative societal outcomes of elite sport. The use of this scale might provide valuable insights for sport policymakers.


Introduction
Across nations, policymakers are required to legitimise their increasing investments in elite sport (De Bosscher et al., 2021).It is argued that elite sport can trigger a wide range of positive societal outcomes, such as inclusion (Naess, 2023), national pride (Mutz & Gerke, 2024), and sports participation (Eather et al., 2023).Nonetheless, research also shed light on the negative societal outcomes elite sport can spawn, such as increased alcohol consumption (Bandura et al., 2024) or stimulation of unethical conduct (Heerdt & Roorda, 2023).However, the evidence base mapping the potential positive and negative societal outcomes is still scarce and fragmented (De Rycke & De Bosscher, 2021;Funahashi et al., 2015).Nevertheless, since the tax-paying population is the primary sponsor of elite sport, these investments should yield additional societal values and benefits for the public (De Rycke et al., 2019;Van der Roest & Dijk, 2021).In an attempt to facilitate these societal outcomes of elite sport, Van der Roest and Dijk (2021) argue that taking into account the public's perception of elite sport's societal outcomes is essential.
Despite increasing research, the body of evidence regarding societal outcomes primarily includes mixed and contradictory findings (De Rycke & De Bosscher, 2019).This is often the result of a lack of methodological clarity, as there are a wide range of concepts and definitions used to describe the phenomenon (e.g., societal outcomes, societal impact, public values, societal values, etc.).Several researchers (e.g., Lee et al., 2013;Van Bottenburg et al., 2012) have attempted to research this complex phenomenon.For example, Lee et al. (2013) aimed to capture sport's societal outcomes in five dimensions, including social capital, collective identities, health literacy, well-being, and human capital.However, the employed assessments vary in format and only investigate part of the (perceived) societal outcomes.The most comprehensive framework is the Mapping Elite Sport's potential Societal Impact' (MESSI) framework, proposed by De Rycke and De Bosscher (2019).An extensive literature review resulted in 391 studies that showed evidence of 79 distinct societal outcome areas of elite sport (47 positive; 32 negative) clustered in 10 dimensions: (i) social equality and inclusion; (ii) collective identity, connection and pride; (iii) ethics and fair play; (iv) feelgood and passion; (v) fans and (media) attraction; (vi) prestige and image; (vii) athletes ability and quality of life; (viii) sport participation and health; (ix) sponsors and commercial activity; and (x) local consumption and living conditions.The authors assume that 'intrinsically, elite sport is neither beneficial nor harmful' (De Rycke & De Bosscher, 2019, p. 486), meaning that every dimension includes both positive and negative outcomes.For instance, elite sports' capacity to boost national pride, but chauvinism, when this sentiment becomes excessive, is encapsulated in the dimension of collective identity, connection, and pride.
Subsequently, the authors developed two MESSI scales (De Rycke et al., 2019;De Rycke & De Bosscher, 2021).In developing their 32-item scale (De Rycke et al., 2019), the authors reduced the initial pool of 79 items to 32 items after checking content validity and performing factor analysis.Although the scale explained 71.9% of the variance in the constructed model, it failed to adequately capture all societal outcomes.For instance, the scale missed important information, such as items related to sports participation, a frequently investigated societal outcome (De Cocq et al., 2021).Therefore, De Rycke and De Bosscher (2021) used the 73 items (i.e., remaining items after content validity, but before factor analysis) in their follow-up study.The scales were developed for the specific Flemish context 1 .
Besides, the researchers used five-point Likert scales, which are often debated in the literature because they might not accurately measure participant responses (Finstad, 2010).
It remains necessary to validate the measurement instrument further to ensure its applicability across studies and cultural contexts (De Cocq et al., 2021;De Rycke et al., 2019).Therefore, this study aims to validate the Mapping Elite Sport's potential Societal Impact (MESSI) scale in a European population.Building on the extensive 73-item MESSI scale, this study seeks to provide a comprehensive, validated, and reliable instrument that has the potential to enhance our understanding of perceived positive and negative societal outcomes of elite sport.Furthermore, this study offers a valuable instrument for policymakers to legitimise elite sport investments (De Rycke et al., 2019).Additionally, understanding public perceptions of elite sport's societal outcomes is crucial, as the public is the main sponsor of elite sport (De Bosscher et al., 2021).

Data Collection and Procedure
This study was part of an Erasmus+ Sport project: 'Athletes 4 Society: Empowering The Public Value Of Sport Through Athletes As Role Models'.Ethical approval was granted by the Ethics Committee of Vrije Universiteit Brussel (i.e., coordinator of the project).Data were collected in August 2021 in seven European countries (i.e., Belgium, Czech Republic, Finland, France, The Netherlands, Poland, and Portugal).A research service company was appointed to obtain a representative sample.Participants were recruited based on age, language, gender, and social class.In total, 10,400 participants were invited to participate in the study.The questionnaire was back-to-back translated into six languages (i.e., Czech, Dutch, Finish, French, Polish, and Portuguese) by native speakers in the project's consortium.

Instrument
This study adopted a scale validation procedure as described in previous literature (e.g., Yamaguchi et al., 2022).As mentioned, the study builds upon prior research that seeks to measure public perceptions of the societal outcomes of elite sport through the application of the MESSI scale (De Rycke & De Bosscher, 2021;De Rycke et al., 2019).Scale items were constructed based on the 73-item MESSI scale.Similar to the study of De Rycke and De Bosscher (2021), bipolar statements were used for each item to measure the perceptions of individuals regarding the potential positive and negative societal outcomes of elite sport.For example: "In general, I perceive that elite sport . . .statement) encourages young people to do sport themselves versus (negative statement) discourages young people to do sport themselves".In contrast, this study used a seven-point Likert scale (1 = strongly disagree to 7 = strongly agree) instead of a five-point Likert scale.Research indicates that a using seven-point rating scale is more suitable as this (a) improves validity (Taherdoost, 2019), (b) enables participants to express their feelings more adequately (Finstad, 2010), (c) conveys more useful information (Bendig, 1954), and (d) is most accurate and easier to use (Diefenbach et al., 1993).Scores with a critical value of 4.5 or higher (cut-off value as used by Hirschfeld and Thielsch (2015)) were considered as a perceived positive outcome of elite sport.
In order to make the scale applicable to an international context, focus group discussions with international experts (N = 14) and academics (N = 5) in the field of elite sport's societal outcomes were carried out to ensure content validity.These focus group discussions led to a number of adaptations: (a) five items were removed due to their close resemblance to other remaining items, thereby measuring identical societal outcomes ('promotes/obstructs human justice'; 'provides athletes with a profound/minimal role model function'; 'enhances/declines health awareness'; 'awards sport organisations through ticketing and licensing a profitable/losing source of income'; 'offers sponsors financial profits/losses'), (b) one item regarding the connecting effect of elite sport was added ('produces athletes and teams with whom people feel connected or identify/dis-like or don't want to be associated with'), (c) to establish a clear distinction between adults and youth, the original item regarding sports participation was divided into two items ('encourages/discourages young people to do sport themselves'; 'encourages/discourages the adult population to do sport themselves'), (d) three items were reclassified under a different dimension as they were found to be incongruent with their initially designated dimension.The item 'preserves/ruins nostalgia and traditions' was transferred from ethics and fair play to happiness and experiences; the item 'boosts/deteriorates the image of sport in general' was transferred from international prestige and political power to fans and media; and the item 'discourages/encourages the use of performance-enhancing drugs among elite athletes' was transferred from athletes' quality of life and competences to ethics and fair play, and (e) enhancement of the formulation of statements and refining linguistic expressions (e.g., the rather vague item 'provides athletes/teams where people can relate to in a positive/negative way' was clarified by explicitly stating the, whether or not important, inspirational effect of role models: 'is important/unimportant for providing inspiring role models for young people').
The final instrument that is used in the current study consisted of 70 items divided over ten MESSI dimensions: social equality and inclusion (6 items); collective identity, connection, and pride (6 items); ethics and fair play (9 items); happiness and experiences (6 items); fans and media (7 items); international image and political power (6 items); athletes' quality of life and competences (7 items); sports participation and inspiration (8 items); economic development and partnerships (7 items); and local consumption and environment (8 items).

Data Analysis
Confirmatory Factor Analysis (CFA) with maximum likelihood (ML) estimation was performed.It was deemed appropriate to conduct CFA rather than exploratory factor analysis since a priori expectations of the factor structure already existed (Funahashi et al., 2015).Prior to data analysis, assumptions were checked, revealing no violations and confirming the dataset's appropriateness.Analysis was completed using the statistical software R. Before performing the CFA, the sample was randomly split into two subsamples.One sample was used as a calibration sample (n = 5,077) to construct the measurement model, while the other sample was used as a validation sample (n = 5,058) to validate the measurement model.Six common indices were used to assess the goodness-of-fit: (1) chi-square value (χ 2 ), (2) Root Mean Square Error of Approximation (RMSEA), (3) Comparative Fit Index (CFI), (4) Tucker Lewis Index (TLI), (5) Standardized Mean Square Residual (SRMR), and (6) the Normed Fit Index (NFI).The acceptable goodness-of-fit threshold for each index was set on χ 2 as non-significant, CFI, TLI, and NFI greater than 0.9, RMSEA less than 0.08, and SRMR less than 0.09 (Hooper et al., 2008;Hu & Bentler, 1999;Kline, 2005;Wheaton et al., 1977).Three tests were employed to assess convergent validity: (1) construct reliability (CR) must exceed the threshold of 0.6 (Bagozzi & Yi, 1988), (2) average variance extracted (AVE) must exceed the threshold of 0.5 (Fornell & Larcker, 1981), and (3) factor loadings must be significant (p < .05)and greater than 0.5 (Fornell & Larcker, 1981).As the software R did not provide CR and AVE values, those values were calculated using procedures outlined by Fornell and Larcker (1981).The reliability of the scale was assessed using Cronbach's α, with a cut-off of 0.7 (Nunnally & Bernstein, 1994).Discriminant validity was investigated using two methods as outlined by previous research (Algesheimer et al., 2005;Yamaguchi et al., 2022).First, correlations between latent constructs must be significantly less than one.Second, significant chi-square differences must be found between the correlation-constrained model and the unconstrained baseline model.

Validation of the Measurement Model
Results of the goodness-of-fit indices revealed that the 68-item 10-dimension measurement model fit the data of the validation sample well: χ 2 (2165) = 23027.37,p < .001,RMSEA = 0.044 (90% CI [0.043, 0.044]), CFI = 0.93, TLI = 0.93, SRMR = 0.04, NFI = 0.93 (Table I).Apart from χ 2 , the other fit indices reached an acceptable level.Convergent validity was established: (1) CR values of all constructs exceeded the threshold of 0.6 (Bagozzi & Yi, 1988), (2) AVE values of all constructs exceeded the threshold of 0.5 (Fornell & Larcker, 1981), and (3) as shown in Table II, factor loadings were significant (p < .05)and ranged from 0.51 to 0.89 (Fornell & Larcker, 1981).The Cronbach's α indices exceeded the recommended threshold of 0.7 for all latent constructs, indicating reliability (Nunnally & Bernstein, 1994).The evidence for discriminant validity was provided in two stages.First, correlations (± two standard deviations) amongst the latent constructs were significantly less than one, with the exception of the correlation between the constructs of happiness and experiences, and fans and media (Bagozzi & Yi, 1988).As previous studies indicated that the dimensions of elite sport outcomes are highly interrelated (e.g., De Rycke & De Bosscher, 2019), the high correlations observed between constructs were considered acceptable.Second, chi-square difference tests between the correlation-constrained model and the unconstrained baseline model were conducted for each pair of latent constructs (a total of 45 tests) and resulted in significant differences for each test (Algesheimer et al., 2005).In conclusion, the measurement model of the 68-item MESSI scale was accepted.Results of the CFA can be found in Table II.

Discussion and Conclusion
Given the scarce and fragmented evidence base regarding public perceptions of the societal outcomes of elite sport (De Rycke & De Bosscher, 2021;Funahashi et al., 2015), this study aimed to provide a validated and reliable instrument to assess public perceptions of the societal outcomes of elite sport.Accordingly, this paper advances the limited knowledge and extends the existing evidence base regarding the potential positive and negative societal outcomes of elite sport.Previous studies (De Rycke & De Bosscher, 2021;De Rycke et al., 2019) lacked validation, were constrained to a specific context (i.e., Flanders), and investigated partial societal outcomes, omitting crucial details concerning specific societal outcomes (e.g., sports participation).
This 68-item MESSI scale represents the ten dimensions of the MESSI framework proposed by De Rycke and De Bosscher (2019), emphasizing that a holistic (i.e., all potential outcomes) perspective on elite sport's outcomes on society is appropriate.Moreover, this scale adopted a multidimensional approach wherein both positive and negative tendencies related to the same construct were measured simultaneously (Lee et al., 2013).Such a multidimensional approach is essential and highly valuable, as recent research indicates that elite sport does not exclusively yield positive outcomes (Balk & Veldman, 2023).Furthermore, this study provided a representation of Northern (e.g., Finland), Southern (e.g., Portugal), Eastern (e.g., Poland), and Western (e.g., Belgium) Europe, making the instrument suitable for cross-country comparison and generalisability.
From a practical perspective, the study provides a tool for policymakers to effectively assess public perceptions of elite sport's societal outcomes.This is essential, as there is still a lack of understanding regarding the societal values and outcomes of elite sport and their potential benefits for the population (Maennig & Vierhaus, 2016).Accordingly, policymakers can proactively address the values of the population to enhance positive and reduce negative societal outcomes while simultaneously justifying their elite sport investments to taxpayers (De Bosscher et al., 2021) and evaluate the public support of enacted policies (De Rycke et al., 2019;Ohmann et al., 2006).
This study also has limitations.First and foremost, perceptions can be subjective, biased, and potentially inaccurate (Sant & Mason, 2015;Taks et al., 2020).Hence, caution is advised when interpreting perceptions.Second, the study employed a cross-sectional research design.However, research showed that perceptions can be influenced by momentums in sport, such as the Olympic Games or scandals (e.g., Calciopoli Scandal in Italian football; Buraimo et al., 2016).Elling et al. (2014) demonstrated that elite sport only triggers short-term changes, such as increased pride, while Helsen et al. (2022) argued that perceptions can change over time.Therefore, future research can adopt longitudinal research designs to examine the influence of time effects on perceptions.Third, the present study did not consider demographic and socio-psychological variables that might potentially influence individuals' perceptions.Prior research has shown that interest in elite sport (Hallmann et al., 2013), consumption of elite sport (De Rycke et al., 2019), gender (Hallmann et al., 2013), migration background (De Rycke & De Bosscher, 2021), trust in policymakers (Funahashi et al., 2015), and athletes as role models (Hallmann et al., 2020) might influence perceptions.Balk and Veldman (2023) emphasized clarifying the perspective from which these outcomes are evaluated.The perception of whether a societal outcome is positive or negative depends on the target audience, as well as the societal and psychological contexts and trends in which individuals are embedded.To illustrate, one group might perceive a particular aspect of elite sport positively (e.g., watching a sports game brings happiness, involves alcohol consumption, and loud cheering), while another group might see it negatively (e.g., considering it a disturbance).Therefore, future studies might focus on the influence of social trends and socio-psychological factors on public perception.Finally, this study was limited to a European context, affecting the generalisability of the findings to a global context.Future research should aim to validate the scale for broader, universal (e.g., Asia, North America) application.
In sum, this study aims to validate the MESSI scale in a European context.Study findings support the content, convergent, discriminant validity, and reliability of a ten-dimension 68-item MESSI scale.Study findings established a foundation for generalisability and comparison across contexts while providing a practical tool to map and evaluate the potential societal outcomes of elite sport among the population.
(positive 1 1 Belgium consists of two parts: (i) Flanders which is the Dutch-speaking northern part, and (ii) Wallonia which is the French-speaking southern part.Public Perceptions on Elite Sport's Societal Outcomes Praet et al.

TABLE I :
Goodness-of-Fit Indices of the CFA Note.RMSEA = Root Mean Square Error of Approximation; CFI = Comparative Fit Index; TLI = Tucker Lewis Index; SRMR = Standardised Mean Square Residual; NFI = Normed-Fit Index.

TABLE II :
Factor Loadings, Construct Reliability, Average Variance Extracted, Cronbach's Alpha, Means, and Standard Deviations for the Validation Sample (n = 5,058)

TABLE II :
Continued

TABLE II :
Continued

TABLE II :
Continued