Skip to main content
Log in

Can we Rate Public Support for Democracy in a Comparable Way? Cross-National Equivalence of Democratic Attitudes in the World Value Survey

  • Published:
Social Indicators Research Aims and scope Submit manuscript

Abstract

In this study we examine the cross-cultural equivalence of two scales that measure attitudes toward democracy across 36 countries in the World Value Survey (WVS) 2000. We examine the equivalence of these scales in order to explore if we can meaningfully compare democratic attitudes across countries. Multiple group confirmatory factor analyses (MGCFA) is applied to answer this question. The analyses indicate that the scales may be compared but only to a certain extent and not across all the countries. We close this article by discussing the implications of the findings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. The two scale items include rating of the importance of having a democratic system and whether it is better than any other form of government (WVS 1995–1997).

  2. The scale was based on a respondent’s quality assessment (good vs. bad) of the political system as it was in [reference to previous regime], as it is today, and as it is expected to be tomorrow? (WVS 1995–1997).

  3. For further details, check the website of the World Values Survey Association. http://www.worldvaluessurvey.org/.

  4. Several studies have used multiple group CFA (MGCFA) to assess measurement invariance of scales in such cross-national surveys as the European Social Survey (ESS) and the International Social Survey Program (ISSP) (e.g., for human values, Davidov 2008; Davidov et al. 2008; for national identity, Davidov 2009; for trust, Reeskens and Hooghe 2008).

  5. In the current analyses, we firstly ran the models using pairwise deletion to deal with the problem of missing values (Schafer and Graham 2002). As a final test, we have also analyzed the models using the Full Information Maximum Likelihood (FIML) procedure which is preferred when portions larger than 5% of the data are missing (Schafer and Graham 2002).

  6. For cut off criteria for global fit measures, see Hu and Bentler (1999) and Marsh et al. (2004).

  7. Brown (2006) suggests that standardized factor loadings lower than 0.3–0.4 should indicate that the question is inadequate to measure the latent construct.

  8. Although scholars have argued that MGCFA may be an inappropriate tool for testing for invariance of Likert scales (see, e.g., Lubke and Muthén 2004), studies have demonstrated that it works well even when data are not continuous or normally distributed but ordinal (De Beuckelaer 2005; Welkenhuysen-Gybels and Billiet 2002; Welkenhuysen-Gybels 2004).

  9. Model outputs, factor loadings and global fit measures may be available from the first author upon request.

References

  • Adcock, R., & Collier, D. (2001). Measurement validity: A shared standard for qualitative and quantitative research. American Political Science Review, 95(3), 529–546.

    Article  Google Scholar 

  • Arbuckle, J. L. (1995–2007). Amos 16.0 user’s guide. Chicago, IL: SPSS.

    Google Scholar 

  • Billiet, J. (2003). Cross-cultural equivalence with structural equation modeling. In J. A. Harkness, F. J. R. Van de Vijver, & P. P. Mohler (Eds.), Cross-cultural survey methods (pp. 247–264). New York: John Wiley.

    Google Scholar 

  • Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.

    Google Scholar 

  • Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: Guilford Press.

    Google Scholar 

  • Byrne, B. M. (2001). Structural equation modeling with AMOS: Basic concepts, applications, and programming. London: Lawrence Erlbaum Associates.

    Google Scholar 

  • Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethods matrix. Psychological Bulletin, 56, 81–105.

    Article  Google Scholar 

  • Canache, D., Mondak, J. J., & Seligson, M. A. (2001). Meaning and measurement in cross-national research on satisfaction with democracy. Public Opinion Quarterly, 65(4), 506–528.

    Article  Google Scholar 

  • Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504.

    Article  Google Scholar 

  • Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255.

    Article  Google Scholar 

  • Curtice, J. (2007). Comparative opinion surveys. In J. R. Dalton & H. Klingemann (Eds.), Oxford handbook of political behavior (pp. 898–909). Oxford: Oxford University Press.

    Google Scholar 

  • Dalton, R. J. (1999). Political support in advanced industrial democracies. In P. Norris (Ed.), Critical citizens: Global support for democratic governance (pp. 57–77). Oxford: Oxford University Press.

  • Dalton, J. R., & Ong, N. T. (2005). Authority orientations and democratic attitudes: A test of the ‘Asian values’ hypothesis. Japanese Journal of Political Science, 6(2), 211–231.

    Article  Google Scholar 

  • Dalton, R., Shin, D. C., & Jou, W. (2007). Understanding democracy: Data from unlikely places. Journal of Democracy, 18(4), 142–156.

    Google Scholar 

  • Davidov, E. (2008). A cross-country and cross-time comparison of the human values measurements with the second round of the European Social Survey. Survey Research Methods, 2(1), 33–46.

    Google Scholar 

  • Davidov, E. (2009). Measurement equivalence of nationalism and constructive patriotism in the ISSP: 34 countries in a comparative perspective. Political Analysis, 17(1), 64–82.

    Article  Google Scholar 

  • Davidov, E., Schmidt, P., & Schwartz, S. H. (2008). Bringing values back in: The adequacy of the European Social Survey to measure values in 20 countries. Public Opinion Quarterly, 72(3), 420–445.

    Article  Google Scholar 

  • Davidov, E., Schmidt, P., & Billiet, J. (Eds.). (2010). Cross-cultural analysis: Methods and applications. New York: Taylor and Francis.

  • De Beuckelaer, A. (2005). Measurement invariance issues in international management research. Unpublished dissertation, Limburgs University Centrum, Centrum, Limburg, the Netherlands.

  • Diamond, L. J. (1999). Developing democracy: Toward consolidation. Baltimore: Johns Hopkins University Press.

    Google Scholar 

  • Dixon, J. C. (2008). A clash of civilizations? examining liberal-democratic values in Turkey and the European Union. British Journal of Sociology, 59(4), 681–708.

    Article  Google Scholar 

  • Esmer, Y. (2002). Is there an Islamic civilization? Comparative Sociology, 1(3–4), 265–298.

    Article  Google Scholar 

  • Guerin, D., Pétry, F., & Crete, J. (2004). Tolerance, protest and democratic transition: Survey evidence from 13 Post-Communist countries. European Journal of Political Research, 43(3), 371–395.

    Article  Google Scholar 

  • Haerpfer, C. W. (2008). Support for democracy and autocracy in Russia and the Commonwealth of independent states, 1992–2002. International Political Science Review, 29(4), 411–431.

    Article  Google Scholar 

  • Harkness, J. A., Van de Vijver, F. J. R., & Mohler, P. P. (2003). Cross-cultural survey methods Hoboken. New Jersey: Wiley-Interscience.

    Google Scholar 

  • Heath, A., & Martin, J. (1997). Why are there so few formal measuring instruments in social and political research. In L. E. Fyberg, et al. (Eds.), Survey measurement and process quality (pp. 71–86). New York: Wiley.

    Google Scholar 

  • Heath, A., Fisher, S., & Smith, S. (2005). The globalization of public opinion research. Annual Review of Political Science, 8, 297–333.

    Article  Google Scholar 

  • Hofmann, S. R. (2004). Islam and democracy: Micro-level indications of compatibility. Comparative Political Studies, 37(6), 652–676.

    Article  Google Scholar 

  • Horn, J. L., & McArdle, J. J. (1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18(3), 117–144.

    Article  Google Scholar 

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(4), 1–55.

    Article  Google Scholar 

  • Inglehart, R. (2003). How solid is mass support for democracy—and how can we measure it? Political Science and Politics, 36(1), 51–57.

    Google Scholar 

  • Inglehart, R., & Norris, P. (2003). The true clash of civilizations. Foreign Policy, 135, 63–70.

    Google Scholar 

  • Inglehart, R., & Welzel, C. (2005). Modernization, cultural change, and democracy. Cambridge, UK: Cambridge University Press.

    Google Scholar 

  • Johnson, T. P. (1998). Approaches to equivalence in cross-cultural and cross-national survey research. ZUMA Nachrichten Spezial, 31–40.

  • Jowell, R., Roberts, C., Fitzgerald, R., & Gillian, E. (Eds.). (2007). Measuring attitudes cross-nationally: Lessons from the European social survey. London: Sage Publications.

    Google Scholar 

  • King, G., Murray, C. J. L., Salomon, J. A., & Tandon, A. (2004). Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review, 97(4), 567–583.

    Google Scholar 

  • Kittilson, M. C. (2006). Research resources in comparative political behavior. In J. R. Dalton & H. Klingemann (Eds.), Oxford handbook of political behavior (pp. 865–895). Oxford: Oxford University Press.

    Google Scholar 

  • Klingemann, H.-D (1999). Mapping political support in the 1990s: A global analysis. In P. Norris (Ed.), Critical citizens: Global support for democratic Governance (pp. 151–189). Oxford: Oxford University Press.

    Google Scholar 

  • Knoppen, D., & Saris, W. E. (2009). Do we have to combine values in the Schwartz’ human values scale? A comment on the Davidov studies. Survey Research Methods, 3(2), 91–103.

    Google Scholar 

  • Lagos, M. (2003). Support for and satisfaction with democracy. International Journal of Public Opinion Research, 15(4), 471–487.

    Article  Google Scholar 

  • Linde, J., & Ekman, J. (2003). Satisfaction with democracy: A note on a frequently used indicator in comparative politics. European Journal of Political Research, 42(3), 391–408.

    Article  Google Scholar 

  • Lubke, G. H., & Muthén, B. O. (2004). Applying multigroup confirmatory factor models for continuous outcomes to Likert scale data complicates meaningful group comparisons. Structural Equation Modeling, 11(4), 514–534.

    Article  Google Scholar 

  • Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11(3), 320–341.

    Article  Google Scholar 

  • Mishler, W., & Rose, R. (2001). Political support for incomplete democracies: Realist vs. idealist theories and measures. International Political Science Review, 22(4), 303–320.

    Article  Google Scholar 

  • Norris, P. (Ed.). (1999). Critical citizens: Global support for democratic government. Oxford: Oxford University Press.

    Google Scholar 

  • Norris, P. (2009). The globalization of comparative public opinion research. In N. Robinson & T. Landman (Eds.), Handbook of comparative politics (pp. 522–539). London: Sage.

    Google Scholar 

  • Pettersson, T. (2008). Different patterns of compartmentalisation among Muslims and Christians? In J. Haynes (Ed.), Routldge handbook of religion & politics (pp. 246–270). New York: Routledge.

    Google Scholar 

  • Reeskens, T., & Hooghe, M. (2008). Cross-cultural measurement equivalence of generalized trust. Evidence from the European Social Survey (2002 and 2004). Social Indicators Research, 85(3), 515–532.

    Article  Google Scholar 

  • Rizzo, H., Abdel-Latif, A., & Meyer, K. (2007). The relationship between gender equality and democracy: A comparison of Arab versus non-Arab Muslim societies. Sociology, 41(6), 1151–1170.

    Article  Google Scholar 

  • Rowley, C. K., & Smith, N. (2009). Islam’s democracy paradox: Muslims claim to like democracy, so why do they have so little? Public Choice, 139(3), 273–299.

    Article  Google Scholar 

  • Saris, W. E., & Gallhofer, I. N. (2007). Design, evaluation, and analysis of questionnaires for survey research. New Jersey: Wiley.

    Book  Google Scholar 

  • Saris, W. E., Satorra, A., & Van der Veld, W. M. (2009). Testing structural equation models or detection of misspecifications? Structural Equation Modeling, 16(4), 561–582.

    Article  Google Scholar 

  • Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177.

    Article  Google Scholar 

  • Schedler, A., & Sarsfield, R. (2007). Democrats with adjectives: Linking direct and indirect measures of democratic support. European Journal of Political Research, 46(5), 637–659.

    Article  Google Scholar 

  • Shin, D. C. (2007). Democratization: Perspectives from global citizenries. In J. R. Dalton & H. Klingemann (Eds.), The oxford handbook of political behavior (pp. 259–282). Oxford: Oxford University Press.

    Chapter  Google Scholar 

  • Smith, T. W. (2003). Developing comparable questions in cross-national surveys. In J. A. Harkness, Van de Vijver, J. R. Fons, & P. H. Mohler (Eds.), Cross-cultural survey methods (pp. 69–91). New York: John Wiley & Sons.

    Google Scholar 

  • Steenbergen, M. R. (2000). Item similarity in scale analysis. Political Analysis, 8(3), 261–283.

    Google Scholar 

  • Steenkamp, J. E. M., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25(1), 78–107.

    Article  Google Scholar 

  • Tessler, M. (2002). Islam and democracy in the Middle East: The impact of religious orientations on attitudes toward democracy in four Arab countries. Comparative Politics, 34(3), 337–354.

    Article  Google Scholar 

  • Tessler, M., Moaddel, M., & Inglehart, R. (2006). What do Iraqis want? Journal of Democracy, 17(1), 38–50.

    Article  Google Scholar 

  • Toros, E. (2010). The relationship between Islam and democracy in Turkey: Employing political culture as an indicator. Social Indicators Research, 95(2), 253–265.

    Article  Google Scholar 

  • Tusicisny, A. (2007). Security communities and their values: Taking masses seriously. International Political Science Review, 28(4), 425–449.

    Article  Google Scholar 

  • Van de Vijver, F. J. R. (2003). Bias and equivalence: Cross-cultural perspective. In J. A. Harkness, Van de Vijver, J. R. Fons, & P. H. Mohler (Eds.), Cross-cultural survey methods (pp. 143–155). New York: John Wiley & Sons.

    Google Scholar 

  • Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70.

    Article  Google Scholar 

  • Wang, Z., Dalton, R. J. D., & Shin, D. C. (2006). Political trust, political performance, and support for democracy. In J. R. Dalton & D. C. Shin (Eds.), Citizens, democracy, and markets around the Pacific rim (pp. 135–154). Oxford: Oxford University Press.

    Google Scholar 

  • Welkenhuysen-Gybels, J. (2004). The performance of some observed and unobserved conditional invariance techniques for the detection of differential item functioning. Quality & Quantity, 38(6), 681–702.

    Article  Google Scholar 

  • Welkenhuysen-Gybels, J., & Billiet, J. (2002). A comparison of techniques for detecting cross-cultural inequivalence at the item level. Quality & Quantity, 36(3), 197–218.

    Article  Google Scholar 

  • Welzel, C. (2007). Are levels of democracy affected by mass attitudes? Testing attainment and sustainment effects on democracy. International Political Science Review, 28(4), 397–424.

    Article  Google Scholar 

  • Wu, A. D., Li, Z., & Zumbo, B. D. (2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with TIMSS data. Practical Assessment Research & Evaluation, 12(3), 1–26.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Gal Ariely.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ariely, G., Davidov, E. Can we Rate Public Support for Democracy in a Comparable Way? Cross-National Equivalence of Democratic Attitudes in the World Value Survey. Soc Indic Res 104, 271–286 (2011). https://doi.org/10.1007/s11205-010-9693-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11205-010-9693-5

Keywords

Navigation