Hostname: page-component-848d4c4894-x5gtn Total loading time: 0 Render date: 2024-06-11T12:25:21.766Z Has data issue: false hasContentIssue false
  • English
  • Français

Word Embeddings for the Analysis of Ideological Placement in Parliamentary Corpora

Published online by Cambridge University Press:  03 July 2019

Ludovic Rheault Open the ORCID record for Ludovic Rheault [Opens in a new window]  and
Christopher Cochrane
Ludovic Rheault*
Affiliation:
Assistant Professor, Department of Political Science and Munk School of Global Affairs and Public Policy, University of Toronto, Canada. Email: ludovic.rheault@utoronto.ca
Christopher Cochrane
Affiliation:
Associate Professor, Department of Political Science, University of Toronto, Canada. Email: christopher.cochrane@utoronto.ca
*
*Email: ludovic.rheault@utoronto.ca
Get access Rights & Permissions [Opens in a new window]

Abstract

Word embeddings, the coefficients from neural network models predicting the use of words in context, have now become inescapable in applications involving natural language processing. Despite a few studies in political science, the potential of this methodology for the analysis of political texts has yet to be fully uncovered. This paper introduces models of word embeddings augmented with political metadata and trained on large-scale parliamentary corpora from Britain, Canada, and the United States. We fit these models with indicator variables of the party affiliation of members of parliament, which we refer to as party embeddings. We illustrate how these embeddings can be used to produce scaling estimates of ideological placement and other quantities of interest for political research. To validate the methodology, we assess our results against indicators from the Comparative Manifestos Project, surveys of experts, and measures based on roll-call votes. Our findings suggest that party embeddings are successful at capturing latent concepts such as ideology, and the approach provides researchers with an integrated framework for studying political language.

Keywords

word embeddingsparliamentary corporatext as datapolitical ideologynatural language processing
Type
Articles
Information
Political Analysis , Volume 28 , Issue 1 , January 2020 , pp. 112 - 133
Copyright
Copyright © The Author(s) 2019. Published by Cambridge University Press on behalf of the Society for Political Methodology. 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Authors’ note: We thank participants in the annual meeting of the Society for Political Methodology, the Canadian Political Science Association annual conference, the Advanced Computational Linguistics seminar at the University of Toronto, as well as anonymous reviewers for their helpful comments. Replication data is available through the Political Analysis Dataverse (Rheault and Cochrane 2019).

Contributing Editor: Jeff Gill

References

Bäck, H., and Debus, M.. 2016. Political Parties, Parliaments and Legislative Speechmaking . New York: Palgrave Macmillan.Google Scholar
Beelen, K., Thijm, T. A., Cochrane, C., Halvemaan, K., Hirst, G., Kimmins, M., Lijbrink, S., Marx, M., Naderi, N., Polyanovsky, R., Rheault, L., and Whyte, T.. 2017. “Digitization of the Canadian Parliamentary Debates.” Canadian Journal of Political Science 50(3):849–864.Google Scholar
Benoit, K., and Laver, M.. 2006. Party Policy in Modern Democracies . New York: Routledge.Google Scholar
Bird, K. 2010. “Patterns of Substantive Representation Among Visible Minority MPs: Evidence from Canada’s House of Commons.” In The Political Representation of Immigrants and Minorities , edited by Bird, K., Saalfeld, T., and Wüst, A. M.. New York: Routledge.Google Scholar
Bishop, C. M. 2006. Pattern Recognition and Machine Learning . New York: Springer.Google Scholar
Budge, I., Klingemann, H.-D., Volkens, A., Bara, J., and Tanenbaum, E.. 2001. Mapping Policy Preferences: Estimates for Parties, Electors, and Governments (1945–1998) . Oxford: Oxford University Press.Google Scholar
Budge, I., and Laver, M. J., eds. 1992. Party Policy and Government Coalitions . London: Palgrave Macmillan UK.Google Scholar
Caliskan, A., Bryson, J. J., and Narayanan, A.. 2017. “Semantics Derived Automatically from Language Corpora Contain Human-Like Biases.” Science 356(6334):183–186.Google Scholar
Castles, F. G., and Mair, P.. 1984. “Left–Right Political Scales: Some ‘Expert’ Judgments.” European Journal of Political Research 12(1):73–88.Google Scholar
Clarke, H. D., Sanders, D., Stewart, M. C., and Whiteley, P.. 2004. Political Choice in Britain . Oxford: Oxford University Press.Google Scholar
Clinton, J. D. 2012. “Using Roll Call Estimates to Test Models of Politics.” Annual Review of Political Science 15:79–99.Google Scholar
Cochrane, C. 2010. “Left/Right Ideology and Canadian Politics.” Canadian Journal of Political Science 45(3):583–605.Google Scholar
Cochrane, C. 2015. Left and Right: The Small World of Political Ideas . Montreal, Kingston: McGill-Queen’s University Press.Google Scholar
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., and Harshman, R.. 1990. “Indexing by Latent Semantic Analysis.” Journal of the American Society for Information Science 41(6):391–407.Google Scholar
Denny, M. J., and Spirling, A.. 2018. “Text Preprocessing For Unsupervised Learning: Why It Matters, When It Misleads, And What To Do About It.” Political Analysis 26(2):168–189.Google Scholar
Diermeier, D., Godbout, J.-F., Yu, B., and Kaufmann, S.. 2012. “Language and Ideology in Congress.” British Journal of Political Science 42(1):31–55.Google Scholar
Freeden, M. 1998. Ideology and Political Theory: A Conceptual Approach . Oxford: Oxford University Press.Google Scholar
Gabel, M. J., and Huber, J. D.. 2000. “Putting Parties in Their Place: Inferring Party Left–Right Ideological Positions from Party Manifestos Data.” American Journal of Political Science 44(1):94–103.Google Scholar
Garg, N., Schiebinger, L., Jurafsky, D., and Zou, J.. 2018. “Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes.” Proceedings of the National Academy of Sciences 115(16):E3635–E3644.Google Scholar
Gentzkow, M., Kelly, B. T., and Taddy, M.. 2017. “Text as Data.” NBER Working Paper w23276.Google Scholar
Gentzkow, M., and Shapiro, J. M.. 2010. “What Drives Media Slant? Evidence from U.S. Daily Newspapers.” Econometrica 78(1):35–71.Google Scholar
Gentzkow, M., Shapiro, J. M., and Taddy, M.. 2016. “Measuring Polarization in High-Dimensional Data: Method and Application to Congressional Speech.” NBER Working Paper: 22423.Google Scholar
Glavaš, G., Nanni, F., and Ponzetto, S. P.. 2017. “Cross-Lingual Classification of Topics in Political Texts.” In Proceedings of the 2017 ACL Workshop on Natural Language Processing and Computational Social Science , 42–46. Association for Computational Linguistics.Google Scholar
Godbout, J.-F., and Høyland, B.. 2013. “The Emergence of Parties in the Canadian House of Commons (1867–1908).” Canadian Journal of Political Science 46(4):773–797.Google Scholar
Grimmer, J., and Stewart, B. M.. 2013. “Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts.” Political Analysis 21(3):267–297.Google Scholar
Hastie, T., Tibshirani, R., and Friedman, J.. 2009. The Elements of Statistical Learning . Berlin: Springer.Google Scholar
Hirst, G., Riabinin, Y., Graham, J., Boizot-Roche, M., and Morris, C.. 2014. “Text to Ideology or Text to Party Status? In From Text to Political Positions: Text Analysis across Disciplines , edited by Kaal, B., Maks, I., and van Elfrinkhof, A., 93–116. Amsterdam: John Benjamins Publishing Company.Google Scholar
Hix, S., and Noury, A.. 2016. “Government–Opposition or Left–Right? The Institutional Determinants of Voting in Legislatures.” Political Science Research and Methods 4(2):249–273.Google Scholar
Huber, J., and Inglehart, R.. 1995. “Expert Interpretations of Party Space and Party Locations in 42 Societies.” Party Politics 1(1):73–111.Google Scholar
Iyyer, M., Enns, P., Boyd-Graber, J., and Resnik, P.. 2014. “Political Ideology Detection Using Recursive Neural Networks.” In Proceedings of the 2014 Annual Meeting of the Association for Computational Linguistics , 1113–1122. Association for Computational Linguistics.Google Scholar
Jensen, J., Kaplan, E., Naidu, S., and Wilse-Samson, L.. 2012. “Political Polarization and the Dynamics of Political Language: Evidence from 130 Years of Partisan Speech.” Brookings Papers on Economic Activity Fall:1–81.Google Scholar
Johnston, R. 2017. The Canadian Party System: An Analytic History . Vancouver: UBC Press.Google Scholar
Kim, I. S., Londregan, J., and Ratkovic, M.. 2018. “Estimating Spatial Preferences from Votes and Text.” Political Analysis 26(2):210–229.Google Scholar
Lai, S., Liu, K., Xu, J., and an Zhao, L.. 2016. “How to Generate Good Word Embedding? IEEE Intelligent Systems 31(6):5–14.Google Scholar
Lauderdale, B. E., and Herzog, A.. 2016. “Measuring Political Positions from Legislative Speech.” Political Analysis 24(3):374–394.Google Scholar
Laver, M., Benoit, K., and Garry, J.. 2003. “Extracting Policy Positions from Political Texts Using Words as Data.” American Political Science Review 97(2):311–331.Google Scholar
Le, Q., and Mikolov, T.. 2014. “Distributed Representations of Sentences and Documents.” In Proceedings of the 31st International Conference on Machine Learning , edited by Xing, E. P. and Jebara, T., II-1188–II-1196. PMLR.Google Scholar
Levy, O., Goldberg, Y., and Dagan, I.. 2015. “Improving Distributional Similarity with Lessons Learned from Word Embeddings.” Transactions of the Association for Computational Linguistics 3:211–225.Google Scholar
Lowe, W., and Benoit, K.. 2013. “Validating Estimates of Latent Traits from Textual Data Using Human Judgment as a Benchmark.” Political Analysis 21(3):298–313.Google Scholar
MacKay, D. J. C. 1992. “A Practical Bayesian Framework for Backpropagation Networks.” Neural Computation 4(3):448–472.Google Scholar
Manning, C. D., Raghavan, P., and Schütze, H.. 2009. An Introduction to Information Retrieval . Cambridge: Cambridge University Press.Google Scholar
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., and Dean, J.. 2013. “Distributed Representations of Words and Phrases and their Compositionality.” In Proceedings of the 26th International Conference on Neural Information Processing Systems , 3111–3119. Neural Information Processing Systems Foundation.Google Scholar
Mikolov, T., Chen, K., Corrado, G., and Dean, J.. 2013. “Efficient Estimation of Word Representations in Vector Space.” In Proceedings of Workshop at ICLR , 1–12. International Conference on Representation Learning.Google Scholar
Mullainathan, S., and Spiess, J.. 2017. “Machine Learning: An Applied Econometric Approach.” Journal of Economic Perspectives 31(2):87–106.Google Scholar
Nay, J. J. 2016. “Gov2Vec: Learning Distributed Representations of Institutions and Their Legal Text.” In Proceedings of the 2016 EMNLP Workshop on Natural Language Processing and Computational Social Science , 49–54. Association for Computational Linguistics.Google Scholar
Nokken, T. P., and Poole, K. T.. 2004. “Congressional Party Defection in American History.” Legislative Studies Quarterly 29(4):545–568.Google Scholar
Pennington, J., Socher, R., and Manning, C. D.. 2014. “Glove: Global Vectors for Word Representation.” In Conference on Empirical Methods in Natural Language Processing (EMNLP) , 1532–1543. Association for Computational Linguistics.Google Scholar
Poole, K. T., and Rosenthal, H. L.. 2007. Ideology and Congress . New York: Transaction Publishers.Google Scholar
Powell, G. B. 2004. “Political Representation in Comparative Politics.” Annual Review of Political Science 7(1):273–296.Google Scholar
Preoţiuc-Pietro, D., Liu, Y., Hopkins, D., and Ungar, L.. 2017. “Beyond Binary Labels: Political Ideology Prediction of Twitter Users.” In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics , 729–740. Association for Computational Linguistics.Google Scholar
Proksch, S.-O., and Slapin, J. B.. 2010. “Position Taking in European Parliament Speeches.” British Journal of Political Science 40(3):587–611.Google Scholar
Proksch, S.-O., and Slapin, J. B.. 2015. The Politics of Parliamentary Debate . Cambridge: Cambridge University Press.Google Scholar
Proksch, S.-O., Lowe, W., Wäckerle, J., and Soroka, S.. 2018. “Multilingual Sentiment Analysis: A New Approach to Measuring Conflict in Legislative Speeches.” Legislative Studies Quarterly 0(0):1–35.Google Scholar
Řehůřek, R., and Sojka, P.. 2010. “Software Framework for Topic Modelling with Large Corpora.” In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks , 45–50. European Language Resources Association.Google Scholar
Rheault, L., and Cochrane, C.. 2019. “Word Embeddings for the Analysis of Ideological Placement in Parliamentary Corpora.” https://doi.org/10.7910/DVN/K0OYQF, Harvard Dataverse.Google Scholar
Rheault, L., Beelen, K., Cochrane, C., and Hirst, G.. 2016. “Measuring Emotion in Parliamentary Debates with Automated Textual Analysis.” PLoS ONE 11(12): e0168843.Google Scholar
Schwarz, D., Traber, D., and Benoit, K.. 2017. “Estimating Intra-Party Preferences: Comparing Speeches to Votes.” Political Science Research and Methods 5(2):379–396.Google Scholar
Shafer, B. E., and Johnston, R.. 2009. The End of Southern Exceptionalism: Class, Race, and Partisan Change in the Postwar South . Cambridge: Harvard University Press.Google Scholar
Sim, Y., Acree, B. D. L., Gross, J. H., and Smith, N. A.. 2013. “Measuring Ideological Proportions in Political Speeches.” In Proceedings of the 2013 Conference on Empirical Methods of Natural Language Processing (EMNLP) , 91–101. Association for Computational Linguistics.Google Scholar
Slapin, J. B., and Proksch, S.-O.. 2008. “A Scaling Model for Estimating Time-Series Party Positions from Texts.” American Journal of Political Science 52(3):705–722.Google Scholar
Spirling, A., and McLean, I.. 2007. “UK OC OK? Interpreting Optimal Classification Scores for the UK House of Commons.” Political Analysis 15(1):85–96.Google Scholar
Sundquist, J. L. 2011. Dynamics of the Party System . Washington, DC: Brookings Institution Press.Google Scholar
Taddy, M. 2013. “Multinomial Inverse Regression for Text Analysis.” Journal of the American Statistical Association 108(203):755–770.Google Scholar
Tran, D., Hoffman, M. D., Saurous, R. A., Brevdo, E., Murphy, K., and Blei, D. M.. 2017. “Deep Probabilistic Programming.” In Proceedings of the 5th International Conference on Learning Representations , 1–18.Google Scholar
Wittgenstein, L. 2009. Philosophical Investigations . West Sussex, UK: Blackwell.Google Scholar
Supplementary material: File

Rheault and Cochrane supplementary material

Online appendix

Download Rheault and Cochrane supplementary material(File)
File 425 KB