The analysis of partisan differences in advanced democracies. The “independent variable problem” reconsidered

The paper challenges the idea that using governing parties’ policy positions is necessarily desirable in quantitative analyses of partisan differences in public policies. Rather, for some research questions parties’ affiliation with certain party families is the preferable solution. The paper shows that the use of policy positions instead of party family affiliation shifts the research question from asking whether parties make a difference in public policy to the question whether parties in government do what they promise. This shift can have considerable analytical costs and can potentially blindfold scholars for certain dynamics. Moreover, the use of party family affiliation has less drawbacks than is often claimed even under multidimensional party competition if the simple distinction between left, center and right parties is abandoned and more fine-grained data for various party families are used.

categorizations do not allow for changes of party positions over time (which are frequent ...), (iii) they do not allow for differentiations between single policy areas (however, a liberal position in economic policy is certainly not equivalent to a rightwing position in immigration policy, and vice versa)". 1 The growing relevance of the "policy-positions-approach" was at least to some extent triggered by the increasing availability of quantitative data for the policy positions of virtually all parties in democratic systems with the release of the Comparative Manifesto Project Dataset (now MARPOR; see Lehmann et al. 2022). Conceptually, the MARPOR data measure issue emphasis, not policy positions as such, however, and scholars have criticized a number of other issues regarding these data, too (Laver and Garry 2000). Researchers who deem this criticism relevant do not need to do without policy positions, however. Data on the policy positions of political parties can be read off various expert surveys (Benoit and Laver 2006;Castles and Mair 1984;Huber and Inglehart 1995;Jolly et al. 2022;Laver and Hunt 1992). Furthermore, various methods to extract parties' policy positions from political texts with the help of computer software have been suggested (e.g., Benoit et al. 2009). Finally, also surveys of the general public can be used to determine the policy positions of parties (Döring and Schwander 2015, p. 182-183). Consequently, there is a wealth of data sources for parties' policy positions.
In this paper, I challenge the idea that resorting to parties' policy positions (weighted by cabinet seat share) is always the best solution for the quantitative analysis of partisan differences and suggest that (cabinet seat shares of) party families are the preferable solution for some research questions. In the first section, I argue that the use of policy positions instead of party families shifts the research question from asking whether parties make a difference in public policy to the question whether parties in government do what they promise in their manifestos. I show that this shift can have considerable analytical costs and can potentially blindfold scholars for certain dynamics. In the second section, I show that the party families approach has less drawbacks than is often claimed even under multidimensional party competition if the simple distinction between left, center and right parties is abandoned and more fine-grained data for various party families are used. The final section concludes.

What do we investigate when employing party policy positions?
The use of data on party positions has many advantages for quantitative research. Most importantly, they vary over time for each party and there is also variance between parties of the same party family in different countries. Therefore, they allow mapping programmatic changes of parties over time, which happen quite extensively (Adams et al. 2009;Budge et al. 2010;Schumacher et al. 2013). While some of these programmatic changes could be considered oscillations around a long-term mean, others are doubtlessly substantial and relevant. For example, who would disagree that Tony Blair's New Labour Party was a completely different party programmatically compared with the 'old' Labour Party of Michael Foot that lost the 1983 UK general election? Although less extreme, many other social democratic parties recalibrated their programmatic positions in the wake of the "third way" debate around the turn of the century (Keman 2011;Merkel et al. 2008).
Moreover, there is no doubt in the literature, that there are significant differences between the programmatic positions of parties that belong to the same party family. For example, the British Labour Party under Tony Blair and the French Parti Socialiste under Lionel Jospin are both considered as social democratic parties, but their policy positions in economic and social policy differed quite sharply. While Blair favored a comparatively liberal economic policy stance, Jospin remained true to the more statist tradition of the PS (Merkel et al. 2008). Therefore, proponents of the use of data on policy positions like Döring and Schwander (2015, p. 178-9) argue that we need to take these changes over time and differences across countries into account, which can only be done by using data on policy positions. Aren't these authors right?
The answer to this question depends upon the research question. If, for example, we are interested in whether parties do in government as they say in their manifestos, data on policy positions (and the MARPOR data in particular) are obligatory. It would not make sense to investigate pledge fulfillment with data on party family affiliation. If, however, the research question is whether it makes a difference policywise which party is in government, the suitability of data on policy positions is less evident.
To make this point clearer, let us assume we find a positive correlation between parties' weighted policy positions and some policy outcome. For example, we might find that the higher governing parties score on MARPOR's item "welfare state expansion", the more generous unemployment benefits are. Would this finding suggest that parties make a difference in terms of policy outputs?
The simple answer is: Not necessarily! For simplicity's sake assume the sample for which we find the above correlation only consists of countries in which conservative and social democratic parties compete and alternate in government. Assume further that in some countries, both parties advocate and adopt generous unemployment benefits, while in others both competitors agree on rather meagre benefits in their manifestos and in the policies they adopt when in government. Hence, we would indeed find a very strong correlation for policy positions and policy outputs because all parties do as they say. In contrast, party family affiliation and policy outputs would be perfectly uncorrelated-in some cases social democrats produce high benefits, while in others they do not, and the same is true for conservatives. It would thus be perfectly correct to conclude from these data that parties adopt the policies they promise, but it would be wrong to argue that parties make a difference in public policies or that voters had a choice in terms of unemployment benefits in any of the countries.
Policies along a "Nixon-goes-to-China" logic can produce similar patterns. Consider welfare state retrenchment as an example for which the "Nixon-goes-to-China" argument has been made (Ross 2000). The argument might go as follows: During the post-war years until the mid-1970s, left parties were strongly emphasizing the welfare state programmatically and adopted generous welfare programs when in government. Conservatives, in contrast, did not put as much emphasis on social policy in their manifestos and did not increase welfare generosity as much as their left competitors. In this situation, the choice of the independent variable would not make a difference: We would find a positive correlation between the cabinet seat shares of either left parties or of parties with pro-welfare positions and welfare generosity.
From the 1980s onwards, however, when economic problems began to loom large, welfare retrenchment got on the agenda. Conservative parties could have shied away from retrenching the welfare state because they feared the negative electoral repercussions of the announcement or adoption of substantial cuts to benefits (Pierson 1994). Instead, left parties that voters considered as parties protecting the welfare state might have had an easier time justifying retrenchment, which they might have deemed necessary, for example to improve the employment situation (Green-Pedersen 2001). One could further assume that in order not to be seen as deceiving voters, these parties not only adopted the retrenchment after being elected but they also announced it in advance. As a consequence, we see left parties announcing and adopting welfare retrenchment while right parties do neither touch welfare retrenchment in their manifestos nor in the policies they adopt. Hence, the use of policy position data would point at high levels of continuity in the relation between parties and welfare generosity (high correlation between policy positions and generosity) while a look at party families would lead us to speak of a dramatically different situation in the retrenchment period compared with the expansion era (left parties retrenching instead of expanding).
Admittedly, these are counterfactuals. But they show that analyses using policy positions may have blind spots and are not necessarily well-suited to analyze whether it makes a difference for public policies which party is in government. To reiterate: This is not to deny that employing data on policy positions can yield useful results. My point is simply that there are some research questions that are better analyzed using data on party families. The question of whether parties make a difference in public policies is a case in point.

Party families and the multidimensionality of electoral competition
Another problem of the party-families-approach according to the proponents of the policy-positions-approach is that the policy space has become two-or even multidimensional. Thus, while it may have made sense to look at party families along a left-right-continuum as long as the economic cleavage dominated the party systems of advanced democracies, there has emerged a consensus among scholars that essentially all current party systems are shaped by (at least) two cleavages (Benoit and Laver 2006). There is less agreement in the literature on how these different cleavages are to be named and which issues they entail, but the bottom line is that we need to include the multi-dimensionality of today's party systems in our analyses. This is easily done with the help of data on party manifestos or policy positions of parties. The MARPOR dataset, for example, covers many different issues and Bakker and Hobolt (2013) have suggested which items could be collapsed to represent the economic left-right and the libertarian-authoritarian dimensions. Similarly, there are several expert surveys which score parties on various policy dimensions (for example Benoit and Laver 2006;Jolly et al. 2022;Laver and Hunt 1992). Thus, using these data, it poses no difficulty to analyze partisan effects on various government outputs beyond the economic dimension. In contrast, it is often assumed to be very difficult to incorporate the multi-dimensionality of modern party systems if one is to use party family affiliations.
But is this really the case? My main argument is that this is only true if we collapse party families into simple left-(center)-right categories. If we use such a classification, we might indeed run into problems when looking at issues like migration, abortion or environmental protection. Here, we are likely to find policy disagreement among parties of the right camp, for example between Christian democrats and liberals on abortion, as well as among the left camp, for example between social democrats and greens regarding environmental protection. Similarly, we may also find strange bedfellows across the camps, for example, (left-wing) greens and (right-wing) liberals favoring permissive abortion rules. Indeed, an analysis of partisan politics in these issue areas based on distinguishing between left or right party families may not make much sense. Nevertheless, that may simply be a problem of too broad categories and not one of using party family affiliation as such.
In fact, if we look at individual party families rather than the aggregate left and right camps, the party labels may very well be helpful also when analyzing issue areas beyond economic and social policy. This has been shown convincingly for environmental policy in a recent study by Jahn (2022). He first deduces hypotheses for no less than eight party families and then tests these empirically. His findings are quite nuanced. While it almost goes without saying that green parties favor environmental protection more than other parties, the same is not necessarily true for social democrats whose effect is small and only marginally significant. Similarly, while non-Christian center parties like the Scandinavian center parties or the US Democrats have a negative effect on environmental policy, the same is not true for Christian democrats.
A similar argument can be made for morality policy, an issue area that many scholars will concede is even more different from (re)distributive policies than environmental policy. Nonetheless, Engler and Dümig (2017) as well as Budde et al. (2018) have convincingly theorized how different party families stand regarding various issues of morality policies. Conservative parties, for example, are expected to defend traditional values and thus favor restrictive morality policies. The same prediction is made for Christian democrats who are likely to advocate the protection of the sanctity of life. Hence, they should oppose abortion or preimplantation genetic diagnosis (PGD), for example. Their opposition (and that of conservative parties) should be less stiff, however, when business interests are at stake like with regard to stem-cell research. While leftist (i.e. social democratic, post-communist and green) but also liberal parties should generally favor post-materialistic values and the right of self-determination, not all of these parties do so to the same extent. Green parties are a particularly interesting case in point. While they are certainly in favor of a liberal abortion regime, they might even side with Christian democrats K on the regulation of stem-cell research and maybe also PGD because they could fear severe interventions into human nature. It is not necessary to go through every morality policy issue for every party family for the purpose of this paper. Rather, the bottom line is that plausible and nuanced expectations can be derived about which party family might favor which policy option in issue areas far remote from the classic fields of economic and social policy.
But also classic welfare state issues may necessitate moving beyond a simplistic left-right-dichotomy. It is well-known that Christian democrats in particular have developed a distinctive welfare state as shown by qualitative as well as quantitative studies (Huber and Stephens 2001;Kalyvas and van Kersbergen 2010;van Kersbergen 1995;Zohlnhöfer and Voigt 2021). But differences may be still more finegrained than that. For example, Schmidt (2021, p. 307-9) discusses the distinct social policy positions of no less than eight party families.
The empirical literature on welfare policies has only very rarely taken advantage of such a nuanced view on party families, partly due to a lack of data that are finegrained enough (but see below). The few papers, however, that look at party families in a more disaggregated way suggest that this could be a promising way forward. For example, Wolf et al. (2014), in an analysis of changes in welfare state generosity in advanced democracies, consistently find substantial and significant positive coefficients of liberal parties like the Danish Venstre or the Swiss or German FDP for unemployment benefit and sick pay generosity. In contrast, the authors report a substantially and statistically significant negative coefficient for Conservative parties in government on these same benefits. Thus, the coefficients of Liberals and Conservatives point in opposite directions, although they are usually lumped together in the available datasets (for example in Armingeon et al. 2022 andBrady et al. 2020 for most liberal parties). Moreover, Wolf et al. (2014) also report a negative (albeit small) effect of non-religious center parties like the Scandinavian center parties on pension generosity in the 1990s-again a party family that is usually ignored in quantitative studies. Consequently, these findings suggest that there is something to be gained from analyzing party family affiliation in a more fine-grained way even for the classic field of welfare state research. 2

Conclusion
This paper has tried to make two important points, which are relevant for the study of the relationship between the partisan composition of governments and public policies. First, parties' policy positions are not the best way to measure the partisan composition of governments for all research questions. If we wish to investigate whether it makes a difference in terms of public policies which party is in government, parties' policy positions may blur as much as they clarify. Indeed, we might find a positive correlation between parties' policy positions and policy outputs but voters nonetheless do not have a meaningful choice between different policy options at a given election. Hence, for some questions, the party-families-approach is still more suitable.
Moreover, the party-families-approach is of greater analytical as well as democratic relevance than the policy-positions-approach. Analytically, the causal path covered by a correlation between government participation of party families and policy outputs is much longer than the causal path covered when using policy positions of governing parties as independent variable. If party family affiliation is expected to matter for policy outputs, we assume that party families have a specific ideology or basic programmatic position that is shared by the members of the party family (step 1). This general ideology informs the individual party programs and manifestos, which can be understood as adaptations of the basic programmatic position of the party family to specific circumstances under which the party has to act like the political system, the configuration of party competition, the specific problems and available resources etc. (step 2). 3 Finally, the party program and specific policy positions of a party inform the policies the party adopts when in government (step 3). Thus, policy positions are causally much more proximate to policy outputs (only covering step 3) than a parties' membership in a specific party family which is based on party ideology (covering steps 1-3). Hence, in a sense we can explain 'less' if we use parties' policy positions compared to the affiliation with a party family.
In addition, the use of the two independent variables has also immense implications for what voters need to know in order to get a sense of what the next government is going to do. To put it bluntly: If policy positions are positively correlated with policy outputs voters need to read the governing parties' manifestos if they want to predict what the next government is likely to do. If, however, we can say something about policy outputs when looking at party family affiliation it will suffice for voters to look at which party families are in government-probably a much easier way to get an idea of what the government is going to do! That is not to say, of course, that data on policy positions are useless or systematically flawed. To the contrary, these data can certainly be employed to help answer many important research questions. But for the specific purpose of the do-partiesmatter-question, party family affiliation should still be considered suitable.
Second, this paper has argued that party family affiliation can be employed for the analysis of issue areas outside the realm of redistributive policies, too. The main point here, however, is to be very careful with aggregating party families into a simple left-right-dichotomy. If we take a more fine-grained view and look at individual party families, we are likely to indeed find interesting and nuanced results about how political parties shape public policies.
This last point implies that scholars should be very careful when choosing which data they use. Many commonly used datasets only distinguish between left, center and right parties-a distinction which is not fine-grained enough in many instances. But other sources provide more nuanced data. Brady et al. (2020), for example, start out by distinguishing between left, center and right parties, but they also take into account Christian democratic parties in a specific way, namely by dividing center and right parties in secular, Christian and Catholic parties respectively. Swank's (2018) dataset is even more nuanced, distinguishing between left, right, Christian democratic, center, right-populist and left-libertarian parties. Finally, the recently published PACOGOV dataset (Schmidt et al. 2021) provides information on the cabinet seat shares of no less than 12 different party families (including among others greens, liberals, regional parties and independents in addition to the ones mentioned above). As the studies by Wolf et al. (2014) or Jahn (2022) show, employing these nuanced data can help further increase our knowledge about whether, how and under what circumstances political parties make a difference in public policy.