Matching voters to parties: Voting advice applications and models of party choice

Abstract

Online voting advice applications (VAAs) have become very popular and may significantly influence voting behaviour. It is therefore important to ask which model of party choice VAAs follow. We establish that VAAs see party choice largely as proximity-based issue congruence, with some elements of the directional and salience models. We then assess how well VAAs follow the proximity model by comparing policy positions extracted from 13 VAAs in seven European countries with established policy measures from expert surveys and party manifestos. Party positions extracted from VAAs show strong convergent validity with left-right and economic positions, but compare less favourably with immigration and environment measures. The voting advice given to users is also inherently limited: VAAs mostly disregard accountability, salience, competence and non-policy factors; they treat policy positions and not outcomes as paramount; and they can be subject to strategic manipulation by political parties. As recommended by their designers, voters should treat these applications as tools and guides rather than as stringent recommendations.

Introduction

In this article we investigate how online voting advice applications (VAAs) match voters to parties. These websites, which provide voting advice by matching user-entered policy views with parties’ positions, were first developed in Finland and the Netherlands in the mid-1990s, and similar tools have now been created in most European countries as well as in the USA and Canada. VAAs now have millions of users in many countries. For example, nearly 40 per cent of the electorate in the Netherlands and Finland used at least one VAA in the run-up to the parliamentary elections in 2006 and 2007, respectively (Ruusuvirta and Rosema, 2008; J. Mykkänen, personal communication, 2009). High user figures have also been reported in Germany, Belgium and Switzerland (Germany: 6.7 million VAA recommendations or 11 per cent of the electorate in 2009; Bundeszentrale fur politische Bildung, 2009; Belgium: a million users or 13 per cent of the electorate in 2004, Walgrave et al, 2008; Switzerland: over a million users or 21 per cent of the electorate in 2007, Ladner et al, 2008, p. 4). VAAs are clearly not just another passing Internet gimmick.

VAAs may also have important political consequences: considered against a backdrop of declining party loyalty and cleavage-based voting and an increase of issue voting (Barnes, 1997), VAAs could potentially affect the electoral decision-making of many citizens. For example, they have become the most important source of election information for young voters in Finland (Strandberg, 2009, pp. 78–83), while a large proportion of website users say that the VAAs help them to decide how to vote (for Germany, see Marschall, 2005, p. 45; for Switzerland, see Fivaz and Schwarz, 2007, p. 10; for Finland, see Bengtsson and Grönlund, 2005, p. 249). VAAs may also persuade some voters to change their candidate or party choice. In the 2005 German federal elections, 6 per cent of Wahl-O-Mat users surveyed said that they will change their vote choice as a result of VAA use (Marschall, 2005, p. 45), whereas the equivalent number was approximately 10 per cent in the 2006 Dutch elections (Kleinnijenhuis and van Hoof, 2008, p. 8). In the 2007 Finnish parliamentary elections the conversion rate was just 3 per cent, but a surprisingly high 15 per cent of the surveyed VAA users said that they did not have a favourite candidate and voted for the candidate suggested by a VAA (Mykkänen et al, 2007, p. 5).

Although such self-reported assessments of influence are of course of uncertain validity, VAAs are clearly exceedingly popular online tools to help voters gain political information and make decisions. As such, they may have a non-trivial impact on voting behaviour. This means that it is important to understand what kinds of recommendations VAAs give. Taking a VAA test may influence the way voters ‘think’ about politics and their electoral choice. There are thus democratic implications to the design of VAAs: what notion of democracy do they represent, and what kind of voting do they encourage?

In this article, we therefore examine the model of voting that underlies these online applications. Specifically, we consider how VAAs match users’ and parties’ issue preferences. By understanding how VAAs treat voter-party congruence, we also clarify which model of voter representation underlies these applications. In doing so, we concentrate on proximity, directional and salience models of vote choice. We find that VAAs reflect a view of party competition and voting that is largely based on issue-based proximity models. This means that parties are seen as congruent with voters if their ideological distance to each other is low.

Having established the dominant model of voting encoded in VAAs, we investigate whether the actual recommendations they give in fact follow the proximity logic. Specifically, we analyse 13 VAAs from Austria, Belgium, Finland, France, Germany, the Netherlands, and Switzerland and compare party positions extracted from these applications with expert survey and manifesto data. We find that in general terms VAAs reflect party positions well, but that accuracy depends on the number of questions asked. This means that party-voter matches on secondary issues, which are represented by fewer VAA questions, are less exact. Although Finnish researchers have used candidate preferences recorded in VAAs to study the ideology of MPs and voter-representative congruence (Paloheimo et al, 2005; Kestilä-Kekkonen and Wass, 2008; Reunanen and Suhonen, 2009), this article is the first cross-national attempt to systematically study how VAAs match voters to parties.

After briefly describing how VAAs work, we consider the logic of voting and representation that underlies most VAAs, concentrating on proximity, directional and salience models of issue voting. We then extract policy positions from VAAs and compare these to existing measures of expert surveys and party manifestos to establish the accuracy of VAA party positions. The final section concludes by considering possible limitations in the way VAAs match voters to parties.

How VAAs Match Voters to Parties

Although VAAs in each country differ slightly, they all follow a basic model that is based on comparing parties’ and voters’ issue preferences. The central feature of every VAA is a multiple-choice questionnaire, usually containing 20–35 statements on current political issues or more general political questions. Users are asked to provide their opinions on each of these statements and after completing the questionnaire are shown how closely their views match with those of the parties.

Although all VAAs follow the same basic model, they also differ in key ways. This variation is best illustrated by presenting the sample of 13 VAAs from seven countries for which we have collected detailed information and party placements (Table 1). These VAAs were online between 2002 and 2009 and represent all the national-level VAAs from which we were able to encode party placements during the data collection period in March–April 2010. Although we focus on national-level VAAs in order to ensure comparability with expert survey and manifesto data, there is also an increasing number of VAAs at the European level (for example, the EU-Profiler, Trechsel and Mair, 2011) and at the sub-national level (for example, the Wahl-O-Mat and Wahlkabine in German and Austrian Länder, respectively). The popularity of VAAs also means that more such applications are constantly being added. Since most new VAAs follow existing designs, we believe that this list is representative of these applications in general.

Table 1 VAAs used in positional comparison with expert surveys and party manifestos

As Table 1 shows, VAAs differ in more ways than just the number of parties placed and the number of questions asked. First, the applications are set up by different organisations. VAA providers are usually either political education agencies, as in Germany and the Netherlands, or media corporations, as in Finland and Belgium. Other VAAs have been developed by teams of political scientists as in Switzerland and in the case of the EU Profiler (Trechsel and Mair, 2011), by civil society organisations and by interest groups.

Second, VAAs differ in how the statements used to establish the voter-party match are chosen and formulated. As with any survey, the choice of question items and their phrasing affects the final results (Walgrave et al, 2009). Although providers could potentially skew VAA recommendations, most appear genuinely keen to produce balanced applications. During the statement selection process, VAA providers tend to study the public discourse as reflected in the media and consult the general public, academics or journalists. Statements used as basis of comparison usually cover the most important policy areas. For instance, the statements used in the German Wahl-O-Mat VAA are formulated by a group of young voters, together with political scientists, statisticians, journalists and representatives of the Federal Political Education Agency. After the group has identified and formulated a large number of questions, responses are sought from the parties. Only the questions that prove to be sufficiently divisive are selected for the final application. This is done to ensure that the VAA will help voters to find differences between parties.

VAAs also differ in how they establish the positions taken by parties and candidates on the various statements. To enable comparison between users’ and parties’ preferences, the VAA provider must construct a party or candidate position database. Developers can either ask parties or their candidates to respond to a questionnaire or search party manifestos, party websites and press releases, newspaper reports and other material for parties’ issue positions (see Table 1 for the method used by each VAA in this analysis). To prevent parties from abusing the application by entering inaccurate or untrue responses, some VAA providers require parties to provide evidence for their chosen position, whereas others simply give them the option of justifying a response. In the Netherlands, the staff of the Kieskompas VAA check parties’ responses to the VAA questions to ensure their accuracy and even enter into correspondence with parties on the correctness of individual issue positions. If parties insist on a seemingly unusual position, they have to prove its accuracy with reference to party documents. The Swiss smartvote and Finnish VAAs, on the other hand, seek responses from individual election candidates. Given that there are hundreds of candidates, many of whom are unknown or even new to politics, checking positions would be labour-intensive to the point of being unfeasible. Public scrutiny, however, provides a degree of quality control even in candidate-based VAAs (see below for an example from Finland).

Asking parties and candidates to place themselves without checking the accuracy of the positions gives them an opportunity to manipulate the application. In Finland, for example, some candidates have placed themselves in the middle of the response scale on all statements, thus capturing voters from both sides of the political spectrum (O. Ainola, personal interview, 23 February 2009). Having been exposed and ridiculed in the media for having ‘no opinions’, these candidates were later forced to change their responses to be more substantial. In sum, there are inherent difficulties in how parties should be involved in the VAA design process.

Third, the applications differ in how voters can express their opinions on the statements they are presented with. Some VAAs (for example, the Austrian and French VAAs and the Dutch Stemwijzer) simply ask each user whether they agree or disagree with the statement. The German Wahl-O-Mat and the Swiss Politarena also give voters the option of saying that they are ‘neutral’ on the issue, while the Kieskompas in Belgium and the Netherlands and the Finnish YLE VAA all provide full 5-point Likert scales. For instance, the 2006 Belgian Kieskompas asked users to respond to the following statement: ‘The use of soft drugs should be legalised’. The response options ranged on a 5-point scale from ‘completely agree’ to ‘completely disagree’, with an additional ‘no comment’ option if the user wanted to ignore that particular question.

Finally, VAAs also differ in whether they allow users and parties to express issue importance by weighting individual statements or themes consisting of several questions. In our sample of 13 VAAs, only four do not include salience at all. Another five VAAs, such as the Dutch Stemwijzer, just let users select those statements or policy areas that are of particular importance to them. The Austrian Wahlkabine and Politikkabine as well as the Finnish YLE VAA go furthest in allowing voters to assign personal salience to issues.

VAA Recommendations and Issue-based Voter Representation

The advice VAAs give is based overwhelmingly (and usually only) on the issue-based congruence between voters and parties.¹ In other words, VAAs elicit voter preferences on a given set of issues and then based on these calculate a party recommendation. Yet there are various perspectives on how party policies can match those of voters: citizens can make use of issues in different ways in deciding how to vote.

The three best-known models are the proximity, directional and salience models. Thus, voters may prefer parties that are close to them ideologically (proximity model); parties that are on the same side as them on key issues and that hold these views intensely (directional model); or parties that focus on the issues they care about most (salience model). Each of these three models is a separate way in which a party may be ‘congruent’ with a voter, and we will examine the extent to which each is implemented in VAAs.

The recommendations of VAAs are calculated primarily by matching the policy positions of voters and parties. In doing so, VAAs could use either a proximity or a directional logic. A proximity logic would mean that recommendations are based on voter-party distances, usually measured on a continuous scale (Downs, 1957). In contrast, a directional logic would be indicated by three features: each issue has two ‘sides’, for and against; it is possible to express intensity of preference; and parties are not generally punished for preferences more intense than those of voters (Rabinowitz and Macdonald, 1989).

The VAA algorithm, which establishes the degree of opinion congruence between the user and parties, reveals which model of issue voting the application follows. Although some VAA providers do not publicise the exact algorithm used to compare user and party or candidate positions, for those that do it is clear that VAAs follow the assumptions of the proximity model. This means that the closeness of the match between a party or candidate and the user is determined by the degree of agreement across all the statements. The algorithm of the Swiss smartvote, for instance, gives points to candidates according to the distance between the responses of the candidates and the user. The candidate receives the most points when the user selects the same response alternative as the candidate and the least when the options selected are the furthest apart. The same logic is also apparent in the algorithm used by the Dutch Stemwijzer and the many other VAAs inspired by it. In other words, what matters is the distance, or degree of agreement, between the preferences of the voter and those of the party or candidate.

This differs from the directional approach in two important respects. First, parties are not rewarded if they are more extreme (or hold their views more intensely) than the voter. Under the directional logic a voter would not generally penalise a party for holding more intense preferences than him or her unless the party is outside the ‘region of acceptability’ (Rabinowitz et al, 1991). However, VAAs tend to assign fewer points to a party as soon as the agreement on an issue is not perfect, no matter in which direction the disagreement exists. Second, in VAAs that offer more than a simple ‘yes’ and ‘no’ as response options, there is arithmetically no ‘side’ to issues. This means that being on the same side of an issue as the user does not yield a better rating for a party than being on the other side of the issue. For example, using a 4-point Likert scale a user might say that she ‘agrees somewhat’ with a certain statement. These types of VAAs interpret the voter to be equally distant from a party that ‘agrees completely’ and one that ‘disagrees somewhat’. Each party then receives the same amount of points towards the final matching score.²

There is thus undeniably a strong proximity-based logic underlying the way that existing VAAs match voters to parties. However, the fact that the response alternatives are generally very limited in number, with a maximum of five choices available, means that voter preferences are not measured on a true ‘interval-level’ scale. Indeed, as we have seen many VAAs even restrict the choices to simple binary options. Here, VAAs may be closer to the directional approach, under which simple Euclidean distances are in any case not the key to understanding a voter's decision-making process (Rabinowitz and Macdonald, 1989). Instead, it is argued that most of the time voters do not even have clear issue positions, but rather have diffuse preferences and tend to simply side with one camp of a policy debate. Expressing such preferences is generally possible in VAAs: in all applications we examine users can choose one side of the debate. Some VAAs, such as the German Wahl-O-Mat, also offer a ‘neither agree nor disagree’ option, which fits the directional logic by providing a position of no preference. The proximity logic would include the median position between the extremes of preference instead. Although VAAs do not appear to perfectly fit either the proximity or the directional model, the underlying logic of these applications is nevertheless closer to Downsian proximity model as the degree of agreement, that is, issue proximity, fundamentally determines the outcome of the matching process and thus the recommendation given to voters.

Do VAAs also take into account the importance a party accords to an issue? Salience theory suggests that parties compete not by taking different positions on the same issues, but by stressing those issues on which they have a positional advantage (Budge and Farlie, 1983). Similarly, congruence can also be conceived of as the extent to which the priorities and agendas of voters and parties align (Jones and Baumgartner, 2004). Accordingly, voters might be expected to choose parties partly because they address the issues that concern them, for example the environment or immigration, regardless of the specific positions the party holds. A second possibility is that voters weight positional distances based on the personal salience of each issue, so issue importance can also form part of the standard proximity model.

As noted above, most VAAs, for example the Dutch Stemwijzer and applications related to it (for example, Wahlkabine in Austria and the German Wahl-O-Mat), allow voters to increase the weight of topics and statements that they consider to be of special importance or decrease the importance of issues that do not make a difference to them. The weighting is incorporated into the distance calculation, for instance by doubling the effect of a statement marked as important and halving it for those selected as not important. Nevertheless, issue salience is not part of all VAAs. Moreover, many applications, such as the Dutch Stemwijzer and Kieskompas as well as the German Wahl-O-Mat, ask users to assign salience in one of the last screens shown to the user. Users may therefore pay less attention to assigning salience to issues in that case or emphasise issues in the latter part of the questionnaire as these are fresher in their minds. In interviews, VAA providers also report that the many users do not use the weighting option when it is available. This appears to be the case both with VAAs that allow users to express issue salience as they go through the questionnaire issue by issue as well as in those VAAs where issue salience can be assigned at the end of the questionnaire. Moreover, as parties have to answer all questions, VAAs do not allow parties and candidates to concentrate only on issues on which they take popular positions and de-emphasise those issues where they take positions with less public approval. Thus, while VAAs do not disregard salience altogether, this aspect of issue voting clearly plays a subordinate role to positions, and therefore VAAs cannot be said to follow a salience logic of vote choice. Instead, it is more accurate to say that the applications allow for issue distances to be weighted in a way compatible with the proximity model.

In sum, VAAs primarily follow a proximity logic of voter representation: parties are seen as matching voters if their policy positions are relatively close to one another. Other ways in which issues can be used to match voters to parties – the directional and salience models – are either secondary considerations or disregarded completely. Having established that VAAs follow a proximity logic, we can now consider the accuracy of the recommendations provided by the online applications as compared to other established measures.

The Convergent Validity of VAA Party Placements

If VAAs are to provide useful guidance to voters, then the party positions encoded in VAAs need to be relatively accurate. In this section, we examine whether party positions recorded in VAAs match established measures, specifically expert survey and party manifestos. Comparing VAA measures with other existing assessments of party positions is possible as all approaches follow a proximity logic. Of course, even if there is such a thing as a ‘true’ and ‘objective’ party placement, discovering this is essentially impossible, and the assessment of the location of political parties will always be approximate (Benoit and Laver, 2006). When estimating the ‘accuracy’ of VAA positioning, an appropriate approach is to decide whether the recorded placements are plausible, convincing and accord with existing assessments. In the absence of a ‘true’ measure of party positions, we examine the convergent validity of VAA party placements by comparing them with other established measures (Adcock and Collier, 2001; Ray, 2007). We use the expert surveys of Benoit and Laver (2006) and Hooghe et al (2010) as well as party manifestos as coded by the Comparative Manifesto Project (CMP) (Budge et al, 2001; Klingemann et al, 2007). We coded the placements from the 13 VAAs in seven countries listed in Table 1 and compared these with the general left--right measures as well as more specific policy scales.

We begin by comparing left--right scores extracted from VAAs with the three established measures. This broad comparison is appropriate because left-right summary dimensions exist in all countries where parties compete on policies (McDonald and Budge, 2005). Moreover, extracting left-right scores allows us to make use of all the available questions in VAAs.³ The left-right scores were calculated from the 13 VAAs using multidimensional scaling (MDS). This procedure was chosen due to the low number of items in most of the VAAs and the low number of response alternatives (between two and five). Using MDS also means that no assumptions need to be made concerning the policy content and direction of the questions included in the VAA: instead, the procedure provides an estimate of the overall relative party positions across all topics based only on the party answers to the VAA questions.

The precise steps used to extract left-right positions were as follows. The party positions on each question were coded numerically, with the specific coding depending on the response alternatives available in each VAA.⁴ Then, metric MDS was run on these coded positions.⁵ Two dimensions were extracted using random starting points and 50 iterations. Scores were then recorded for the first dimension extracted; this is assumed to be the dimension that best summarises the positional relationships in party responses and should be most similar to a left-right summary dimension. Finally, the scores were standardised in order to make them more comparable cross-nationally; they were also reversed where necessary so that well-known left-wing parties are to the left on all extracted scales.

First, we present broad cross-national patterns of similarity between VAA-extracted and other measures of left-right position, following the approach used by Benoit and Laver (2007).⁶ Figure 1 plots the left-right scores from the VAAs and those from the two expert surveys and the manifestos.⁷ Each point is identified by its country and party name. A fitted regression line and a 95 per cent confidence interval are also shown. Vertical and horizontal lines are inserted at 0 for the VAA scores, at the midpoints of the expert surveys (10.5 and 5, respectively) and the mean of the CMP ‘vanilla’ scores (−0.56).

Figure 1

Across-country scatterplot of VAA-extracted left-right scores and expert survey and manifesto estimates. Note: For sources of VAA data, see Table 1. Benoit and Laver data from Benoit and Laver (2006); ‘Hooghe and Marks’ data from Hooghe et al (2010); CMP data from Klingemann et al (2007). See text for details on how positions were extracted from the VAAs and the CMP data.

The first noticeable pattern is that the fit of the VAA left-right positions with the other measures is very close. In all three cases, the predicted regression line intersects with the midpoint of the two scales. In addition, the linear fit of the regression line is very good.⁸ Moreover, the party placements are overwhelmingly very close to predicted regression line, with the manifesto scores showing the most ‘noise’. Finally, the VAA scores classify parties as ‘left’ and ‘right’ in the same way as the other measures, as can be seen by the fact that the top left and bottom right quadrants are overwhelmingly empty. Only the Dutch Christian Union (CU) is placed differently compared to the expert surveys. The VAA measure and the manifesto scores vary a little more in this regard, with the VAA coding a small number of parties as left that are on the right according to the manifesto data (top left quadrant). No clear pattern is, however, visible from the parties that are placed differently, and the parties in question differ from measure to measure. The overall impression from this first comparison is thus that the left-right placements of parties in VAAs are very similar to other, established measures of party positions.⁹

Second, beyond this broad cross-national approach it is possible to compare the VAA scores with the established measures in each country separately. It is worth doing so because scores are not necessarily comparable across countries and because it is important that VAA placements are accurate for each country and not just cross-nationally. Here, we again follow the approach used by Benoit and Laver (2007) to evaluate expert survey and CMP scores: in each country, the ranking of parties (rather than their scores) is compared. In this way, any artificial ‘accuracy’ of numerical left-right scores is abandoned in favour of a simpler comparison.

The details of this ordinal comparison are presented in Table 2. The left-right placements for each country are shown along with the results for two measures of association for ordinal-level data, Kendall's τ and Spearman's ρ. Both range from –1 to 1. Parties that differ in their placement in the three established measures (compared to the VAA rank) are highlighted in bold. Looking first at the actual rankings extracted from the VAAs, these have quite strong face validity.¹⁰ The German left-right rank of PDS-Green-SPD-FDP-CDU, for example, would accord with most observers’ left-right placement of parties. This does not mean that there are not some obviously odd ranks: the Austrian Greens might be surprised to learn that they are to the left of the Austrian Communists, while the liberal Swiss FDP has probably rarely been placed to the left of the PdA, the workers’ party (as occurs in the Politarena VAA). Overall, the level of agreement between the VAA ranks and the other measures is nevertheless high. The Spearman's ρ is above 0.8 for almost all expert survey comparisons, with only the Finnish YLE VAA falling a little below that. The VAA ranks match a little less well with the CMP ranks, with four VAAs falling below 0.8 in agreement measured by Spearman's ρ, one each in Belgium and Finland and two in Switzerland. Given the close fit between VAA and expert survey party placements, it is not surprising that VAA scores and ranks match less well with manifesto estimates of party positions: it is well-established that CMP and expert survey left-right scores are highly but by no means perfectly correlated (McDonald, 2004; Benoit and Laver, 2007; Ray, 2007).¹¹

Table 2 Within-country comparisons of VAA left-right rankings and rankings by three other established measures

So far, the comparison with established measures has been made only on the broad left-right measure, with scores extracted from all VAA questions available. It is worth considering how well VAA placements compare with other measures on specific issue areas. After all, voters may want their policy preferences represented on particular topics rather than an aggregate left-right dimension. Here, economic policy, immigration and the environmental issues are chosen for comparison. Economic policy is central to both voters and parties in most elections and this is reflected in the large number of questions on this area. Immigration and environment questions on the other hand, although still important, are perhaps not as central to most voters, and VAAs reflect this by including fewer questions on these policy areas. This topic selection should allow us to assess the VAAs’ ability to capture party positions in policy areas with large and small number of questions.

The policy positions on these specific issue areas were calculated using a simpler method than for the left-right scores. For each VAA, we first decided which questions had clear economic or fiscal policy, immigration or environmental content. Then, the direction of the question was coded in order to establish whether answering ‘yes’ indicates a ‘left-wing’ or ‘right-wing’ position. For example, on the German VAA, on the question ‘It should be easier to fire employees’ a ‘yes’ is a right-wing response, whereas on the question ‘Higher taxes for higher earners?’ a ‘yes’ is a left-wing response. The positions were coded independently by two coders. There was agreement on 130 out of the 134 total economic policy items coded in this way; agreement on immigration policy was present on 34 out of 36 items and on 40 out of 41 items on environment policy. The few questions on which the coders disagreed were dropped from the country-specific scale. Then, the overall policy position was calculated as Right minus Left responses.

The results for economic policy are presented in Table 3, following the approach for the ordinal comparison of left-right ranks. Even with this issue-specific measure, agreement between the VAAs and the expert surveys is high. Compared with the Benoit and Laver ranks, only two of the 13 VAAs fall below 0.87 using Spearman's ρ, whereas three fall below 0.87 compared with the Hooghe et al ranks. Generally, then, the measure of economic policy in VAAs is very similar to established measures. However, the VAAs with fewer than ten questions on economic policy matters tend to exhibit less impressive rank order correlations. The fewer the questions, the harder it appears to be to identify party ranks, at least if we take expert surveys as a credible benchmark. Table 3 also compares the VAA economic policy ranks with CMP-extracted ranks.¹² The differences between the CMP and the VAA ranks are again greater than in the comparison with expert surveys. Spearman's ρ is still higher than 0.8 for seven of the 13 VAAs and over 0.68 for 11 VAAs. The lowest agreements between the two ranks are for 2006 Austrian Wahlkabine and the Finnish YLE VAA. Overall, the association between numbers of questions asked and similarity of placements with CMP ranks is less strong: the Finnish YLE VAA with 15 economic questions does badly although the French VAA with just six such questions has the highest convergent validity compared with CMP ranks. Still, the fact that the Swiss smartvote and the Belgian and Dutch Kieskompas, which each have a large number of relevant questions, match CMP ranks particularly well further underlines our finding that the VAAs with more economic policy questions tend to place parties more convincingly on this issue area.

Table 3 Within-country comparisons of VAA economic policy rankings and rankings by three other established measures

We also compared immigration and environment policy ranks with the two expert surveys. The detailed results are available from the authors; here, we restrict ourselves to a brief description of the key findings. Most importantly, we find that on these two issues the rankings of several parties are often identical. In other words, on environmental and immigration issues VAAs fail to distinguish the positions of different parties. On the immigration issue, the clearest such example is the 2008 Austrian Politikkabine, where all parties apart from the FPÖ are seen as having the identical policy position. In France, the PS and the PCF as well as the UMP and the MoDem are also seen as indistinguishable. The only VAA that really manages to separate parties on immigration policy is the Swiss smartvote, which asks 11 questions on this topic. Very similar findings also apply to the environment ranks extracted from the VAAs. In terms of guidance on secondary issue areas, VAAs are therefore relatively limited.

We can therefore conclude that VAA-extracted positions are more likely to be valid using established measures as a benchmark if a relatively large number of questions are asked on that topic. If expert surveys and manifesto data are accepted as a benchmark, the party positions encoded in VAAs are largely accurate in terms of overall left-right and economic policy terms, but on secondary issues the relative limitations of the VAA design become apparent. Overall, though, it appears that VAAs do quite well at providing guidance on general issue-based congruence between parties and voters based on proximity assumptions.

Discussion and Conclusion

VAAs match voters to parties mostly according to the proximity logic, the dominant issue-voting paradigm. Although there are elements of the directional and salience models, recommendations are primarily structured around the degree of policy agreement between a party or candidate and the user. In this sense, VAAs can be said to reflect the responsible party model (APSA, 1950), as they help voters connect their policy preferences to those of internally cohesive parties.¹³ VAAs therefore conceive of citizen representation as voter-party congruence on key policies.

Moreover, we find that VAAs encode party placements in accordance with the measures of policy positions commonly used by political scientists. This means that VAAs fulfil their aim of matching voters to parties well, especially if we focus on left-right congruence between voter and party positions. As a side benefit, the convergent validity of the VAA scores means that data from these applications could be used to provide election-specific party placement information, especially if the VAA is created by a trustworthy provider (for a similar argument, see Trechsel and Mair, 2011).

However, VAAs are less able to establish clear party positions on issues that are less central to the political debate, as the detail and thus accuracy of party placements at least partly depends on the number of questions asked on each topic. Here we make a recommendation to future VAA providers. For more accurate voting advice, provide as many questions as possible on the key policy areas of concern to voters. If voter-party congruence on the environment or immigration, for example, is a key factor, then more than one or two questions need to be asked. Obviously, VAAs are limited in how many questions they can include, but users could be offered short and long versions of the same application, as in the Swiss smartvote and the Finnish YLE VAA. Moreover, the number of questions should be higher when more parties need to be placed. In large party systems, more questions will be needed to effectively distinguish parties from each other.

Finally, VAAs focus on a specific type of voter-party link, namely through policies and opinions. This means that other forms of representation are disregarded: voters may have other concerns and other objectives in determining their vote choice than simple policy congruence. First, VAAs do take into account non-policy attributes of parties and politicians, such as their competence, trustworthiness or public spiritedness, yet these are empirically and normatively important to citizens’ electoral decision making (Stokes, 1963; Fiorina, 1981; Clarke et al, 2009; Mansbridge, 2009). Moreover, accountability for past behaviour and the alternation of governments are also frequent and legitimate democratic concerns (Schumpeter, 1942; Manin et al, 1999).¹⁴ When considering how VAAs match voters to parties, we need to be aware that their purely issue-based perspective focuses on a specific (and perhaps narrow) reason for liking and disliking political parties. Second, voters may not care about parties themselves but rather be outcome-oriented (Kedar, 2009). For example, as Stokes (1963) notes, voters do not make their decisions based solely on positional issues but also consider which party is most likely and most able to bring about their desired state of affairs. In parliamentary democracies, it is often coalitions and not single parties that primarily decide on policy outcomes (Kedar, 2009). Simple party-voter distance may therefore not be the congruence that voters are really interested in if what they actually care about are policy outcomes.

Of course, VAAs are limited in the extent to which they are able to include past performance, party competence and potential coalition agreements in their design. The fact that VAAs focus on ideological proximity is thus not a criticism of the role and impact of VAAs, which after all only claim to provide guidance on how well policy positions match those of voters. Yet, it is worth pointing out the difficulty of capturing complicated decision processes in relatively simple applications.

VAAs reflect a proximity-based logic of matching voters to parties and candidates, and based on our evidence they fulfil this logic well. By emphasising the role of policy agreement in vote choice, these relatively new applications may have an important effect on how voters think about politics and how they evaluate parties. For example, using VAAs may lead voters to depend more on policy positions and relative proximities when making vote decisions. As such a widely-used resource, VAAs may be able to provide an important insight into the complicated decision-making process of voting. Future research should make use of these opportunities.

Appendix

Table A1

Table A1 Web addresses of the VAAs studied