Motivated formal reasoning: Ideological belief bias in syllogistic reasoning across diverse political issues

Abstract This study investigated ideological belief bias, and whether this effect is moderated by analytical thinking. A Swedish nationally representative sample (N = 1005) evaluated non-political and political syllogisms and were asked whether the conclusions followed logically from the premises. The correct response in the political syllogisms was aligned with either leftist or rightist political ideology. Political orientation predicted response accuracy for political but not non-political syllogisms. Overall, the participants correctly evaluated more syllogisms when the correct response was congruent with their ideology, particularly on hot-button issues (asylum to refugees, climate change, gender-neutral education, and school marketization). Analytical thinking predicted higher accuracy for syllogisms of any kind among leftists, but it predicted accuracy only for leftist and non-political syllogisms among rightists. This research contributes by refining a promising paradigm for studying politically motivated reasoning, demonstrating ideological belief bias outside of the United States across diverse political issues, and providing the first evidence that analytical thinking may reduce such bias.


Introduction
Modern societies place high demands on our ability to evaluate information. With so much information constantly at our fingertips, it is crucial not just that we can separate fact from fiction, but also that we can make valid inferences from evidence. The human mind is of course not infallible. that people evaluate information less favorably when it is discordant with their political positions than when it is concordant with their political positions has typically been interpreted in terms of motivated reasoning, although it is also possible that prior beliefs combined with a motivation to reach accurate conclusions could produce such an outcome. To ensure that the results can be explained in terms of politically motivated reasoning, it is, as Tappin et al. (2020) conclude, important to limit the influence of relevant prior beliefs and information on the outcome, through statistical control or design.
The study of ideological belief bias in syllogistic reasoning is a promising paradigm for identifying politically motivated reasoning because logical validity is, by definition, independent of the truth of the premises or conclusions of the syllogism. Prior beliefs and information are irrelevant to the rational evaluation of whether the conclusion follows from the premises (i.e. logical validity). Therefore, the influence of prior beliefs is, insofar as participants are given the proper instructions and resources to understand the concept of logical validity, controlled by design. Controlling a confound by design has many advantages over attempting to control it through statistical analysis (Shadish et al., 2002;Westfall & Yarkoni, 2016).
Ideological belief bias: Findings, limitations, and refinement of the paradigm So far, results from three studies reported by Gampa et al. (2019) and two studies reported by Calvillo et al. (2020) have produced clear evidence of ideological belief bias in the evaluation of syllogisms among both liberals and conservatives in the United States. All five studies demonstrate ideological belief bias, although this bias was contingent on ideological content of the conclusions or validity of the arguments in a handful of cases (e.g. ideological belief bias was only found for arguments with conservative conclusions in one of the studies reported by Gampa et al., 2019).
In all five studies, the syllogisms were balanced in terms of logical validity and ideological believability-half of the syllogisms contained conclusions that were consistent with liberal ideological beliefs (e.g. "Abortion is not murder") and half contained conclusions that were consistent with conservative ideological beliefs (e.g. "Tax increases harm the economy"), and half of these syllogisms were logically valid while the other half were logically invalid. Gampa et al. (2019) included four types of conditional, propositional syllogisms that were balanced in terms of logical structure within each political topic. The two types of valid syllogisms were affirming the antecedent (Modus Ponens; "If P, then Q. P. Therefore, Q.") and denying the consequent (Modus Tollens; "If P, then Q. Not Q. Therefore, not P."). The two types of invalid syllogisms were affirming the consequent ("If P, then Q. Q. Therefore, P.") and denying the antecedent ("If P, then Q. Not P. Therefore, not Q"). The studies reported by Calvillo et al. (2020) instead included two types of valid categorical syllogisms (e.g. "All popular beliefs can be taught in public schools, Creationism is a popular belief, therefore Creationism can be taught in public schools") and two types of invalid categorical syllogisms (e.g. "All moral things are legal, Abortions are legal, therefore Abortions are moral actions").
Despite their considerable strengths, the past studies on ideological belief bias also have a number of limitations. Our goal was to retain the best design features from the previous studies, while trying to address their limitations.
One potential issue is that in two out of three of the studies reported by Gampa et al. (2019), the participants were in fact asked to evaluate the logical soundness rather than the logical validity of the syllogisms. In formal logic, a syllogism is sound if it is valid and has true premises. Logical validity on the other hand is independent of the truth of the conclusions (it means that the conclusion must be true if the premises are true). In other words, it would be rational rather than biased for a person to judge a valid argument with a conclusion that is inconsistent with his or her ideological convictions as unsound. Effects of prior beliefs could therefore have confounded the assessment of bias in syllogistic reasoning in the manner described by Tappin et al. (2020).
It is not likely that this issue substantially affected the results in the aforementioned studies, because most people are probably not familiar with the subtle distinction between logical validity and soundness, and the participants did receive additional instructions; there was even a training session that introduced the essentials of logic in one of the three studies in Gampa et al. (2019). It does however illuminate the importance of making sure that the participants understand what the task is. In the current research, we therefore let participants complete a longer training session with a brief introduction to logic and several training syllogisms followed by immediate feedback on the correct answer and a brief explanation. The syllogisms were designed so as to emphasize the distinction between the validity of a syllogism and the truth of a conclusion (e.g. valid syllogisms with unbelievable conclusions and invalid syllogisms with believable conclusions were included).
A second potential issue concerns the matching of liberal and conservative conclusions needed to accurately measure belief bias on both sides of this ideological spectrum. Although Calvillo et al. (2020) included equal numbers of syllogisms with liberal and conservate conclusions and with affirmative and negated conclusions, the correct response was always consistent with a liberal worldview for four of their topics (atheism, global warming, homosexual marriage, and death penalty) and consistent with a conservative worldview for four other topics (creationism, guns, abortions, and immigrants). In other words, Calvillo et al. (2020) reduced the risk of confounds by using an equal number of affirmative/negated, pro-liberal/ pro-conservative, and valid/invalid syllogisms in total across all their eight topics, but did not balance these aspects of the syllogisms within each topic. Our study reduced this risk even more by manipulating all of these aspects of syllogisms within each political topic, and by keeping the wording identical. In addition, unlike previous studies, we included manipulation checks to assess the extent to which the participants perceived the conclusions of the syllogisms as politically left or right, and the extent to which leftists and rightists agreed with the conclusions.
A further limitation of the previous studies of ideological belief bias is that they have studied ideological belief bias on a just a handful of political issues (e.g. affirmative action, abortion, capital punishment, and government intervention) among liberals and conservatives in the United States (using convenience samples in all studies except one), and they did not investigate ideological belief bias separately for each topic. Our design allowed us to investigate potential variations in ideological belief bias across a diverse set of political issues (labor market, private health care, marketization of the school system, gender-neutral education, multiculturalism, military defense, asylum to refugees, and climate change) with a nationally representative sample of Swedes. In contrast to the United States, which has a biparty system that pits liberals against conservatives, Sweden has a multiparty system with an ideologically diverse group of parties (e.g. economic left, green, social democrat, social liberal, liberalconservative, and nationalist and non-nationalist conservative parties; Nilsson et al., 2019Nilsson et al., , 2020. We selected political issues for the syllogisms so as to make them representative of this ideological diversity, and we measured both ideological left-right self-placement (similar to previous studies) and party preference.

The role of analytical thinking
Why do people fall prey to ideological belief bias? On classical dual-process accounts (e.g. De Neys, 2012;Epstein, 1994;Kahneman, 2011;Stanovich & West, 1998), the ability to avoid biased reasoning requires engaging in more deliberative, effortful, and slow ("Type 2") processes, which inhibit spontaneous erroneous responses, rather than intuitive, fast, and automatic ("Type 1") processes, which elicit errors generated through cognitive biases. Research in support of this explanation has suggested, for instance, that performance on the cognitive reflection test (CRT; Frederick, 2005), which measures the ability and motivation to engage in analytical thinking, predicts the ability to discern fake news from real news (Pennycook & Rand, 2019) and pseudo-profound "bullshit" from meaningful sentences (Nilsson et al., 2019). There is also some evidence that analytical thinking reduces belief bias in reasoning problems (Toplak et al., 2011;Trippas et al., 2015).
By contrast, Kahan et al. (2017) have advanced a competing account, arguing that deeper cognitive processing can be utilized by an individual to interpret information in belief persistent ways and reject information that threatens cherished beliefs (i.e. to rationalize, justify, and defend these beliefs)a phenomenon they refer to as motivated system 2 reasoning. The prediction on this account is that analytical thinking exacerbates rather than mitigates ideological belief bias in reasoning. In support of this prediction, Kahan et al. (2017) found that numeracy predicted less confirmation bias for neutral information but more confirmation bias for politically framed information.
Building on this body of research on the role of analytical thinking in reasoning biases, we wanted to investigate whether these processes might mitigate or magnify ideological belief bias to test the aforementioned theoretical accounts. Calvillo et al. (2020) did take analytical thinking, operationalized in terms of scores on the CRT, into consideration in their study of ideological belief bias. They found that analytical thinking did predict higher accuracy in evaluating the validity of syllogisms in general, consistent with traditional dual-process accounts, but they found no significant effect of analytical thinking on ideological belief bias-that is, participants with better CRT performance did not demonstrate a larger political belief bias than those with worse CRT performance. However, they only had the statistical power to detect a small effect (operationalized in terms of a three-way interaction). In the current research, we addressed this issue using a refined experimental paradigm, as detailed above, and considerably higher statistical power (our sample was larger, and we operationalized the effect of analytical thinking in terms of a two-way interaction, which reduces the risk for type 1 error).

Overview of research
In sum, we investigated ideological bias in syllogistic reasoning in a sample of Swedish adults (N ¼ 1005), which was nationally representative in terms of demographic characteristics. All participants initially completed a training session designed to illuminate the distinction between validity and truth in logic. They thereafter evaluated the validity of ten syllogisms (eight political and two non-political) balanced in terms of validity (valid or invalid), difficulty (affirmative or negated conclusions), and ideology (leftist or rightist conclusion). The ideological content of the syllogisms spanned a range of topics (labor market, private health care, marketization of the school system, gender-neutral education, multiculturalism, military defense, asylum to refugees, and climate change) selected to capture the diversity of ideology in the Swedish multiparty context. Finally, the participants completed manipulation checks and measures of cognitive and meta-cognitive tendencies and ideological orientations.
The general hypothesis was that the participants would be more inaccurate in evaluating the validity of syllogisms when there was a mismatch between validity and believability of the conclusion according to the participant's ideological position-in other words, we expected them to exhibit ideological belief bias. We investigated the robustness of this effect across different political issues in an exploratory manner. We also investigated whether analytical thinking reduces or increases ideological belief bias and accuracy in general.

Participants
We collected the data in June of 2020 in collaboration with Origo, which is an independent research firm. We requested a sample of 1000 participants who had passed our attention check, and that was representative of the Swedish population in terms of age, gender, education level, geographic region, and political sympathies. We received data from 1283 participants and excluded eight participants who did not give their consent to participate, and 270 participants who failed our attention check. Out of the remaining participants, six did not complete the full survey but were included in the analyses, leaving our total sample at 1005 participants (500 women, 497 men, 2 unspecified gender, and 6 who did not complete the demographic section on age or gender; M age ¼ 46.3 years, SD ¼ 15.5). A power analysis conducted in G Ã power 3.1.9 suggested that this sample yielded 94.8% power (two-tailed) to detect a small interaction effect (f ¼ 0.1) in a 2 (leftist/rightist conclusion) Ã 7 (left-right self-placement) mixed ANOVA. This was the analysis that tested the central hypothesis.

Materials and procedure
The participants completed an online survey. In the first section, which measured belief bias, the participants evaluated the logical validity of syllogisms. This was followed by one of two manipulation checks, and measures of analytical thinking, political orientation, and demographics. All of these are described in detail below. 1 The dataset and all materials can be found in the supplementary material.

Training session
Prior to evaluating syllogisms with political content, the participants completed a training session designed to ensure that they understood the concept of logic. The training session consisted of an introduction to logic followed by five syllogisms with non-political content. Similar to Gampa et al. (2019), we used four types of syllogisms that varied in terms of their validity (valid vs. invalid) and their presumable difficulty (affirmative vs. negated conclusion): 1. Valid affirmative (affirming the antecedent or Modus Ponens; If P, then Q. P. Therefore, Q.). 2. Valid negated (denying the consequent or Modus Tollens; If P, then Q.
Each of these four types appeared in the training session. The participants were instructed to disregard any beliefs about the content they may normally have and strictly evaluate the logical validity by responding to whether the conclusion followed logically from the premises ("Yes" or "No"), while assuming that the premises are true. After evaluating each of the syllogisms, the participants were given immediate feedback on whether they had responded correctly or not, along with a brief explanation of why the conclusion was logically valid or invalid.

Logical reasoning task
After the training session, the participants evaluated ten syllogisms in a section that also included an attention check phrased as a syllogism. Before they were shown the first syllogism, the participants were reminded that their task was to evaluate the logical validity of the conclusion while assuming that the premises were true. Due to time constraints, they were asked to try not to think too long before responding. They were once again asked to respond to whether or not the conclusion followed logically from the two premises ("Yes" or "No"). The first and the tenth syllogism were the same for all participants and had non-political content. The first syllogism was of the invalid affirmative type and consisted partially of nonsense words, as inspired by Norenzayan et al. (2002): "If knthzor has two legs, then knthzor can not participate in Umpt; Knthzor can not participate in Umpt; Therefore, knthzor has two legs". The tenth syllogism was similar but was of the invalid negated type. All negations (the word not) in the syllogismsin both the training session and in the syllogisms with political content described belowwere in italics.
In between the two syllogisms with non-political content, the participants evaluated eight syllogisms with political content. We created eight different topics that would make up the political content and aimed to cover an as broad spectrum of issues as possible that would be relevant in a Swedish context. Aside from varying the topics in terms of logical validity (valid or invalid) and difficulty (affirmative or negated conclusion), we created one leftist and one rightist version of each topic. Thus, we used a 2(affirmative/negated) x 2(valid/invalid) x 2(leftist/rightist) design. We used the same wording in all versions for each topic and created the leftist and rightist versions by switching the place of the negation (the word not). For instance, our topic on labor market in the affirmative type of syllogism with a leftist valid conclusion read: "If the labor market is not fair, then the state should intervene to equalize income.
The labor market is not fair. Therefore, the state should intervene to equalize income." The affirmative type of syllogism with a rightist counterpart read: "If the labor market is fair, then the state should not intervene to equalize income.
The labor market is fair. Therefore, the state should not intervene to equalize income".
All topics and their corresponding conclusions without any negations can be found in Table 1. We balanced the topics in terms of negations such that there were equal numbers of syllogisms with negated leftist conclusions and negated rightist conclusions. We randomly assigned participants to one of 24 conditions. The participants in each condition evaluated eight syllogismsone from each topic. These conditions were balanced in terms of difficulty, logical validity, and political orientation, including four affirmative and four negated syllogisms, four valid and four invalid syllogisms, and four syllogisms with leftist conclusions and four with rightist conclusions. See supplementary material for all syllogisms in all topics.
We operationalized performance on this task in terms of the proportion of correctly evaluated syllogisms for (1) non-political syllogisms, (2) Table 1. Topics, leftist and rightist conclusions of corresponding syllogisms, and results from the two manipulation checks (MC1 ¼ perceived political position of the conclusion. MC2 ¼ correlation with right vs. left self-placement).

Issue
Leftist conclusion Rightist conclusion

Labor market and state intervention
The government should intervene to equalize income.
The labor market is fair. Knthzor has two legs.
Balaenoptera musculus is a mammal.
Note. For perceived political position of the conclusion (MC1): The response scale ranged from 1 (strongly left) to 5 (strongly right). One-sample t-tests were conducted for each conclusion to investigate the difference from the theoretical mean (test value ¼ 3).
syllogisms for which the correct response was aligned with leftist ideology (a valid syllogism with a leftist conclusion or an invalid syllogism with a rightist conclusion), and (3) syllogisms for which the correct response was aligned with rightist ideology (a valid syllogism with a rightist conclusion or an invalid syllogism with a leftist conclusion).

Manipulation checks
We created two different manipulation checks to ensure that the conclusions had the political meaning we intended; (1) that our leftist and rightist conclusions were perceived as typically "leftist" and "rightist" respectively, and (2) that participants' level of agreement with each conclusion correlated in the expected direction with their general right (vs. left) self-placement. We randomly assigned participants to one of the two manipulation checks until we had reached 20% of participants for manipulation check 1 (MC1; perceived political orientation), and 80% of participants for manipulation check 2 (MC2; agreement with content of conclusion). The reason for the uneven sample sizes was that we expected stronger effect sizes for perceived political orientation. In MC1, we asked participants to rate the perceived political orientation of all 16 conclusions, phrased as statements without any negations. For instance, the two statements regarding labor market read as follows: "The state should intervene to equalize income", and "The labor market is fair". The participants rated the statements ("Please read the following statements carefully and judge to what degree you think each statement expresses a typical 'leftist opinion' or a typical 'rightist opinion' in a Swedish context") on a scale from 1 ("Very typical leftist opinion") to 5 ("Very typical rightist opinion").
In the second manipulation check (MC2; agreement with content of conclusion), the participants were asked to evaluate the same 16 statements and respond to whether they agreed with them or not on a scale from 1 ("Do not agree at all") to 5 ("Agree completely"). These scores were correlated with general left-right self-placement.
Both manipulation checks supported our operationalization of leftist and rightist conclusions, as shown in Table 1. All statements significantly differed in the anticipated direction from the theoretical mean of the scale (M ¼ 3), when we conducted one-sample t-tests. This shows that all the leftist conclusions were perceived as representative of leftist ideology and all the rightist conclusions were perceived as representative of right-wing ideology. Furthermore, all correlations between agreement with the statements and self-reported political orientation were significant and in the expected direction. This shows that the more to the political right (vs. left) participants were, the more they agreed with the rightist conclusions and the less they agreed with the leftist conclusions.

Analytical thinking
We measured analytical thinking with the three original cognitive reflection test items (Frederick, 2005; M ¼ 0.95, SD ¼ 1.07; a ¼ .67), which has previously been translated into Swedish and used in several studies (e.g. Nilsson et al., 2019). The currency and amount are replaced in the first item to make it fit in a Swedish context: "A bat and a ball cost 110 SEK in total. The bat costs 100 SEK more than the ball. How much does the ball cost?".

Political orientation
The participants reported general left-right self-placement on a scale from 1 ("Very far to the left") to 9 ("Very far to the right"). M ¼ 5.01, SD ¼ 1.91). They also reported party preference ("If there was an election today, what party would you vote for?") by selecting one of 10 options: 1 ¼ The Left Party The participants also reported political self-placement in terms of social conservative and economic issues, respectively. With respect to social conservatism, they responded to whether they leaned more towards social liberalism ("People who are socially liberal prioritize the value of individual freedom and rights") or social conservatism ("People who are socially conservative prioritize the value of maintaining norms and traditions") on a scale from 1 ("Very liberal") to 7 ("Very conservative"); M ¼ 3.63, SD ¼ 1.52. With respect to economic issues, they responded to whether they were more leftist ("People who are more leftist emphasize economic equality") or rightist ("People who are more rightist emphasize individuals' right to reap their financial assets") from 1 ("Very leftist") to 7 ("Very rightist"); M ¼ 4.01; SD ¼ 1.67.

Demographics
The participants reported their gender, age, and level of education (from 1 ¼ "Not finished elementary school" to 5 ¼ "University degree"; M ¼ 3.81, SD ¼ 1.06).

Aggregated belief bias across all political topics
As a first step, we calculated bivariate correlations to examine associations between ability to accurately evaluate syllogisms, analytical thinking, political orientation, and level of education. These correlations are shown in Table 2. The correlations between political orientation and accuracy in solving syllogisms provide evidence for ideological belief bias. Consistent with our expectations, general right (vs. left) self-placement correlated positively with accuracy on syllogisms for which the correct answer was aligned with rightist ideology and negatively with accuracy on syllogisms for which the correct answer was aligned with leftist ideology. In addition, social conservative (vs. liberal) self-placement correlated negatively with accuracy for syllogisms for which the correct answer was leftist, and economic right (vs. left) self-placement correlated positively with accuracy for syllogisms for which the correct answer was rightist (see Table 2). In other words, leftists and social liberals performed worse on syllogisms where the correct answer was not aligned with leftist ideology, whereas general and rightists performed worse when the correct answer was not aligned with rightist ideology.
To formally investigate ideological belief bias across all eight political topics, we next conducted a 2 Â 7 mixed ANOVA with type of syllogism (leftist or rightist) as a within-subject factor and participant's political orientation as a between-subject-factor. We merged the extreme response categories (1, 2 and 8, 9) for the left-right self-placement item to make the groups larger (n ¼ 104, 136, 139, 229, 144, 153, 94). The dependent variable was the proportion of correctly solved syllogisms.
There was no main effect of political orientation F(6, 992) ¼ 1.05, p ¼ .388, which means that leftist and rightist participants solved about equally many syllogisms in total. There was however a main effect of syllogism type, F(1, 992) ¼ 8.42, p ¼ .004, g p 2 ¼ .008, such that the leftist syllogisms were slightly easier to solve overall. Most important, there was a significant interaction effect, F(6, 992) ¼ 4.02, p ¼ .001, g p 2 ¼ .024. This effect is illustrated in Figure 1. Follow-up contrast tests showed that participants who self-classified as left-leaning solved more syllogisms for which the correct answer was leftist (t[992] ¼ À3.40, p ¼ .001), fewer syllogisms for which the correct answer was rightist (t[992] ¼ 2.97, p ¼ .003), and about equally many neutral syllogisms (t[992] ¼ 0.20, p ¼ .843), compared to participants who classified as right-leaning. There was also no difference in the proportion of correctly solved training items (t[992] ¼ .0.32, p ¼ .747). Similar results were found when conducting non-parametric Mann-Whitney tests (p ¼ .002 [.013] for syllogisms for which the correct answer was leftist [rightist], and p's ¼ .643 and .626 for neutral syllogisms and training items respectively).
A very similar pattern emerged when we used party preference as the independent variable. This effect is illustrated in Figure 2. The parties can be roughly divided into three groups: left-wing parties (the Left, Green, and Social Democrat parties), social liberal right-wing parties (the Liberal and Center parties), and conservative right-wing parties (the Moderate, Christian Democrat and Sweden Democrat parties). Planned contrasts showed that participants who supported one of the three leftist parties correctly evaluated more leftist syllogisms (t[804] ¼ À4.46, p <. 001), fewer rightist syllogisms (t[804] ¼ 2.31, p ¼ .021), and about equally many non-political syllogisms (t[804] ¼ À0.24, p ¼ .813), compared to participants who

The role of analytical thinking
Consistent with the traditional dual process account of motivated reasoning, the bivariate correlations reported in Table 2 show that analytical thinking and education were associated with greater accuracy for all types of syllogisms, except for the ones with rightist conclusions. To formally test the system 2 motivated reasoning account, we also conducted two hierarchical regression analyses with accuracy for syllogisms with a correct leftist response and accuracy for syllogisms with a correct rightist response as the outcome variables, education, analytical thinking, and right (vs. left) self-placement added as predictors in a first step, and the interaction between political orientation and analytical thinking added as a predictor in a second step.
Results from these analyses are summarized in Table 3. Leftist self-placement predicted better performance on leftist syllogisms and rightist selfplacement predicted better performance on rightist syllogisms (Step 1). This result shows that ideological belief bias was robust even when analytical thinking was adjusted for. Furthermore, for leftist syllogisms, analytical thinking predicted better performance (Step 1) but the interaction between ideological self-placement and analytical thinking was not significant (Step 2); for rightist syllogisms, analytical thinking did not predict better performance (Step 1), but the interaction effect was marginally significant (Step 2).
To illustrate the marginal interaction effect of analytical thinking and ideological self-placement on accuracy for rightist syllogisms, we plot the Table 3. Results of hierarchical regression analyses predicting accuracy in terms of education, analytical thinking, and ideological self-placement. correlation between analytical thinking and correctly solved syllogisms for different participants and different syllogisms in Figure 3. While analytical thinking predicted better performance on all types of syllogisms for leftists, it did not improve performance on rightist syllogisms among rightist or centrist participants. This pattern is in the opposite of what the system 2 motivated reasoning account would predict. It suggests that analytical thinking might have helped participants on the right to avoid ideological bias, consistent with the traditional dual-process account of motivated reasoning.

Results on each political issue
For each of the eight political issues covered in the syllogisms (see Table 1), we conducted a binomial logistic regression. The dependent variable was whether participants correctly classified the syllogism as valid or invalid (0 ¼ incorrect classification, 1 ¼ correct classification). The independent variables were (1) whether the syllogism was of the simple type with an affirmative conclusion [0] or the more difficult type with a negated conclusion [1], (2) whether the syllogism was valid [0] or invalid [1], (3) the interaction between difficulty and validity of the syllogism, (4) whether the correct response was in line with a rightist [0] or a leftist [1] ideology, (5) left-right political self-placement, (6) analytical thinking, and (7) the interaction between left-right self-placement and whether the correct response was aligned with leftist or rightist ideology that indicates the presence of ideological belief bias. All main results of these analyses are summarized in Table 4 and the interactions between left-right self-placement and leftist or rightist correct response are illustrated in Figure 4.
These results show that there were highly robust effects of ideological belief bias, even when adjusting for variations in the difficulty of the syllogisms as well as analytical thinking, for four out of eight political issues:   marketization of the school system, gender-neutral education, asylum to refugees, and climate change. Analytical thinking had independent effects on response accuracy only for the syllogisms on defense and asylum to refugees. Additionally, right (vs. left) self-placement independently predicted more accurate responses for the syllogism on asylum to refugees, but not other syllogisms, and syllogisms with leftist correct responses yielded higher accuracy when they addressed gender-neutral education, asylum to refugees, and climate change, but not other topics. Finally, it might be noted that the participants very robustly performed better on simple (affirmative) than difficult (negated) syllogisms and on valid than invalid syllogisms, and they consistently did best on valid affirmative syllogisms (i.e. Modus Ponens).

Discussion
The unfortunate tendency to engage in politically motivated reasoningthat is reasoning driven by a desire to reach a politically desirable rather than accurate conclusion-has been covered extensively in research in the last few decades (e.g. Pennycook & Rand, 2019;Taber & Lodge, 2012). This research is, however, plagued by a critical methodological issue in that the most common paradigms make it difficult to disentangle biased conclusions driven by motivated reasoning from rational conclusions based on prior beliefs (Tappin et al., 2020). The study of ideological belief bias in syllogistic reasoning is a promising new paradigm, because the participants evaluate the logical validity of syllogisms, which is independent of the truth of the premises and conclusions. In other words, prior beliefs are irrelevant to the correct response.

Contributions
The current research built on two recent studies that have provided evidence of ideological belief bias in syllogistic reasoning (Calvillo et al., 2020;Gampa et al., 2019). Drawing on the strengths of these previous studies while addressing their limitations, we made several new contributions. Among other things, we refined the paradigm introduced in previous research. The set of syllogisms the participants evaluated was balanced in terms of validity (valid or invalid), difficulty (affirmative or negated conclusion), and ideology (leftist or rightist conclusion) within each topic. This minimized this risk of confounds and enabled us to analyze ideological belief bias in syllogistic reasoning on all of the included political issues separately. We also included manipulation checks to make sure that leftist conclusions were indeed perceived as leftist and endorsed by leftist participants and vice versa. Perhaps most important, we had participants complete an extended training session designed to illuminate the distinction between validity and truth, prior to evaluating the syllogisms. This is a critical methodological element for making sure that effects of bias are disentangled from effects of prior belief. Furthermore, while previous research has investigated ideological belief bias only in the aggregate based on a handful of issues among liberals and conservatives in the context of the U.S. biparty system, we investigated its robustness across a diverse set of political issues (labor market, private health care, marketization of the school system, gender-neutral education, multiculturalism, military defense, asylum to refugees, and climate change) selected to be representative of the rich ideological landscape of the Swedish multiparty system (Nilsson et al., 2020). In addition, our sample of participants was nationally representative, whereas all previous studies except for one (Gampa et al., 2019; Study 3) have used convenience samples. This sample also enabled us to perform the first high-powered test of whether analytical thinking reduces (e.g. Toplak et al., 2011;Trippas et al., 2015) or strengthens (Kahan et al., 2017) belief bias specifically in syllogistic reasoning on ideological topics. Although Calvillo et al. (2020) addressed this issue and failed to find evidence that analytical thinking had any effect on ideological belief bias in syllogistic reasoning, our sample is more than twice the size of theirs and we used a statistical method with higher power.

Ideological belief bias
The results revealed ideological belief bias in both leftist and rightist participants, similar to previous studies (Calvillo et al., 2020;Gampa et al., 2019). Although leftist and rightist participants correctly evaluated roughly the same number of syllogisms in total (and did not differ in performance on the training session), we found interaction effects showing that leftist (vs. rightist) participants correctly evaluated more syllogisms when the conclusion was aligned with leftist ideology, fewer syllogisms when the conclusion was aligned with rightist ideology, and about equally many syllogisms with non-political content. This means that the participants did better when there was a match between the validity of the syllogism and the believability of the conclusion according to their ideological position (i.e. on valid syllogisms with believable conclusions or invalid syllogisms with unbelievable conclusions). The effects held up when we operationalized ideology in terms of self-placement and party preference. It should be noted, however, that the associations between participants' ability to accurately evaluate syllogisms and political orientation were, albeit significant, quite modest. The large number of participants in our study enabled us to obtain significant correlations even when these were in fact rather weak.
The results of analyses separated by political issue did suggest that ideological belief bias may vary according to the political issue. The clearest evidence of ideological bias emerged for syllogisms on the topic of providing asylum to unaccompanied refugees, and significant effects (although barely so) emerged also for syllogisms on marketization of the school system, sex differences and gender-neutral education, and climate change and air travel. There were no significant effects for syllogisms on the topics of labor market and state intervention, public or private health care, multiculturalism vs. tradition, and military defense.
One might speculate that these variations across political issues occurred because the former group of issues are hot-button issues, all of which have generated heated debate in Sweden. This observation is congruent with traditional dual-process accounts (e.g. De Neys, 2012;Kahneman, 2011), according to which motivated reasoning is driven by automatic intuitive reactions and affect-based heuristics. It may also be noted that three out of four issues that yielded significant belief bias effects (and only one that did not) pertain to the so called "GAL-TAN" (Green-Alternative-Libertarian vs. Traditional-Authoritarian-Nationalist) conflict, which political scientists have used to describe ideological clashes concerning lifestyle, ecology, cultural diversity, and immigration in European politics (Hooghe et al., 2002) rather than more traditional left-right issues; these dimensions can be detected in the ideological worldviews and moral intuitions of Swedish voters (Nilsson et al., 2020). At the same time, these results should be interpreted cautiously, because the analyses were exploratory rather than driven by hypotheses concerning what issues would yield effects, the statistical power is naturally lower when conducting analyses of single topics than aggregated across topics, and the variations across issues were generally not very large. Further research is needed to rigorously test hypotheses concerning differences across issues.
The role of analytical thinking Overall, our results are consistent with the suggestion that analytical thinking may confer an advantage in syllogistic reasoning. Although the significant correlations were modest in magnitude, all our results were in the direction of analytical thinking improving rather than reducing accuracy in the evaluation of the validity of the syllogisms, consistent with traditional dual-process accounts (e.g . De Neys, 2012;Kahneman, 2011), which suggests that analytical thinking improves accuracy by overriding affect-and heuristic-based reasoning that is prone to error. We found no evidence that the participants engaged in motivated type 2 reasoning, by using their analytical thinking to justify, rationalize, and defend their ideological convictions (Kahan et al., 2017). If anything, our results go in the opposite direction. At the same time, it could be argued that it may be more difficult for an analytically disposed person to find a way to justify an invalid (but ideology-congruent) argument than it is to justify evaluations of other kinds of information with less clear-cut quality, validity, or truth-in other words, motivated system 2 reasoning may be more likely to occur in other domains than syllogistic reasoning.
The specific findings concerning the role of analytical thinking revealed an interesting pattern. Analytical thinking predicted higher accuracy on political syllogisms for which the correct answer was aligned with left-wing ideology (and on non-political syllogisms) irrespective of the participant's ideology. But there was no significant interaction between analytical thinking and right (vs. left) self-placement, which means that analytical thinking did not reduce ideological bias for these syllogisms. For the syllogisms with correct responses in line with rightist ideology, on the other hand, there was a marginal interaction (but no main effect), such that analytical thinking improved accuracy among leftists but not among rightists and centrists. In other words, analytical thinking did mitigate ideological belief bias among both leftists and rightists-it improved accuracy specifically on valid syllogisms with unbelievable conclusions and invalid syllogisms with believable conclusions-but among leftists, it improved accuracy also for syllogisms in which validity and believability were matched. The interaction effect of ideological self-placement and analytical thinking was marginal even among rightists, so further research is needed to assess the robustness of this finding. It is, however, consistent with findings from other recent studies, including such as more analytical thinkers being better at evaluating politically laden information that goes against their partisan views (Pennycook & Rand, 2019), and that analytical thinking reduces politically motivated confirmation bias (Lind et al., 2018). Moreover, a recent preregistered replication of the seminal study by Kahan et al. (2017) found very weak support for the motivated system 2 reasoning hypothesis (Persson et al., 2021). There are also many studies documenting ideological asymmetries in analytical thinking and related constructs, including higher analytical thinking among social liberals than among social conservatives (Nilsson et al., 2019;Pennycook & Rand, 2019;Yilmaz & Saribay, 2018). Therefore, it is conceivable that right-wing conservatives benefit more from heightened analytical thinking when it comes to overriding ideological belief bias.

Limitations and future work
This research was not pre-registered. Although the hypotheses that were tested are well-established in the literature, the lack of pre-registration still entails a higher risk that confirmation bias may have affected the conclusions. Furthermore, it is important to bear in mind that particularly the results concerning analytical thinking were mixed, and just one out of several effects that were tested was clearly significant. More research is needed to conclusively determine the effect (or lack thereof) of analytical thinking on ideological belief bias and the differences (if any) between leftists and rightists in the role played by analytical thinking in syllogistic reasoning.
Although we tried to ensure that the conclusions of all our political syllogisms had the intended political directions, and that the difficulty for leftist and rightist syllogisms was equal, our participants had better accuracy when the correct response of the syllogisms was aligned with leftist ideology, and this was true particularly for three of the issues that generated ideological belief bias: climate change, asylum to refugees, and sex differences and gender-neutral education. Calvillo et al. (2020), who also found higher accuracy for leftist syllogisms, have speculated that this may be because leftist conclusions are more socially desirable (e.g. Streb et al., 2008) and social liberalism is associated with social desirability (Verhulst et al., 2012). This explanation fits with our findings insofar as leftist views on climate change, asylum to refugees, and sex differences are politically correct and could be disproportionately imbued with social desirability in the Swedish context. On the other hand, Gampa et al. (2019) found dissimilar evidence in one study, where participants exhibited worse accuracy for conservative arguments, accepting more conservative (than liberal) arguments as sound. Future work is needed to ensure not only that the conclusions of the syllogisms are in the intended political directions, but also that the syllogisms are balanced in terms of how emotion-laden, polarising, and socially desirable they are.
Future work could also include equal numbers of political and non-political syllogisms. Due to time-restraints, and the fact that a comparison between ability to solve non-political and political syllogism was not central to our hypothesis, we included only a minimum of syllogisms with non-political content. A more in-depth comparison of performance on syllogisms with political and non-political content could shed light on what might differentiate politically motivated reasoning from motivated reasoning in general.
The time-restraints may have caused additional effects on our results. We asked participants to try to not think too long before responding, which may have increased the tendency to go with their gut-feeling rather than attempt to respond at the height of their potential. Replications of the current paradigm with added incentives for correct responses would lend credence to the interpretation that participants' failure to accurately evaluate the syllogisms indeed is caused by biased reasoning, and not by a mere lack of motivation to be correct. Furthermore, although we included an extensive training session to make sure that the participants understood the concept of logical validity, we did not include a control group to examine the effectiveness of this training. An additional interesting extension of the current study would be to investigate the effect of training in logical reasoning to see if this is an effective tool against biased reasoning, and whether it could complement the strategies for "inoculation" that are currently developed to counter the spread of misinformation (e.g. Rozenbeek & van der Linden, 2019). Yet another possible extension would be to study the ability to evaluate logical validity of statements that are more similar to everyday conversations. Indeed, most laypeople rarely come across political arguments in the form of formally structured syllogisms. This means that the ecological validity of conclusions drawn in the current and past research programs on syllogistic reasoning may be limited.
Finally, researchers may want to address the relationship between ideological belief bias in syllogistic reasoning and politically motivated resistance to misinformation, such as "fake news" (Pennycook & Rand, 2019), and the mitigating effects of other relevant cognitive dispositions, including actively open-minded thinking (Stenhouse et al., 2018), and numeracy (Lind et al., 2018). Although distinct bodies of research on these different topics are evolving, we still have little in the way of an integrative account of the cognitive flaws and imperfections of individuals with different ideological convictions and how to counter them.

Disclosure statement
No potential conflict of interest was reported by the authors.