Effects of the valence and argument substantiveness of others’ comments on viewers’ validation of and attitudes toward pseudo-scientific claims

Abstract This study examines how the valence and argument of comments affect viewers’ attitudes toward and validation of a pseudo-scientific claim. We developed and tested a hypothesized model based on the elaboration likelihood model. Participants watched a video that introduced pseudo-scientific claims with others’ comments on the same screen. We assigned participants (n = 646) to a control condition with no message presentation and a message condition, with the message condition divided into four conditions based on a combination of valence and substantiveness of the comments. Structural equation modeling analysis revealed that valence affected both heuristic and systematic thought, while substantiveness influenced systematic thought. The negativity of the comments not only suppressed the positive impressions, which were irrelevant to the content, but also facilitated the examination of the reasoning for the pseudoscience claims in the video. Including substantive content in the comments also led to an examination of the rationale for pseudoscience claims. The model also showed that positive impressions irrelevant to the content increased validity judgments, the final positive attitude, and agreement to the pseudo-scientific claim, while examination of the rationale for the claims decreased them.


PUBLIC INTEREST STATEMENT
This study looked at how comments can affect people's perceptions of a pseudo-scientific claim presented in a video. We found that the tone of the comments, whether positive or negative, had an impact on how people processed the information. Negative comments led to more critical examination of the claim, and making negative impressions not directly related to the claim led to disagreeable attitudes. The study also found that substantive information in the comments helps critical examination. This research suggests that the comments section can have a significant impact on how people validate and accept the pseudoscience presented in a video.

Introduction
Despite science education being well-developed, pseudo-science is still widespread and the battle against it continues (Chavda et al., 2022;Ede, 2000;Mukenji & Ernst, 2022;Preece & Baxter, 2000). Believers of pseudo-science attempt to convince individuals of seemingly scientific fallacies through various media (Shermer, 2002). Video, in particular, has become a powerful, persuasive tool in both social media and mass media. In addition, the recent use of video often incorporates the opinions of others who watch it through comments (Barra & Scaglioni, 2014).
Research regarding the impact of comments presented alongside videos has focused on the heuristic influence process and less on the systematic process. While previous studies have demonstrated that the valence of viewers' comments affects post-viewing attitudes toward the topic (Cameron & Geidner, 2014;Maruyama et al., 2017) and program evaluation (Waddell & Bailey, 2019;Waddell & Sundar, 2020), they often lack a measurement that distinguishes heuristic thinking from systematic thinking. It is likely that when watching a pseudo-scientific claim, viewers experience systematic thinking, such as making inferences and reasoning. How do others' comments impact viewers' heuristic and systematic processing?
The second issue we would like to address is how the nature of others' comments affects attitudes. Areni and Lutz (1988) highlighted that the nature of persuasive messages can be decomposed into valence and argument strength. Relatively little research has examined the nature of comments other than valence, but it has been shown that the substantive nature of comments facilitates positive attitudes toward topics (Kim & Gonzalez, 2022;Weber et al., 2019). Although it is not clearly discussed in past studies, substantiveness should facilitate systematic thinking about particular content.
To answer these questions, it is necessary to conduct experiments under a unified model, controlling for the valence and substantiveness of comments, since some studies have suggested that there is a relationship between valence and comment content (Lee & McEroy, 2019). We, thus, adopted the framework of the elaboration likelihood model (ELM; Petty & Cacioppo, 1986). In the following sections, we review the relevant literature on how comment valence and argument affect attitude and validation.

Literature Review
A model regarding the influence of others' comments. The ELM (Petty & Cacioppo, 1986;Petty et al., 2015), as well as other similar dual-route processing models, proposes a cognitive process under persuasion through two different routes: the central and peripheral routes (see, Booth-Butterfield & Welbourne, 2002;Todorov et al., 2002). Attitude formation is influenced by a combination of central and peripheral routes of processing. The central route represents systematic thinking, in which deliberate and careful considerations are made. Elaboration related to the claims and critical examination of rationales are examples of the central route process.
The ELM has been widely applied and validated in the examination of attitudes and decisionmaking, including recent studies on marketing communication (Cyr et al., 2018;Townsend & Kahn, 2014;Shahab et al., 2021 as a review) and misinformation (Chen et al., 2021;Sui & Zhang, 2021). Sui and Zhang (2021) revealed that both information quality and source credibility enhance one's belief in the information. Alongside the interaction between participants' prior attitude and the nature of information, Sui and Zhang (2021) suggested how central and peripheral routes of processing are involved in the evaluation of misinformation.
Although the model as a whole has not been examined, the ELM has been introduced in several studies (Kim & Gonzalez, 2022;Kümpel & Unkel, 2020;Maruyama et al., 2017;Waddell, 2020) to explain the effects of others' comments on attitude. Previous studies have often focused on the social nature of others' comments that affects the peripheral route of processing (Cameron & Geidner, 2014;Sundar et al., 2015;Waddell & Bailey, 2019;Waddell & Sundar, 2017). A small but significant body of research has suggested that the argument-related nature of comments also impacts individuals' validation and attitude via the central route of processing (Kim & Gonzalez, 2022;Weber et al., 2019). Thus, we focus on valence as the nature of the comments affecting the peripheral route, and on the argument's substantiveness as that impacting the central route. In the following section, we present related studies and state our hypotheses.
The effect of valence. Regarding the nature of comments, we first focus on their valence: positive valence, which agrees with the claim that is the subject of the comment, and negative valence, which denies or criticizes it. Valence is indicated by direct statements of liking or disliking (e.g., "I hate it"), expressions of agreement (e.g., "That makes sense"), or questioning (e.g., "Can that be true?").
Accumulated studies have shown that the valence of comments influences the affective and cognitive aspects of the people exposed to it. In domains where preference serves as the criterion, for example, entertainment, negative comments have been shown to lead to lower evaluations of program quality and ratings of performers (Cameron & Geidner, 2014;Waddell & Bailey, 2019;Waddell & Sundar, 2017). Studies have also suggested that valence influences cognitive aspects, such as the evaluation and validation of the content (Inuzuka et al., 2018(Inuzuka et al., , 2019Maruyama et al., 2017).
Studies in other related areas have also demonstrated that comment valence influences attitude formation and validity judgments. For example, negative comments regarding online news have been shown to decrease the importance and credibility of the news (Go et al., 2014;Kümpel & Unkel, 2020;Waddell, 2020;Waddell & Sundar, 2017;Weber et al., 2019). Electronic word-ofmouth studies have shown that positive comments increase positive attitudes and purchase intentions, while negative comments decrease them (Ahluwalia, 2002;East et al., 2008). In general, the influence of negative comments is likely to be found, but their dominance is considered to be lower when people are familiar with the subject (Ahluwalia, 2002) or when they are not highly motivated to examine the issue (Meyerowitz & Chaiken, 1987;Shen & Dillard, 2009;Wright, 1981).
However, many studies have focused on the association between comment manipulation and final attitudes, although it remains unclear how the processing of the central and peripheral routes is involved in the impact of comment valence. The first explanation includes the perspective of peripheral routes and describes the impact of comment valence in terms of conformity or the bandwagon effect (Cameron & Geidner, 2014;Sundar, 2008;Sundar et al., 2015;Waddell, 2020;Waddell & Sundar, 2020). Conformity and the bandwagon effect are both examples of social influence. Both involve individuals aligning their behavior, attitudes, and beliefs with those of a group. Conformity refers to the general tendency for individuals to conform to group in order to form accurate interpretation of reality or to obtain social approval from others (Cialdini, 2001;Cialdini & Goldstein, 2004). The bandwagon effect refers to individuals adopting certain beliefs or behaviors because they believe that many others are doing so (Sundar, 2008). In other words, while both conformity and the bandwagon effect involve social influence, the bandwagon effect is a specific type of social influence characterized by a perceived sense of popularity or majority support.
A relatively small number of comments could be recognized as representing the majority of people (Zerback & Fawzi, 2017), thus evoking the bandwagon effect, a heuristic process to follow them (Sundar et al., 2015). Since it is the heuristic to follow the majority and is preferred in situations wherein the topic is of little relevance to the person, who holds no strong prior attitude (Erb et al., 1998), the bandwagon effect is a viable explanation for the affective and cognitive processes of those who watch TV or read online news.
According to the bandwagon effect for comment valence, it is expected that the latter has an impact on impression formation irrelevant to the claim itself. This is because the processing of peripheral routes is an intuitive process that is not related to the content and is, thus, thought to cause the formation of good or bad impressions about factors other than content. Therefore, we hypothesize the following: (H1) The valence of others' comments affects the peripheral route of processing. Positive comments will induce a better impression that is irrelevant to the claim of the video, whereas negative comments will cause a worse impression.
Although not fully explored in studies on the impact of comments, the effects of valence may also be found in the central route of thinking. Related research on message framing effects suggested that negative information is perceived as more diagnostic for evaluating a target (Herr et al., 1991). Thus, negative valence is thought to attract attention, which is observed as an effect of affective valence. It is important to note that the influence of perceived diagnosticity is observed under conditions of high information elaboration (Ahluwalia, 2002;Meyerowitz & Chaiken, 1987;Shen & Dillard, 2009;Wright, 1981) and especially with unfamiliar targets (Ahluwalia, 2002). Ahluwalia (2002) revealed that when presenting a negative article with unfamiliar targets, readers perceived the article as more relevant and useful, and generated more inference in line with the article than they did when presented with a positive article. Hence, negative valence is recognized as a cue for diagnostic information and promote inferences process based on the information.
The primary implication from these studies is that valence of others' comments would also be affects central route of processing of pseudoscientific claims. The situation in which the validity of pseudoscientific claims is considered involves an active evaluation situation. If comments indicating negative values are presented, people will judge that the information is useful for validity judgments and will actively make inferences based on the information. On the other hand, positive valence is perceived as less diagnostic, there will be less processing through the central route. Thus, we examine the following hypothesis: (H2): The valence of others' comments affects the central route of processing. The negativity of the comments facilitates systematic thinking processes; viewers tend to reason and make inferences more when provided with the negative comments of other viewers than when provided with positive ones.
The effects of substantive arguments. The second aspect of the nature of comments upon which we focus is their substantiveness. Areni and Lutz (1988) discussed that the argument quality of persuasive messages consists of valence and argument strength. According to Chaiken et al. (1989), strong arguments that are well-reasoned and logically consistent can be more persuasive than poorly formed, inconsistent ones. Including evidence and other substantive information regarding the claim makes the arguments stronger than those that rely on personal opinions and other unsubstantive information (Wright, 1981;Carpenter, 2015).
Prior studies have also suggested that the nature of arguments contained in comments affects systematic processing (Kim & Gonzalez, 2022;Weber et al., 2019). Kim and Gonzalez (2022) focused on the strength of arguments: Comments that make rational points, provide factual evidence, or raise legitimate questions were considered strong arguments. In their experiment, participants generated elaborate thoughts when positive cues and substantive arguments were present in comments. The results suggested that substantive comments that include reasons and facts facilitate viewers' systematic thought, which comprises the central route of processing. Thus, we hypothesize: (H3) The substantiveness of a comment's arguments affects the central route of processing: Comments that include reasons and facts facilitate viewers' elaboration of the content.
Several previous studies have focused on the valence and argument of others' comments, but with limitations. The first of these limitations is that only a few studies have investigated the content of viewers' systematic thought. For example, it is reasonable to examine what viewers think through a posteriori explanations. However, to our knowledge, only few studies (e.g., Kim & Gonzalez, 2022) have adopted such an approach. Second, prior studies have focused on either the argument or valence of comments and have not systematically manipulated both. Considering that a correlation exists between valence and the information contained in an actual social media post (Lee & McEroy, 2019), studies focusing on only one of these factors have not been able to rule out the influence of the other. Third, studies have not demonstrated the impact of comments on posterior attitudes and validation in a comprehensive model. Although the ELM and other models are cited in prior studies, they have examined the relationship between comment features and posterior attitudes and validation; the involvement of the central and peripheral routes that link the two has not been adequately explored.
Individual differences and attitudes toward pseudo-scientific claims. The ELM also predicts that individual differences affect the processing of information. The central route of processing requires time and cognitive resources, and this elaboration is unlikely to occur when interest, knowledge, or motivation is low. On the other hand, the peripheral route is a quick process that requires less cognitive load, being more influential when motivation is low (Petty & Cacioppo, 1986). Thus, we measured individual differences, namely interest, previous knowledge, and prior attitude toward the topic, and employed them as control variables.
We further included deliberative cognitive style as an individual difference variable. Frederick (2005) proposed the Cognitive Reflection Test (CRT) based on the dual process theory to measure individual differences in the tendency to execute systematic thinking. The CRT consists of simple quantitative questions that are designed to be fallible when answered with an intuitive, peripheral route, and lead to correct answers when using deliberation via the central route. Frederick (2005) found that CRT scores were highly correlated with other tests of analytic thinking. Toplak et al. (2011) also demonstrated that a higher CRT score predicts success in the task where heuristic thinking leads to failure, and success in a rational-thinking task that requires systematic thinking. Toplak et al. (2011) concluded that the CRT is a potent measure of individual differences in cognitive style, particularly regarding the tendency to engage in systematic thinking without performing heuristic thinking. Studies showed that CRT predicts the various tasks that require systematic thinking (Brañas-Garza et al., 2019), such as conjunction fallacy and conservatism in updating probabilities (Oechssler et al., 2009), decision-making (Campitelli & Labollita, 2010;Moritz et al., 2013), and strategic behavior (Brañas-Garza et al., 2012). As the CRT has been proven to measure individual differences in systematic thinking tendencies in various tasks, we consider it appropriate to be included in the present study.

Study AIM and hypotheses
The present study investigated the effects of others' comments introduced into a program that featured pseudo-scientific claims. What role do the different natures of others' comments play in situations requiring deliberation? To answer this question, we examined the model shown in Figure 1 using structural equation modeling (SEM). Our model depicted that the nature of comments, that is, their valence and argument substantiveness, affect the two processing routes that determine one's attitude, validation of a claim, and decision on whether to support the claim. In the model, we predicted that valence affects the peripheral (H1) and central route (H2), and argument substantiveness affects the central route of processing (H3). The significance of this study is that it examines these relationships as a single model by conducting SEM to present an integrated picture.

Open Practices and Ethical Statement
The materials and the data included in the analysis are available at OSF. The research protocol was deemed exempt by the Institutional Review Board (IRB) of Tokyo Gakugei University (approval number 502-2).

Participants
Participants were recruited from Cross Marketing Inc. Panels. Participants received a monetary honorarium determined by the panel's criteria for participating. The terms and conditions for participation were explained at the beginning of the survey page, and participants' consent was confirmed by clicking the "I Agree" button. The survey was conducted as part of a larger online survey over two days. A total of 1,298 participants accessed the survey site, and 812 participated in the two-day study and completed all the survey items. The basic demographic information of the participants is presented in Table 1. Most participants were in their 40s or older, and there were more males. More than half of the participants had a college degree.

Video introducing pseudo-science, "Effective Microorganisms."
We developed a video (approximately eight minutes long) that was modeled after a news show in which an actor played a college professor to illustrate the effects of pseudo-science. The video consisted primarily of verbal explanations, had few complex visual images, and was relatively simple. The video material was initially used in a study by Inuzuka et al. (2018). We edited the original video to keep the program's introduction and explanation of pseudo-science.  A pseudo-science topic, namely "Effective Microorganisms," was chosen for our program. It is a typical topic of pseudo-science and is based on the idea that a particular collection of microorganisms can solve virtually all health and environmental problems. Although this topic is wellknown to those interested in pseudo-science, few students are familiar with it.

Fake social media comments
We developed fake social media comments that simulated Twitter posts to present others' opinions on the video material ( Figure 2).
The total number of comments presented was 29. The first three comments were common to all comment conditions, and three others were divided into two types depending on the valence condition. The remaining 23 comments presented four different types of content based on valence and argument substantiveness ( Table 2). Each comment comprised one or two short sentences. Valence in the comment represented agreement or disagreement with the program's assertion (pseudo-science), and argument substantiveness represented whether the comment contained substantive information, such as suggestions of reasons or evidence to agree/disagree or questionbased arguments. Hence, a positive substantive comment expressed agreement with the claim advocating pseudo-science and indicated that one can infer that essential reasoning was present in the video. On the other hand, a negative substantive comment expressed disagreement with the claim, suggesting that the evidence presented was inadequate or incoherent. An unsubstantive comment did not include information about rationale or consistency and presented only emotional impressions.

Attention check
As the study was conducted online, we needed to ensure that the participants had been watching the screen during the video streaming. To exclude invalid data, a test with six true-false quiz items (e.g., "In the video, it was explained that there are no examples of effective microorganisms being used overseas") was developed to ensure that the participants actually watched the video and grasped its content.

Irrelevant impression questionnaire
We developed six questionnaire items to measure the impression of events or the person (e.g., "The explainer seemed earnest.") that were not essentially concerned with the program's claims. Participants rated the items on five-point Likert scales (1 = Not at all to 5 = Very much). A higher score represented a more positive impression.

Degree of agreement and free-text answer for the reasons for agreement
Participants answered a single seven-point-scaled question regarding to what extent they agreed with the use of the pseudo-scientific techniques presented in the program (1 = Strongly disagree to 7 = Strongly agree). After rating, they explained the reasons why they agreed or disagreed with the claim in short sentences. These reasons for agreeing/disagreeing were used as an indicator to reflect their elaboration on the content of the videos.

Attitude questionnaire
A questionnaire with three items asking about participants' impressions of the effectiveness of "effective microorganisms" (e.g., "I think effective microorganisms will somehow do some good") was administered. Participants rated their agreement with the items on five-point Likert scales (1 = Not at all to 5 = Very much).

Validation questionnaire
Participants answered questionnaire items that assessed the degree to which they believed the video claims were valid. The questionnaire included four items (e.g., "There is sufficient evidence to believe that 'effective microorganisms' are effective in purifying river water"), all of which involved agreement to the statements on five-point Likert scales (1 = Not at all to 5 = Very much).

Index for interest, knowledge, and familiarity
We measured the participants' general interest and knowledge about environmental issues with a single question item, with one item for interest and knowledge. We also asked a single question about the participants' level of familiarity with "effective microorganisms." The participants answered those questions on a five-point Likert scale, with higher scores reflecting more knowledge, interest, and familiarity.

Cognitive Reflection Test
Three original three test items presented in Frederick (2005) were administered to measure the individual difference in cognitive style. The questions on the CRT (e.g., A bat and a ball cost $1.10. The bat costs $1.00 more than the ball. How much does the ball cost?) were meant to be misleading when answered heuristically (10 cents), but with deliberate thought, the correct response (5 cents) could be obtained. We modified words in an item to make it sound more realistic (a bat and ball were modified to a candy and gum).

Procedure
Each experiment was conducted online. After giving informed consent, the participants proceeded to the questionnaire form. They first answered their age group and education level, then answered the index of interest and knowledge. Subsequently, the participants answered the attitude questionnaire before they proceeded to watch the video explaining pseudo-science. Each participant was randomly assigned to one of five conditions. In the comment conditions, namely negative substantive (NS), negative unsubstantive (NU), positive substantive (PS), and positive unsubstantive (PU), the video was presented with others' comments based on the abovementioned four options. In the control condition (C), the video was introduced without any comments on the screen. No instruction was given regarding the inclusion of others' comments. After watching the video, participants answered the attention check items, attitude questionnaires, and questionnaires regarding their impressions of the non-content event and person. The participants then evaluated their degree of agreement and described their reason for agreement/disagreement. There was no time limit, but most participants completed the tasks within 15 minutes, with an average of 13.7 minutes (SD = 3.08). A week later, participants returned another survey questionnaire, in which they answered the CRT.

Data selection
Of the 812 participants who completed the survey, the data from 646 participants were included in the analysis. Data from 33 participants were excluded because they did not peruse adequate time to read and comprehend the instructions and items, as they were instructed to watch the video for eight minutes. Additionally, the data from participants who failed to answer more than three items of the attention check quiz (n = 133) were excluded.

Scoring
Cronbach's alpha coefficient was calculated for each scale of the questionnaire items. We calculated the mean of the questionnaire items for each subscale and used them as the scale scores for subsequent analysis (Table 3). For the CRT, correct answers were scored as one point, incorrect answers as 0 points, and the total score was used for analysis.
The free statements on participants' reasons for agreement/disagreement with the claim (the pseudo-science) were coded from four standpoints: rationale, acceptance, paraphrasing, and associative inference. Rationale was judged on the presence or absence of reasoning that led directly to the grounds for disagreeing/agreeing. We coded for the presence of statements that raised questions or problems with the pseudo-scientific claims (e.g., not enough experiments have been done) or statements of reasoning that assumed the claim was correct (e.g., produces a clean environment without harmful by-products). Acceptance was determined by checking whether there were statements that agreed with the claim (e.g., "Effective microorganisms will clean the river") without any examination. Paraphrasing was coded as 1 if there was a statement that summarized or restated the claim in participants' own words. Associative inference was coded as 1 if there was an associative statement that was not directly related to the claim (e.g., ocean pollution was discussed during the Tokyo Olympics). The first author and an independent rater, who was unaware of this study's hypotheses, coded the free statement as 1 if the statement corresponded to each of the standpoints and 0 if it did not (Table 4). The inter-rater agreement was .86 to .91. For the statements with inconsistent ratings, the other raters reviewed the responses and decided on a rating.

The hypothesized dual-route model of the influence of others' comments
We constructed a model using measurement variables and tested it with the SEM technique to test the hypotheses. The experimental condition was dummy-coded for manipulation of the argument structure. Valence was coded into three categories indicating if the comment included a supportive attitude to the claims made in the video. Thus, the comments incorporated in the positive condition were coded as 1 and those in the negative condition were coded as-1. Since the control condition did not involve the support or rejection of the other viewer, we coded comments contained in the control condition as 0. The substantiveness of the comments was coded depending on whether there was additional information regarding the video's claims. The comments included in the substantive condition provided such information; thus, they were coded as 1. While the unsubstantive condition included others' comments, they provided no additional information concerning the claims; thus, we coded the substantiveness of the comments as 0. Scale scores for irrelevant impressions were considered to reflect peripheral route processing. The coding of participants' statements about their reasons for agreeing/disagreeing with the pseudoscientific claims was also included in the model by using categorical variables (1, 0), considered as an indicator reflecting central route processing. Although we predicted that the argument would not affect the peripheral route, we considered a path for irrelevant impressions to test the prediction. Prior attitudes toward pseudo-science, general interest and knowledge, familiarity, and CRT scores were assumed as factors that directly and indirectly influenced posterior validation, attitude, and agreement with the claim. We also added the test score since it may predict these variables as well.
We used the Mplus software for analysis and employed robust weighted least squares (WLSMV) for estimation since the rationale statement involved categorical data. Model fit was assessed using the absolute fit index (SRMR), comparative fit index (CFI), and root mean square error of approximation (RMSEA). Although the goodness-of-fit index criteria vary widely from study to study, a good fit is considered to be .95 or higher for CFI and .05 or less for RMSEA and SRMR. A CFI of .90 or higher and an RMSEA and SRMR of .08 or lower are considered to be acceptable models (Kyndt & Onghena, 2014).
Analysis was performed on the model with all variables measured. Seventeen data points containing more than one missing value were excluded. The model did not show an acceptable fit with SRMR = .099, CFI = .817, and RMSEA = .154. We eliminated the variables with low correlations to other variables from the model to increase the model fit. Demographic variables, namely age, gender, and education level, were removed from the model. Since these modifications did not improve the model enough, we redefined the model using only the coding of rationale as the index of central route processing; the coding of acceptance, paraphrasing, and associative inference was removed from the model. As a result, the CFI and SRMR improved to an acceptable level (SRMR = .077, CFI = .938), although the RMSEA was slightly higher than the recommended level (RMSEA = .089).
The results of the analysis are shown in Table 5 and summarized in Figure 3 for significant paths. The valence of others' comments affected both the index of the peripheral route-the irrelevant impressions-and the index of the central route-the description of the rationale in the free-text statements. When there were comments with positive valence, the likability of the characters and overall impression were more favorable, and the descriptions of the rationale were less frequent. On the other hand, when there were comments with negative valence, irrelevant impressions tended to be unfavorable, and descriptions of the evidence tended to be more frequent. The paths from the argument of the comments were only significant for the description of the rationale for agreeing/disagreeing in the free response. The substantive argument would facilitate statements of these rationales but did not affect impressions that were unrelated to the content. The indices of both routes had a significant effect on the validation, attitude, and agreement with pseudoscientific claims among the participants. Higher scores on impressions that were not related to the content of the claims were found to enhance the validation of pseudoscientific claims as being more plausible and evoke more positive attitudes towards the claims. This also corresponded with a higher level of agreement with the claims. Conversely, the presence of rationale statements made by the participants was associated with a reduction in validation of pseudo-scientific beliefs, a more negative attitude, and reduced agreement. Table 4 shows that the effect of valence was not symmetrical and that the positive condition differed very little from the control condition compared to the negative one. To further analyze the above impression, we conducted a one-way ANOVA on the posterior attitude and validation score, with the condition as a between-subject factor. The result of posterior attitude reached a significant level (F (4,640) = 8.53, p < .001). Multiple comparisons showed a significant difference between the two negative conditions (NS, NU) and the other three (PS, PU, C), with small to medium effect sizes (ds = 0.34-0.60).

Additional Analysis
The result of validation also reached a significant level (F (4,640) = 7.03, p < .001). Multiple comparisons showed a significant difference between the two negative conditions (NS, NU) and the other three conditions (PS, PU, C). The effect size of the difference among the conditions was small to medium (ds = 0.35-0.47). The ANOVA result was also significant for the agreement score (F (4,640) = 5.01, p < .001). The multiple comparison test (Holm) revealed significant differences between the NU and PS (p < .001, d = 0.52) and NS and PS (p < .05, d = 0.41) conditions.

Discussion
The current study investigated the effects of presenting others' comments on the validation of and attitude toward a pseudo-scientific claim introduced in a video. The hypothesized model we developed based on the ELM (Petty & Cacioppo, 1986) was supported by the data, which showed significant paths for posterior attitude, validation, and agreement from the measurement that reflected the central and peripheral routes of processing, namely the rationale stated by the participants for agreeing/disagreeing with the claim and impressions irrelevant to the claim, respectively. The results supported Hypotheses 1 and 2, demonstrating a significant path from valence to irrelevant impressions and to participants' statements of critical rationale. These results indicate that valence affects both the peripheral and the central route to processing. Hypothesis 3 regarding the argument substantiveness was also supported by the results that showed a significant path only for the participants' rationale statements.
The decrease in the model's goodness of fit when using three categories of participant statements (acceptance, paraphrasing, and associative reasoning) suggests that appropriate critical thinking or considering the rationale of the claim is a crucial aspect of central processing in the present model regarding pseudoscience. Since the category of rationale clearly reflected the participants' critical thinking in the central route of processing, it highlighted the relationship between the nature of comments, critical thinking, and the validation, attitudes, and agreement with the claim.
Although the other categories removed from the model should be considered as the central route of processing, they do not necessarily reflect critical thinking about the claim. The acceptance statements could have been made with or without deliberate thought of the rationale behind the claim. Similarly, the paraphrase category reflects the participants' engagement. However, it may also be the result of memorization without considering the pseudoscientific feature of the claim. The category of associative inference included an elaboration of content and should be distinguished from the peripheral route of processing. However, it reflected the result of thinking that was not directed toward the consideration of validating the claim. In conclusion, this study highlights the importance of appropriate critical thinking in the central processing route for the validation and attitude formation of a model regarding pseudoscience.

The effects of the valence and argument substantiveness of others' comments
The primary contribution of this study is that it demonstrated the pathways through which the valence and substantiveness of comments influence attitudes and validation via the peripheral and central routes. Unlike most previous studies, which have focused on the heuristic influence of comment valence (Cameron & Geidner, 2014;Sundar, 2008;Sundar et al., 2015;Waddell, 2020;Waddell & Sundar, 2020), this study employed measures that reflect central and peripheral route processing to illustrate the process by which valence and substantiveness influence final attitude formation and validation. When the valence of the comment presented in the video was negative, the participants' statements of their reasons for agreeing/disagreeing with the pseudo-scientific claim increased in terms of critical rationale, that is, statements about inadequacies and questions regarding the content presented in the video increased. These results supported our hypothesis, showing that the negativity of the comments may have acted as a sign that the information was diagnostic. Comment negativity may, thus, have drawn attention to the problems and shortcomings of the presented claims, thereby facilitating critical elaboration.
The results were consistent with those of previous studies that have shown that valence also affects the heuristic processes. The valence creates a bandwagon effect, suggesting that other viewers have a favorable/unfavorable impression (Sundar et al., 2015). Thus, the results of this study provide new evidence that valence affects both peripheral and central routes. Negativity would facilitate a critical examination of the content, while simultaneously forming bad impressions that are independent of the content.
Another significant finding was that substantive comments facilitated participants' critical elaboration. Kim and Gonzalez (2022) also employed a dual-route model and indicated that content-related thinking is facilitated when strong arguments are included in others' opinions, which is consistent with the results of the present study. However, Kim and Gonzalez (2022) did not include the variable of whether the comments themselves agreed with the target news, and the effect of comment valence was not examined. The present study is significant in that it employed a single model to focus on two essential features of comments, namely valence and substantiveness, and examined how they are affected when each is controlled for.
The practical implication of the present study is that negative and/or substantive comments can help us to reject pseudo-scientific claims. These types of claims can be difficult to dismiss because they are often presented in a way that appears scientific and convincing. In this study too, the attitude toward pseudoscience changed in a more positive direction after viewing the video for the group that was not presented with comments and the group that was presented with positive comments, suggesting that even highly problematic claims are difficult to deny when they are developed in a seemingly plausible manner. Moreover, it is not easy for us to alter the representation once constructed as truth, even when a fault is apparent (Gilbert et al., 1993;Rapp, 2008), so we may need further aid in constructing appropriate representations. The present study suggests that both valence and argument are key features of others' opinions that support viewers' systematic thinking, leading to the rejection of illogical claims. It is particularly important to note that the negative valence and substantiveness of the comments, respectively, were shown to promote appropriate critical thinking of the viewer. Negative valence does not merely promote heuristic thinking in the peripheral route of processing, but can also work on systematic thinking in the central route of processing.
The methodological implication of this study is that we need to control for both valence and the quality of the argument when examining the impact of others' comments. We suggest that argument substantiveness would reconcile the mixed results regarding valence and attitude. For example, Maruyama et al. (2017) observed that participants were more deliberate when presented with others' positive comments, which appears to contradict the current study's results. The seemingly different results may be because valence and argument were not controlled for in the comments presented in Maruyama et al. (2017); the positive comments presented in that study contained stronger arguments than the negative comments, which may have facilitated elaboration. Namely, if the valence and substantivity of comments were separately controlled for, as we did in this study, Maruyama et al. (2017) may have found the same effects of negative comments and the facilitation of elaboration by substantive information as in other previous studies. Conducting a study that controls for the valence and essentiality of comments would help to properly identify the impact of each.

Negativity bias in the impact of comments
A few studies have revealed that the difference in attitude is only significant between control and negative comments, while positive comments showed little effect on attitude (Cameron & Geidner, 2014;Inuzuka et al., 2019;Waddell, 2020;Waddell & Bailey, 2019;Waddell & Sundar, 2017). Although the effect sizes were small to medium, the additional analysis of our data supported the negativity bias, showing a significant difference in attitude and validation between the negative and control conditions, while the difference between the positive and control conditions was not significant.
The difference in impact between positive and negative valence is often explained by the saliency and dominance of negative information over positive information (Baumeister et al., 2001;Rozin & Royzman, 2001). As Baumeister et al. (2001) suggested, this seems to be a principle that runs through various psychological phenomena: "Bad is stronger than good." Negative information is judged to be more diagnostic than positive information, especially when the situation requires evaluation (Skowronski & Carlston, 1989) or the person is more strongly motivated (Smith et al., 2006). As shown in the additional analysis, when we examined the impact of negativity on post-viewing attitudes and validity judgments, pooling factors such as interest and familiarity, we observed a negative bias. Negativity may, thus, have been a more diagnostic information cue for the participants in the experiment.
The negative bias observed in previous studies and in everyday life may reflect not only the effect of valence but also the substantiveness of the argument in the comment. Lee and McEroy (2019) held that there is a correlation between the valence and the substantiveness of comments. While facts and sources were found to be more likely in negative comments, opinion-based sentences appeared more in positive comments (Lee & McEroy, 2019). Thus, at least in some cases, the negative comments impacted validation and attitude not because they were negative, but because they contained strong arguments. Exposure to substantive arguments that present reasons for opposing the claims presented facilitates critical elaboration, leading to a low validation and negative attitude.

Limitations and Future Directions
One limitation of our study involved the measurement of elaborative thought that the participants engaged in while watching the video. As an index reflecting thought in the central and peripheral routes, it is desirable to measure elaborative thought while the video is being viewed. In this study, we asked participants to respond to a question item a posteriori, as an indicator corresponding to the processing of the peripheral route, while describing their reasons for agreeing or disagreeing with the claims as an indicator of elaboration in the central route. These procedures were adopted because measuring an index that reflected differences in the quality of thinking in an online experiment was difficult. Since previous studies (Kim & Gonzalez, 2022;Maruyama et al., 2017) have also employed posterior measurement as an index that reflects processing in the peripheral and central routes, we followed suit in this study. However, future research should consider using an index that reflects the participants' thinking process more accurately, such as asking them to think about speech while processing the material.
Another limitation of the study was the lack of control over the experimental environment. As the study was conducted online, both exogeneous and endogenous factors were not fully controlled. Although the impacts of these factors should have been randomized, the attention paid to the comments may explain the residuals of the model. Further research should consider using measures to assess differences in attention among the participants.
We also would like to point out that the results of the present study should be interpreted in the context wherein the claim is invalid. Whether the rejection of a claim is socially ideal depends on its quality. Since this study used a video that made pseudo-scientific claims, we can conclude that the presence of negative valence and substantive arguments was effective in changing attitudes in the ideal direction. Future studies may explore the role of the valence and argument of others' comments on the acceptance/rejection of science, technology, or medicine for which it would be socially desirable to accept the claim.

Conclusion
The present study examined the effects of presenting other viewers' comments when watching videos that made pseudo-scientific claims. The study untangled the nature of the comments and showed the process of how valence and substantiveness impact attitudes and validity judgments based on a single model. We empirically demonstrated that when exposed to seemingly cogent arguments, the presentation of others' critical opinions containing essential information allows viewers to perceive them as diagnostic messages and to respond more cautiously.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Data Availability Statement
The data and the materials that support the findings of this study are openly available in Open Science Framework (OSF) at https://doi.org/10.17605/OSF.IO/ PWEV8.

Citation information
Cite this article as: Effects of the valence and argument substantiveness of others' comments on viewers' validation of and attitudes toward pseudo-scientific claims, Miwa Inuzuka, Yuko Tanaka & Kazunori Fujimoto, Cogent Social Sciences (2023), 9: 2185287.