The Anatomy of Conspirators: Unveiling Traits using a Comprehensive Twitter Dataset

The discourse around conspiracy theories is currently thriving amidst the rampant misinformation in online environments. Research in this field has been focused on detecting conspiracy theories on social media, often relying on limited datasets. In this study, we present a novel methodology for constructing a Twitter dataset that encompasses accounts engaged in conspiracy-related activities throughout the year 2022. Our approach centers on data collection that is independent of specific conspiracy theories and information operations. Additionally, our dataset includes a control group comprising randomly selected users who can be fairly compared to the individuals involved in conspiracy activities. This comprehensive collection effort yielded a total of 15K accounts and 37M tweets extracted from their timelines. We conduct a comparative analysis of the two groups across three dimensions: topics, profiles, and behavioral characteristics. The results indicate that conspiracy and control users exhibit similarity in terms of their profile metadata characteristics. However, they diverge significantly in terms of behavior and activity, particularly regarding the discussed topics, the terminology used, and their stance on trending subjects. In addition, we find no significant disparity in the presence of bot users between the two groups. Finally, we develop a classifier to identify conspiracy users using features borrowed from bot, troll and linguistic literature. The results demonstrate a high accuracy level (with an F1 score of 0.94), enabling us to uncover the most discriminating features associated with conspiracy-related accounts.


Introduction
Conspiracy culture has been brewing on social media for over a decade, fueled by the misinformation, polarization, and science denial typical of the online ecosystem [1,2,3,4].Conspiracy theories provide alternative explanations to significant historical or current events with claims of secret plots by people or groups having ambigous intentions (e.g., usurpation of power, violation of rights, alteration of the bedrock institutions, societal disruption, etc.) [5,6,7].Social media platforms have enabled faster communication and dissemination of conspiratorial narratives.As such, recent times have seen a plethora of online conspiracy beliefs concerning a broad range of topics.Notable examples encompass unconventional interpretations of climate change [8], the 9/11 attacks [9], political movements like QAnon [10], and, more recently, theories related to the COVID-19 pandemic [11,12].As a result, the spread of such information can have far-reaching implications for both individual users and society at large [13,14,15,16,17].For these reasons, research into online conspiracy has grown in recent years, aimed at comprehending the dynamics of online conspiracy culture across various academic disciplines by using models and analytical approaches primarily based on linguistic and rhetorical theory [7].Understanding users' inclinations towards conspiracy theories is of significant interest, as it can offer valuable insights into the propagation of ideologies, without limiting the analysis to a specific conspiracy theory.This understanding is crucial for assessing the roles played by the involved individuals and taking appropriate measures to mitigate the impact of this phenomenon.Nonetheless, despite research advances in the detection of emerging or predefined conspiracy theories [18,19,20], the study of conspiracy users' characteristics remains limited.In fact, most of the existing datasets and collection techniques are either focused on specific conspiracy topics or domains, or rely on manual annotation or subjective criteria to label users as conspiratorial or not.This limitation poses challenges for developing and evaluating automated methods for user-level conspiracy detection and analysis.Therefore, there is a need for a large-scale, diverse, and reliable dataset that can capture the general characteristics and behaviours of conspiracy users across different topics.Such a dataset would also enable researchers to explore various aspects of online conspiracy, such as network analysis, content analysis, sentiment analysis, misinformation and stance detection.

Contributions
In this study, we collect and share a Twitter dataset of 15K users, categorized into two groups: a set of textitconspiracy users engaging with diverse conspiracy theories in May 2022, and a control group of random users collected from those posting on the same topics during the same period.Additionally, we collect their timelines, totaling of 37M tweets.Numerous social platforms, ranging from fringe to mainstream, have been exposed to various types of conspiracy narratives [21,22,10,23,24,25,26,27]. Here, we focus on Twitter data due to its reported extent of conspiracy engagement, wide audience reach, rapid dissemination, and ease of accessibility.With a robust dataset, we analyze the distinctions between the two user groups across three dimensions: topic preferences, profile metadata, and behavioral patterns.
In particular, our approach allows us to explore new research directions and address the following research questions: RQ1 -How can we construct a robust and comprehensive dataset of online conspiracy users?Existing datasets are often limited by simplistic gathering methods.In this study, we collect with a rigorous methodology users endorsing conspiracy beliefs posted by known conspiracy source on Twitter.We also collect a control group of random users with similar metadata properties who discuss the same topics but do not show signs of conspiracy involvement, ensuring a fair comparison between the two groups.RQ2 -What are the differences in attitude and approach when discussing topics between conspiracy and random users?Different user groups can interpret and engage with topics in diverse ways.We delve into and compare the manners in which these two groups relate to and discuss specific subjects.RQ3 -Which features predominantly differentiate the conspiracy users?Our analysis of feature importance involves training various classifiers using three classes of features.The results reveal that conspiracy users tend to engage more in conversations, reply frequently, and exhibit less lexical variability.This linguistic distinctiveness holds even when compared to a control group of randomly selected users with similar activity, nature, and language characteristics.Furthermore, we compare our findings with state-of-the-art techniques that leverage deeper linguistic properties, resulting in an 11% increase in F-score.
Our main contributions stemming from the aforementioned RQs can be summarized as follows: • We create and publicly share a large, robust, and balanced dataset of 15K conspiracy and random control users, along with 137M tweets from their timelines.
• We show that these two user groups display divergent attitudes and perspectives on specific topics, thereby reinforcing their distinctions.
• Our analysis indicates that both groups have an automation rate below 1%, suggesting the involvement of genuine, real individuals.
• We provide analysis on discriminative features by employing a classifier that leverages profile metadata, behavioral characteristics, and linguistic features borrowed from the literature on bot, troll, and conspiracy detection, obtaining a high accuracy (average F1 score of 0.98%).
• We compare our detection capabilities with state-of-the-art methods, showing a better performance in terms of F1 score.
Reproducibility.We release an anonymized, privacy-preserving version of the dataset1

Roadmap
The remainder of this paper is organized as follows.In Section 2, we review recent literature concerning online conspiracies, with a focus on works that offer accessible datasets and profiles of conspiracy users compared to random users.In Section 3.1, we present and motivate our data collection strategy, and provide the first descriptive statistics about the resulting dataset.In Section 3, we provide an overview of the methods leveraged for the analysis and characterization of conspiracy compared to a control group of random users.In Section 4, we present the results, showing the differences between the two groups in terms of topic and profile metadata, as well as present the classification results and show the most discriminative features.Finally, in Section 5, we summarize the main findings, contextualize them within the broader research effort against online conspiracy, address the limitations, and outline potential future research directions.

Related work
In recent years, there has been an increasing focus on identifying and understanding conspiracy theories.Unlike other forms of misleading content, like misinformation, disinformation, fake news, or rumors, conspiracy theories present a unique challenge.They raise the crucial and ongoing to question whether they are fundamentally false, alternative explanations, or situated somewhere between reality and fiction [5,7].Interestingly, individuals on social media who embrace conspiracy theories aren't solely automated accounts, trolls, or spreaders of fake news and rumors.Many genuinely hold beliefs in hidden agendas or plots, some of which have been verified as true2 , while others remain unverified.Furthermore, not all conspiracy theorists actively scheme or propagate them.Among them are regular people who simply hold these beliefs without actively disseminating them.To discriminate between these different types of users, the availability of a high-quality, robust social media dataset is essential.Such a dataset must encompass the wide array of dynamic behaviors associated with conspiracy beliefs, but its creation presents significant challenges that necessitate meticulous planning and assessment.Previous studies have attempted to create social datasets for similar purposes, not without some limitations and trade-offs.
The following two subsections delve into the current leading methods for constructing datasets comprising both conspiracy-affiliated users and those in a control group, as well as describe how these two groups are compared and characterized.

Datasets of conspiracy and control group users
In literature, the comparison between users who engage in conspiracy theories and a control group is commonly performed by analyzing the presence of specific keywords and/or URLs related to these theories within social media posts [28,29,30,20,31,21,32,33]. In this context, the control group typically consists of users who consume content that directly opposes specific conspiracy theories.Some studies employed such approach and collected datasets of conspirative and non-conspirative users.For instance, the authors in [28] collected and annotated tweets related to COVID-19 conspiracy, sampled 109 conspirative users annotated as posting conspiracy theory content and retrieved their timelines.As control group, they identified 109 non-conspirative users exhibiting a tweet behaviour focusing on coronavirus related content in general, using generic related keywords (e.g., corona, covid, pandemic).However, this study faces certain limitations.Notably, the dataset size is relatively small due to the challenging task of manual annotation, and the focus is narrowed to a specific conspiracy theory.
Similarly, prior investigations have predominantly centered around specific conspiracy theories [34,35,36,33,37].In [31], the authors focused on the conspiracy narrative surrounding COVID-19, leveraging hashtags in support or opposed to specific conspiracies.Similarly, the authors in [20] collected and analyzed online discussions related to four distinct conspiracy theories.Nevertheless, findings tied exclusively to a single conspiracy theory may not be easily extrapolated to other societal events that might trigger beliefs in conspiracies.Therefore, our primary objective is to explore users involved in conspiracy-related discussions without being confined to a particular information operation or theory.
The authors in [29] advanced the state-of-the-art by considering six conspiracy theories.They curated and annotated tweets containing hashtags likely used to either endorse or refute these theories.This resulted in retrieving 977 users who engage with conspiracy content and 950 users who counteract such narratives.Meanwhile, the authors in [30] managed to expand the conspiracy user base by adopting a different method.They exploited five known conspiracy-affiliated and five science-oriented Twitter influencers, retrieving a sample of their followers to create a dataset of proand anti-conspiracy groups.However, these approaches have some drawbacks.Specifically, hashtags or relationships do not always accurately reflect the content of a tweet, as they can indicate both support for or rejection of conspiracy theories.Our approach tackles this issue by considering likes, which better capture individual user preferences.This eliminates the need for manual annotation and allows for the examination of a larger user base.
Close to our approach, previous works [32,21] labelled conspiracy enthusiasts and science-minded users based on the number of likes they gave to conspiracy and science-related posts on Facebook.However, their analysis was confined to the comments these users left in conspiracy and science groups, disregarding their broader Facebook posting history.Finally, it's worth highlighting that our focus is on comparing conspiracy theorists with the broader population of Twitter users, rather than exclusively contrasting them with those who oppose conspiracies.Our focus is on Twitter, for its (formerly) data retrieval ease and its role in the dissemination of conspiracy theories [38,39].However, the approach we employ holds the potential for application across various social media platforms.

Characterization of users engaging conspiracies
The characterization and detection of malicious users has received a lot of attention during the last years.Researchers have primarily concentrated on analyzing the traits of various types of problematic users, such as social bots, state-sponsored trolls, and more recently, conspirators.These investigations have predominantly focused on elements like profile metadata, demographic details (e.g., gender, age) [40,41,42], social activity [43], interactions [44,42], and relationships [12,16,42,22].
A more recent trend in conspiracy detection involves examining linguistic patterns present in textual content [30,29,22,32].This approach aims to identify specific language cues associated with conspiratorial discourse.For instance, the authors in [43] explored linguistic features like average text length, word redundancy, emotional responses, and psycholinguistic aspects to distinguish conspiracy-related content.Likewise, other researchers explored linguistic cues to detect trolls [45] and conspiracy [46,12,29].Similarly, our work investigates features commonly used to spot malicious users (such as fake accounts, bots, trolls, and spreaders of misinformation), which have not been extensively explored for identifying conspirators [47], while also incorporate linguistic features into our analysis.
To obtain the features that better discriminate conspirative users from random ones, we leverage the outcomes from several standard machine learning classifiers.Similar efforts were made by previous works, such as [30], in which authors employed a logistic regression model to confirm their qualitative findings on psycholinguistic traits associated with conspiracy believers and science enthusiasts.However they did not present detection results for direct comparison with our work.In another relevant study, [33], the authors analyzed covid19 conspiracy discussions and classified users into misinformation and non-misinformation groups based on their profile metadata and tweet embeddings.However, their approach relied on graph-based methods and did not provide sufficient details for reproducibility (e.g., dataset, graph-data), which limits the comparison with our work.
In contrast, our study proposes a model that encompasses diverse features to identify online users engaged in conspiracy discussions.We also benchmark our classification outcomes against a study, [29], that used a CNN-based model incorporating linguistic traits to differentiate between users who share posts supporting or refuting conspiracy theories.
In summary, we adopt a computational approach to study the conspiracy phenomenon and compare online users who engage in conspirative discussions with random users.In particular, we examine the profile, activity and psycholinguistic characteristics of conspiracy and random users based on the tweets that they post.Furthermore, we introduce a model that utilizes different features to identify online users participating in conspiracy-related conversations.Therefore, our study offers an orthogonal view and contribution building on prior work by employing distinct methodologies for constructing a reliable and robust conspiracy dataset and applying extended methodologies for characterization and detection.

Methods and data
In this section, we delve into the methods employed to gather and analyze data for our study.We first delineate our data collection strategies for collecting users engaged in conspiracy theories and random users.Next, we describe feature extraction methodologies.

Data Collection Strategy
To address our first research question (RQ1) on creating a robust dataset of users engaged in conspiracy content, we propose a strategy based on users' liking behavior towards posts from various conspiracy accounts.We argue that liking a post indicates stronger approval and endorsement of the message compared to a mere re-share [48].
To ensure a fair comparison, we introduce a control group, on the line of prior research [22].This control group comprises randomly selected users with similar metadata and engaging in discussions around the main topics.In this way, we can compare the two groups fairly.
In summary, our dataset consists of a conspiracy group comprising 7,394 Twitter users and a control group with an equal number of randomly selected users.We will refer to the control group users as textitrandom users throughout this work.The subsequent sections describe in detail the strategy and criteria used to select both conspiracy and random users.

Strategy for collecting conspirative users
The idea behind the collection is to identify the users who are most likely to believe in various conspiracy theories, by focusing on those appreciating, by means of a like, posts from various conspiracy sources (e.g., websites).The adopted strategy took place in June 2022 and led to the collection of the latest 3,200 tweets for 7,394 users known for liking conspiratorial content.Our approach comprises three key steps, which correspond to Figure 1: 1.
Step 1 -Initial Set of Conspiracy Sources: We identify an initial set of websites rated as conspiracy Media Bias/Fact Check (MBFC) 3 , a non-profit organization assessing online source credibility and bias.We extract the associated Twitter accounts and leverage them as seed accounts for the following steps.

2.
Step 2 -Collecting User Likes: We gather Twitter users who have liked posts from the seed accounts, indicating potential conspiratorial engagement.

3.
Step 3 -Applying Filters: We retain users who both follow at least one seed account and have shown uniform interest across multiple seed accounts.The goal is to balance between diversity (number of distinct seed accounts liked by a user) and intensity (total user likes across seed accounts).Additionally, we strike a balance between the absolute number of likes per user for seed accounts and the number of liked seed accounts.We retain users based on this trade-off, constituting our conspiracy group.
In summary, our goal is to identify users likely to be conspiratorial based on interactions with seed accounts.We apply the method as follows: Step 1.We select an initial set of 26 conspiracy sources from MBFC as seeds, as shown in Table 1.As mentioned, MBFC is an indipendent website that aims to provide an objective and transparent assessment of the credibility and bias of online news media sources.By rating more than 4K media sources and employing a team of trained experts and journalists across the political spectrum, MBFC has become the most comprehensive media bias resource on the internet.The website's ratings are based on rigorous criteria and methodology, and have been utilized by researchers for various academic purposes [49,50].In particular, we leverage a list of 300 news source websites rated as engaging with various topics of dubious veracity and scientific validity, conspiracy theories and pseudo-science.We then look for the twitter accounts associated with these websites and find 100 matches.For computational time purposes, we apply manual annotation and filter the matches to keep only the top 26 accounts that have more than 90 tweets endorsing any conspiracy theory in their most recent 100 tweets.Finally, we leverage them as the seed set of conspirative Twitter seed accounts.
Step 2. In this step, we leverage the like interaction on Twitter, which allows users to express their appreciation or interest in a tweet.Unlike the sharing, which amplifies the message to a wider audience and allow for fact-checking or criticism, or the replying, which initiates a conversation or provides feedback that may or may not agree with the message, the like interactions convey approval or endorsement with posts.
By employing Twitter API V2 endpoints, we retrieve likes from seed accounts' posts between July 19th, 2021, and February 28th, 2022.This method captures user engagement with seed accounts over time, providing a comprehensive view.We obtain 8, 935, 961 likes from 968, 824 users for 54, 559 tweets by seed accounts.
Step 3. In this step, we perform a series of filtering steps to refine our user selection process.Initially, out of a pool of 968, 824 potential conspirators, we retained only those (378, 144 users) who also follow at least one of the seed accounts.This demonstrated not only an initial attraction to the tweets of these seed accounts but also an ongoing interest for their overall content.Subsequently, we filter users (345, 936 users) who display a well-balanced engagement with multiple seed accounts.In other words, if a user likes content from several seed accounts, their level of interest across these accounts is relatively consistent.We measured this by applying a coefficient of variation (Cov ) of the number of likes per seed account, keeping it at or below 1.Finally, as mentioned, we establish the third filter based on two key factors: the total number of likes given to the set of seed accounts and the count of liked seed accounts.These factors provide insight into the intensity of activity related to conspiracy sources and the range of interest in different conspiracy theories.The combined analysis of these factors is summarized in Table 2.The table cross-references the absolute number of likes (up to 35) along the X-axis with the number of distinct seed accounts liked (up to 7) along the Y-axis.Each cell in the table represents the count of users who have distributed a minimum of Y likes across a minimum of X distinct seed accounts.
As we move towards the lower right corner of the table, the number of users naturally decreases.Ideally, we would like to select conspirators who extensively liked a large number of seed accounts.However, this approach total sources (s)  would yield a very small pool of users, potentially less than 100.Instead, our strategy is to strike a balance by focusing on a diverse range of sources while maintaining a reasonable number of likes.The aim was to have around 10, 000 users for meaningful analysis.This process culminated in selecting 7,394 conspiracy users for in-depth analysis.These users liked at least 4 different conspiracy sources and exhibited a consistent distribution of at least 25 likes.By leveraging Twitter API, we gathered the timeline (the most recent 3,200 tweets) from these 7,394 selected conspirators, accumulating a total of 18, 273, 565 tweets.The final dataset comprises tweets covering the time span from February 27th, 2008 to June 13th, 2022.

Strategy for collecting random users
To enable a meaningful comparison with our conspiracy group, we establish a set of random users as control group.The selection criteria ensure parity in terms of discussed topics, account creation period, and language usage.This comprehensive approach involves three steps, as shown in Figure 2: • Step 1 -Collecting Topic-Related Discussions: We collect tweets linked to the top 10 hashtags used by conspiracy users (Table 3).We establish June 13th, 2022 as the end date for this data collection process.This specific date aligns with the termination of the collection of conspirators' timeline.
• Step 2 -Extracting Users discussing this Topics: This step resulted in the retrieval of 152, 588 tweets authored by 82, 796 distinct users.
• Step 3 -Filter Random Users: We exclude users engaging with any of the 26 conspiracy seed sources to ensure a non-conspiracy profile.Additionally, we ensure uniformity of the predominant language in tweets.Finally, chose random users whose creation dates match with conspiracy users, maintaining equal distribution among these groups.
This rigorous approach results in a set of 7,394 random users.We gather their timelines, providing a comprehensive dataset for comparative analysis, ending up with 19,268,801 tweets.

Feature extraction
We aim to identify the features that separate conspirative users from control users.We use data and meta-data to compute 93 different features that provide insights into various aspects of our users.Each feature is either a continuous numeric value, a binary value, or a set of statistics calculated from distribution (i.e., minimum, maximum, median, mean, standard deviation, skewness, and entropy).Each feature falls into one of three categories: continuous numeric values, binary values, or statistical measures derived from distributions (such as minimum, maximum, median, mean, standard deviation, skewness, and entropy).Some of these features are drawn from prior studies on bots and trolls, specifically selecting those proven effective in detecting or characterizing social bots and state-backed trolls [47].
In details, our approach involves extracting account features organized into three groups that capure different aspects of social network behavior.These groupings are inspired by previous research [47] that identified attributes related to account trustworthiness, topical focus [51], behavioral dynamics [51,52], and strategic goals [51].These feature groups, referred to as "traits," offer a suitable framework for describing and distinguishing diverse types of social network accounts.Unlike other studies that categorized features broadly into conventional domains (such as user-based, friends, network, temporal, content, sentiment, etc.) [53,54,55], we adopt a more intuitive grouping that aligns with the various roles an account can assume within the context of a social network.Our features are summarized by class and presented in Table 4.We discuss them briefly in the following sections.

Credibility features
This category encompasses features that evaluate the credibility and trustworthiness of social media users based on their profile characteristics.The underlying assumption is that discussions are more likely to be organic if they involve mostly credible users.These features capture profile attributes that can differentiate between low-credibility and high-credibility accounts, such as the quantity and nature of social relationships, account age, and activity level.These features primarily draw from profile metadata, easily observable and assessable when viewing a social network account.These attributes have long served as discriminators for simplistic fake accounts [56,54,57,58].The extracted features for each trait [47].The distribution parameters are the min, max, mean, median, std, skewness, and entropy.

Initiative features
This class measures an account's influence in initiating and guiding discussions, shaping online conversations, and producing diverse and original content.To achieve this, we employ a set of features that quantify the quality and quantity of a user's activity, building on prior works [59,47].These features include metrics like the ratio of original to retweeted content, indicating an account's contribution to generating fresh and unique material rather than amplification.Additionally, metrics like the ratio of tweets to replies reflect the user's engagement in dialogues and exchanges with other users, rather than just broadcasting its own messages.These features help measure the quality and diversity of online discussions.

Adaptability features
Adaptability refers to the ability or willingness to change in order to suit different conditions.In our study, we measure account adaptability based on how it alters and adapts its behavior and profile over time in response to encountered or contributed topics.For instance, we examine linguistic aspects such as language novelty, entropy, and diversity, alongside other linguistic characteristics reflecting temporal changes.Thus, adaptability ties into the account's temporal and topic-related dynamics and its language usage [45,51,59].

Dataset Creation (RQ1)
In literature, users engaged in conspiracy activities are often identified as accounts who employ specific conspiracy-related keywords or share URLs from conspiracy websites [28,29,30,20,31,21,32,33].However, some of these users may be bots or trolls attempting to spread panic and skepticism in authorities by pushing alternative explanations for events [60,61,62,63].Misinformed users may inadvertently spreading conspiracy theories [35,36,34].Our data collection accounts for this subtle difference between malicious users, misinformed users and actual conspirators by adopting a strategy that does not rely on the usage of either keywords or URLs.
Our approach is grounded in the idea that "liking" a post expresses approval or support for its content, which does not necessarily apply to sharing [48].This implies that users who frequently like posts from a particular account are likely endorsing the themes promoted by that account, and this endorsement is even stronger if the user follows the account.Consequently, a user who consistently likes posts from various conspiracy accounts, while also being a follower of at least one such account, is more likely to believe in conspiracy theories.Furthermore, relying solely on conspiracy keywords or URLs to identify conspiracy users results in capturing only those who actively spread and support specific plots.Our strategy, on the other hand, enables us to identify conspiracy users who may not necessarily propagate the theories they believe in, and who might endorse a range of conspiracy theories if the source accounts are of a generic nature.Here, "generic" refers to Twitter accounts that discuss multiple conspiracy theories concurrently.A similar liking-based strategy was used by authors in [32] and [21], who identified conspiracy users based on their significant liking activity on conspiracy-related  posts.In our approach, we also consider the "follow" relationship from users to conspiracy accounts, providing a stronger validation of their affiliation with the conspiracy realm.
For the control group, our aim is to include users that represent the broader social media population while minimizing superficial differences between an average random user and a conspirator.Utilizing anti-conspiracy keywords or URLs (e.g., science-based) [29,21,30] or general keywords related to broad topics [28,20,31] helps reduce these disparities.However, an additional mechanism is required to ensure similarity between the control and conspiracy groups while maintaining the integrity of both.In our work, instead, we ensure that conspiracy users and random users discuss similar topics and are created around the same time period.It is important to note that these common topics may not be conspiracy-related.To filter potential conspirators from the control group, we exclude random users who have liked posts from any of our seed accounts.A similar concept of a control group was used in [22], where users with similar initial activity to conspirators were identified and tracked as they diverged over time.
We briefly provide an overview of the characteristics of our dataset.Figure 3 shows descriptive statistics of the collected users.In terms of content, random users exhibit a higher proportion of original tweets (19%) compared to conspirative users.Conversely, the latter group displays a greater inclination towards engaging in replies (35% as opposed to 21%).Regarding retweets and quotes, no substantial differences are evident.
Finally, we verify the presence of automated accounts by employing Botometer v4 to compute the bot scores for both conspiracy and random users [64].The analysis revealed no bot presence within the conspiracy group.However, approximately 1.5% of random users show a likelihood of being bots with a confidence level greater than 90%.We consider the 1% noise acceptable for our study's purposes.

Topic characterization (RQ2)
In this section, we answer to RQ2, by focusing on the key subjects discussed by the two distinct groups.First, we extract the main topics through social network analysis based on co-occurring hashtags.Then, we employ topic modeling to highlight the primary themes of conversation within each user group.In this way, we highlight highly correlated words that may give a hint on the attitude towards a specific topic.Notably, conspiracy users were collected by leveraging likes, while random users were collected by leveraging hashtags over the same time period.Taking that into consideration, we perform the following analysis on timelines and properly handle hashtag seeds, as detailed as follows.

Visualizing co-occurring hashtags per group
We begin by computing and visualizing the graph of co-occurring hashtags for conspiracy users.We compare it with the graph generated from hashtags that co-occur in tweets posted by random users.These co-occurrence graphs depict the interconnections between hashtags based on their simultaneous appearance within tweets.For clarity, figures 4 and 5 show only the top 50 hashtags based on weighted degree.Figure 4 shows the co-occurrence graph of hashtags mentioned in all tweets posted by conspiracy users.As shown, the core of this graph is predominantly composed of two clusters.One cluster centers around topics related to covid-19 and vaccination discourse.The other cluster involves hashtags commonly used for describing images on Instagram [65,66], possibly due to cross-platform social media sharing.In Figure 5 we reconstruct the co-occurrence graph of hashtags used by the random users.Given that we collected data for these users by focusing on the top 10 hashtags used by conspiracy users, we omit those hashtags from our analysis.In this scenario, the core of the graph is mainly composed of hashtags associated with cryptocurrency.Notably, covid-related hashtags appear on the periphery of the graph.This suggests that during the data collection from random users, cryptocurrency held a stronger influence than the other topics supplied as input.Nevertheless, the popularity of certain topics (e.g., cryptoworld) might surpass others like (e.g., covid), based on factors such as current trends and individual user preferences.
In the next section, we provide a more extensive exploration of the topics and analyze the different user groups' attitude and stances on these subjects.To gain a deeper understanding of the different attitudes towards the online discourse between conspiracy and random users, we employ topic modeling using a recent, advanced, cutting-edge algorithm known as Anchored Correlation Explanation (CorEx) [67].The CorEx algorithm learns latent topics from documents without assuming an underlying generative model.It maximizes the correlation between groups of words and latent topics, leveraging the dependencies between words in documents.This approach ensures enhanced flexibility, enabling hierarchical and semi-supervised variants [67].An essential feature of CorEx is also the ability to anchor words, which facilitates semi-supervised topic modeling and enhances topic separability with minimal intervention.Anchoring involves injecting prior knowledge (anchor words) into the topic model to identify and differentiate underrepresented or significant topics.This process enables us to extract pertinent topics and the associated terminology.Given our focus on studying the attitude towards shared main topics by both user groups, we capitalize on the word anchoring capability of CorEx to enhance topic separability.

Characterizing topic discussions and attitudes
We build two distinct models for conspiracy users and random users to account for potential variations in topics and forms of speech.We select the top 10 hashtags from Table 2 as anchor words.After experimenting with various configurations, we set the expected number of topics to 10, as additional topics yielded negligible correlation improvement.Finally, we rank the resulting topics based on the correlation fraction they explain.The outcomes of this analysis are summarized in Table 5, with topics ordered by the amount of total correlation explained.Within each topic, words are arranged according to mutual information with the topic, and anchor words are highlighted in bold.Anchoring substantively augmented the contribution of topics of interest to the model's correlation.High topic quality is confirmed by the presence of non-anchored words with strong coherence within each topic.We report the most informative topics, in Table 5.We uncover some notable differences in the discussion of the same topics.For instance, conspiracy users discussing the topic of covid-19 topic deploy other highly correlated non-anchored words tied to conspiracy terminology [68,69].Notably, phrases like wake up and appeals for free speech stand out.Other terms encompass bigpharma and vaccinedeaths.In contrast, random users use milder language in relation to this topic, such as fakenews, humanrights, and breakingnews, highlighting the moderation of this group.Similarly, words strongly correlated with the "pfizer" topic among conspiracy users revolve around adverse symptoms (e.g., adverse, myocarditis, vaers).Random users, on the other hand, use more general terms (e.g., biontech, wef ).
Another example pertains to the discussion about the international treaty for pandemics prevention and preparedness established by the World Health Organization (WHO) aiming to ensure equitable sharing of vaccines, drugs, and diagnostics during future pandemics [70] [70].Conspiracy users' correlated words include billgates bioterrorist, trudeaufortreason, crimesagainsthumanity, and other words with a nuance of adversion against the act 4 .In contrast, words used by random users are more generic, and neutrale (e.g., population).Finally, regarding discussions on Ukraine and cryptocurrency, no substantial variations emerge between the two user groups.

Leveraging classification for extracting conspiracy discriminating features (RQ3)
In order to identify conspiracy-related users and determine the key features that differentiate them from regular users, we leverage a set of 13 off-the-shelf machine learning algorithms (i.e., Light Gradient Boosting Machine (LIGHTGBM), Random Forest (RF), Gradient Boosting Classifier (GBM), Ada Boost Classifier (ADA), Extra Trees Classifier (ET), Decision  Tree Classifier (DT), Logistic Regression (LR), Linear Discriminant Analysis (LDA), Ridge Classifier (RIDGE), K Neighbors Classifier (KNN), Support Vector Machine (SVM), Naive Bayes (NB), Quadratic Discriminant Analysis (QDA)).These classifiers are trained using a stratified 10-fold crossvalidation approach.We assess several models, beginning with a baseline model, and progressively adding more features to each subsequent model.As a preprocessing step, we initially divide our dataset into training and testing sets using an 80/20 split.To provide a rigorous evaluation, we randomly select users for the training and test splits, preserving the balance of the two types of users.As shown in Table 6, the training set includes 12, 708 users, 6, 354% of which are labelled as conspirators and 6, 354% as random.The test set consists of 3, 178 users, of which 1, 589% are conspirators and 1, 589% control users.We address missing categorical values by replacing them with the most frequent value within the respective column.Missing numerical values are substituted with the mean value of their respective columns.
Table 7 shows the outcomes of the classification process.The table presents the performance of two baseline models: Majority Class, which always predicts the majority class; and a random predictor.We show alongside the outcomes of the optimal classifier (LIGHTGBM), evaluated in terms of the F1 score.Within the context of the LIGHTGBM classifier, we incorporate varying sets of features to assess their effectiveness.

Feature importance evaluation
Here, we explore the feature importance of the best-performing classifier, specifically the LIGHTGBM algorithm, in a comprehensive model that incorporates all features related to credibility, initiative, and adaptability.The goal is to gain a better understanding of which features contribute significantly to the accurate identification of users engaging in conspiracy activities.Among the discriminating features, the character entropy in tweets emerges as the most influential.A closer examination, as demonstrated in Figure 7a, reveals that random users exhibit greater diversity and richness in their character usage.In contrast, conspirators tend to employ a narrower array of characters and words, suggesting a focus on specific topics and discussions.As second discriminating feature, the mean number of tweet per different language provides insights into the tweet's global reach and potential for cross-cultural engagement.A diverse language usage suggests broader appeal and variety of topic.Conspirators tend to use a single language for their tweets, while random users employ a wider spectrum of languages in their content.
As second discriminating feature, the reply rate provides insights into the level of engagement a tweet generates and its consequential relevance and impact on the audience.A high reply rate implies a tweet's ability to initiate discussions and encourage interactions.Figure 7b reveals that conspirators exhibit a higher reply rate compared to random users.When replying, conspiracy users connect with a wider audience and engage in prolonged conversations with respect to random users.In addition to the aforementioned features, additional variables contribute to understanding the significance of certain characteristics in the analysis.For instance, the number of shared URLs, as prevalent among conspirators, offers insights into the extent of their engagement with external content supporting their beliefs, potentially influencing the reception of their tweets.
In summary, adaptability-related features are the most influential factors in user categorization, followed by initiative and credibility-related features.These adaptability features, particularly tied to linguistic aspects, stand out as pivotal even when user groups share similar activity, nature, and language traits.This highlights the significant role of linguistic properties in distinguishing between these categories.In the subsequent section, we deeper examine them by comparing our findings and feature importance with a cutting-edge technique from the state-of-the-art that primarily focuses on analyzing the psycholinguistic properties of users.This exploration aims to provide deeper insights and understanding of conspiracy activity.

Comparison with the state-of-the-art
As mentioned, we conduct a comparison of our results with those in [29], which explored the psycholinguistic characteristics of 977 conspiracy users and 950 anti-conspiracy users.Similarly, we leverage: • Emotions: the amount of emotions expressed by the users in their tweets, which includes eight emotional categories (i.e., anger, anticipation, disgust, fear, joy, sadness, surprise, and trust) as defined in [71], computed by leveraging the National Research Council (NRC) emotions lexicon [72].
• Sentiment: the amount of sentiment polarity (i.e.positive, negative) expressed by the users in their tweets, computed by leveraging the National Research Council (NRC) sentiment lexicon [72].
• Personality traits: we infer the personality traits of users from their tweets by utilizing the IBM Personality Insights API5 .These traits consist of the renowned Big Five traits [73]   Values (conservation, hedonism, openness to change, self-enhancement and self-transcendence) and 12 Needs (challenge, closeness, curiosity, excitement, harmony, ideal, liberty, love, practicality, self-expression, stability and structure).
Following the methodology called ConspiDetector in [29], we incorporate these user-specific characteristics and GloVe embeddings of user tweets io a dual-branch Neural Network.We exclude 772 random users from the analysis due to insufficient text content for computing IBM Personality traits.Table 8 shows the details of the training, validation, and test sets.
In addition to running ConspiDetector on our conspiracy and random users, we evaluate again the best-performing machine-learning classifier on the unbalanced dataset leveraging the credibility, initiative, adaptability and psycholinguistic features.Table 9 indicat that training a standard machine learning algorithm on our dataset using psycholinguistic traits obtains similar results as ConspiDetector, leading to an F1 score of 0.90.Furthermore, the performances obtained on the unbalanced dataset are in close alignment with those achieved on the balanced dataset.
Finally, when looking at the most discriminative features, Figure 8 illustrates that psycholinguistic traits hold less prominence compared to behavioral characteristics.Nevertheless, within the top 20 features interestingly emerge the emo-tion disgust trait (i.e., disgust, as the opposite of trust), which conveys the conspirators' tendency to exhibit less assertiveness and sociability, as well as suspiciousness and longing for building knowledge [71].

Conclusions
Online conspiracy detection is a challenging task that requires a combination of robust data and tools.In this paper, we proposed a comprehensive methodology for collecting a rigorous Twitter dataset to study conspiracy theorists' characteristics and compare them to randomly selected accounts that exhibit similar characteristics.In particular, we leveraged the "like" behavior on social media platforms as it can reveal affiliation with conspiracy theories better than other behaviors (e.g., retweets, relying on the use of URLs, etc.).In fact, users who frequently like posts from a specific account are likely to support the themes promoted by that account, especially if they also follow the account.This endorsement of themes is stronger when users both like posts and follow conspiracy-related accounts, making them more prone to believing in conspiracy theories.For the control group, we collected users representing the broader social media population whose activity matches the topics discussed and account creation time of conspiracy users.In this way, we created a more balanced comparison between conspiracy users and regular users while maintaining the integrity of both groups.
In addition, we presented a robust approach to detect online conspirative users based on their behavioral characteristics, linguistic features, temporal patterns, and other features proposed in the literature for identifying bots and trolls.The goal of this classification task is twofold.On one hand, we showed that using a standard machine learning classifier on linguistic features and temporal patterns outperforms several baselines and a model proposed in the state-of-the-art as measured by accuracy and F1 score.On the other hand, we employ these findings to profile the two user groups and highlight features that differentially characterize conspiracy-oriented users on social media.
Results show that the most discriminating features are the linguistic characteristics.The development of methods to detect conspiracy users based on linguistic traits and patterns, rather than the content of their claims, can be pivotal in identifying and monitoring the proliferation of conspiracy beliefs across diverse platforms and domains.

Limitations and feature work
Our approach presents some limitations that need to be addressed in future work.
One primary limitation pertains to the usage of Media Bias Fact Check (MBFC).While widely employed to assess the conspiratorial inclination of news sources, MBFC's categorization process is subjective and potentially influenced by evaluators' personal biases.The criteria used by MBFC to evaluate conspiracy may not be universally agreed upon and can vary from person to person.Additionally, the methodology and transparency of MBFC's factchecking process may not be fully disclosed, making it difficult to assess the accuracy and reliability of their assessments.Moreover, MBFC's database might not cover all news sources, especially smaller or less-known outlets, resulting in potential gaps in general coverage.It is essential to approach MBFC's ratings with a critical mindset and to consider multiple sources and perspectives when dealing with conspiracy.
Another limitation is our singular focus on Twitter, potentially overlooking the multifaceted nature of online conspiracy discourse across various media.Recent shifts in the Twitter policies further challenge the replicability of results.To better understand and tackle online conspiracy activities, future studies should encompass data from multiple platforms.
A further constraint stems from our data collection process, relying on features derived from users' timelines.In fact, this approach can be computationally intensive and susceptible to data availability issues.For real-time detection, an efficient and robust alternative could involve simpler features based on users' current activities and interactions, that do not depend on the users' history.Future research might also harness broader social network information, such as followers and followees, to gain additional insights about user credibility and influence.
Additional avenues for future research may encompass investigating the political orientations of users engaging with conspiracies and identifying the propaganda strategies and rhetoric used by conspiracy theorists to persuade their audience.
In conclusion, our work contributes to the growing field of online misinformation and disinformation research, presenting a valuable dataset and methodology for understanding and combating the propagation of harmful and false beliefs.conspiracy theorists and their followers on twitter, Group Processes & Intergroup Relations 24 (4) (2021) 606-623.

Figure 1 :
Figure 1: Overview of the proposed strategy for collecting conspirator users.

Figure 2 :
Figure 2: Overview of the proposed strategy for collecting random users.

Figure 4 :
Figure 4: Co-occurrence graph of hashtags mentioned by conspiracy users.

Figure 5 :
Figure 5: Co-occurrence graph of hashtags mentioned by random users.

Figure 6
Figure 6 exhibits the features in descending order of their impact on the Gini criterion, providing insights into their predictive importance within the model.The figure highlights the top 20 features, offering a ranked view of Entropy of characters in the tweets.random users adopt a greater variety of characters with respect to conspirators.Reply rate.Conspirators are characterized by a higher reply rate with respect to random users.

Figure 7 :
Figure 7: Most discriminating features of the model.
Extraversion trait as defined in the Big Five personality traits.Random users are more extroverted.Curiosity trait as defined by IBM Personality Insights API.Conspirators are more curious.As defined by Plutchik [71], disgust is the opposite of trust.Conspirators are characterized by higher disgust.

Table 1 :
The 26 selected conspiracy websites.

Table 2 :
Number of users who distributed at least Y likes (up to 35) on at least X distinct S accounts (up to 7).

Table 3 :
Top 10 hashtags used by conspiracy users.
number of replies containing a URL and the number of tweets Words in Tweets Distribution parameters Distribution of the number of unique words in tweets Words entropy in tweets Distribution parameters Distribution of the number of unique words entropy in tweets Adaptability Language Novelty Distribution parameters Percentage of new tokens in a tweet compared to those previously used Time Between Tweets Distribution parameters Distribution of time differences between consecutive tweets Time Between Retweets Distribution parameters Distribution of time differences between consecutive retweets Time Between Mentions Distribution parameters Distribution of time differences between consecutive tweets containing mentions

Table 5 :
Topic modeling results, obtained by applying Anchored Correlation Explanation (CorEx) to conspirative and random users.Conspirative users are characterized by the use of more extreme and intense words when discussing a relevant topic.

Table 6 :
Dataset composition (ground-truth and train/test split) for the classification task.

Table 7 :
Performance of the conspiracy and random users detection on different groups of features.

Table 9 :
Performance of baselines, ConspiDetector and LIGHTGBM on our (unbalanced) dataset.all includes credibility, initiative, adaptability and psycholinguistic features.