Agency Aspirations: Understanding Users' Preferences And Perceptions Of Their Role In Personalised News Curation

2.1 The Role of Users in Personalisation

The proliferation of novel techniques that support the personalisation of news means that editors and journalists are no longer solely responsible for curation. Equally, it elevates the role of the user, from being mostly at the receiving end of a readily curated selection of news to being an active agent in a reciprocal process [42, 48]. Bastian et al., in their research with media practitioners’ and their perceptions of ANRs, noted that while opinions about users’ ability to influence outcomes of personalised news curation were divided, overall three different options for integrating user agency could be identified: 1) Giving users more control over their data; 2) Allowing users to opt out of personalisation features; 3) Giving users the means to influence the outcomes of ANRs [5]. These options aside, the consequences of ANRs on user agency need further study, with Bastian et al. arguing that personalised news curation tailored to individuals may increase their agency while the limited ability of users to control them may diminish it [5, 48]. Bastian et al. therefore highlights the need to find balance between user, editorial, and algorithmic curation, and also observe media practitioners’ scepticism about whether users would actually make use of this agency [5].

Prior research has shown that user attitudes and knowledge about how personalised and recommender systems work and their perception of their role in them vary greatly. Obvious instances in which users can exercise their agency are defined by Thorson and Wells as “personal curation” [65], by Van den Bulk as “explicit agency” [71], or by Helberger as “self-selected recommendations” [35]. These instances in the interaction are for example when a user is prompted to make a choice or encouraged to disclose information that would help customise and personalise different features and elements, such as the user interface and content curation. Counterparts are defined as “algorithmic curation”, “implicit personalisation”, or “pre-selected recommendations”; more invisible instances in which an algorithm (and by extension the media outlet employing the recommender system) handles personalisation based on data previously gathered about the user – provided voluntarily or inferred. Furthermore, although different in theory, in practice the distinction between personal and algorithmic curation can become blurry as users can influence algorithmic curation by purposely producing signals that they believe may be picked up by the algorithm to co-curate their personalised experience. In literature, these user behaviours have been defined as “consumptive news feed curation” [40].

Hesitations among users to exercise agency in the form of disclosing preferences may stem from the internal trade-off Wadle et al. have reported on between users’ own data disclosure and the perceived benefits of personalised recommendations [73]. This trade-off is further supported by Torkamaan et al. who found that overly accurate recommendations resulted in systems being perceived as “creepy” [69]. A growing body of literature has also investigated users’ understandings of recommender systems, which have pointed at a diversity of views, attitudes, and perceptions as well as gaps and inconsistencies in the understanding of recommender systems [28, 53, 54]. Considering this variety of user standpoints on top of the internal trade-offs of perceived benefits and harms, the question becomes if there may be a behaviour-intention rift at play. This may manifest itself in, for instance, the trading of increased data and preference disclosure for more relevant recommendations – a scenario where actions may conflict with reported preferences.

Past work has sought to understand user perceptions of their own agency when engaging with personalised news through self-reporting, and found that users wished for more agency and transparency [48]. While participants had a self-reported intuitive understanding of algorithmic feedback loops, their frustration over lack of transparency suggests that their understanding of the mechanisms at play may be limited and thus only speculative. It is also this frustration over limited transparency that they reported that hinders them to be as actively involved as they wished to be in their co-curation of news recommendations [48].

2.2 Implications and Potentials of Transformed User Role

Giving users a means to influence the nature of the news content they receive, implicit or explicitly, raises potential risks in undermining editorial intent to curate balance. A large body of literature has explored the threats to democratic society arising from personalised news. The filter bubble effect, when algorithms curate content that aligns with an individual’s profile and content preferences, can result in an overemphasis of content that matches their world view rather than challenging it when needed [35, 56]. Other work has explored poor transparency in how “black-box” ANRs generate recommendations, making it difficult to hold these systems accountable for their outputs [20]. Such opaque decision-making in algorithmic dissemination of information can further information fragmentation, and consequently increase polarisation and the general demise of the public sphere [35, 50]. These concerns relate to long-standing arguments and debates around the role of media in society. As early as in the 20th century scholars have warned about an increasingly fragmented media landscape: Haberman’s (1962) concern was related to the increased commercialisation of media resulting in a fragmented public sphere. Negroponte (1996) warned about the dangers of over-customisation and over-personalisation in journalism [31, 32, 51].

Understanding these threats requires consideration of the purpose of ANRs when providing news in a democratic society. If likened to commercial platforms or social media, which through large quantities of data can provide highly accurate and resonating results (and therefore are at an advantage in the competition for user attention) [49], commercial success metrics may become “proxies to journalistic ideals” [23]. As such, it has been argued that ANR design must first and foremost preserve the societal function of journalism, however this requires revisiting the norms and values upon which this function is built [10, 19].

In a review of different types of democratic ANRs, Helberger pointed that different values and functions are highlighted in specific contexts. In relation to user agency, an inconsistency presents itself in wider social values and perceptions of the common-good, as when there is a strong interest for well-informed citizens, having the media be less responsive to users’ individual interests is considered better. However, when the social focus shifts to individual freedoms and self-development, the importance of user agency in curation processes is elevated [35].

This line of thought is furthered by considerations of the opportunities afforded by ANRs when centring the audience, they are meant to serve. As Usher (2010) describes it, data-driven models can help journalists and news organisations better engage with audiences and can turn journalism from “elitism of writing for itself and back to what people are actually looking for” [70]. In Helberger’s words, the heterogeneity of audiences in terms of their information needs, ways of processing information, interests, and preferences must be accommodated [35]. Møller points out that by employing personalised recommenders the value of content may even increase, as users can more efficiently discover relevant content beyond its regular lifetime where appropriate [50].

This appropriateness can be ensured through two means explored in literature: editorial measures and hybrid curation. Editorial measures include assigning maximum lifetimes to articles, numerical values to weigh the relevance of individual stories, having a safe pool of stories to be used for recommendations, and not recommending opinion pieces as they may polarise and lure into filter bubbles [50]. Hybrid curation is characterised by the emphasis of the editors’ role who can use ANRs as a tool for curation. This way, biases to which algorithmic aggregators are prone (such as popularity bias, where content is ranked higher due to its popularity among other users) can be better avoided. Instead, algorithms can be used as a tool for scalability, after editors capture the context of relevant stories, thereby remaining true to the journalistic values of diversity and novelty whilst offering accessibility for deeper dives into topics of interest [2].

Other works have tried to understand how other core journalistic values, especially universality, can co-exist with personalisation and increased audience curation and agency through analysing strategies of European public service media outlets. These strategies ranged from technological determinism, where universality is made to fit personalisation and concerns about social implications due to personalised dissemination are perceived as manageable hurdles on one side, to a social constructivist stance, where personalisation is bent to fit universality on the other [38, 71]. These stances however seem to perceive the role of recommender systems in a vacuum rather than in a collaborative or even reciprocal process. Contrastingly, more recent literature seems to agree that the human element of the editor is indispensable and cannot be removed or replaced [2, 17, 19].

Another important question to ask is about user perception and satisfaction in relation to personalised news, as Møller’s research with news practitioners revealed a mix of positive and negative attitudes towards ANRs. Negative attitudes were due to concerns about breaking user expectations regarding universality, however these concerns seemed to stem from “gut feeling and partly their own limited audience research” [50]. In order to explore a middle ground where these concerns are addressed while enabling users to experience the benefits of personalisation, studies are needed that examine user perceptions of such systems to validate these concerns.

Moreover, the above discussions predominantly centre on issues and risks related to how personalisation is achieved (comprising of algorithmic and personal curation) rather than turning the conversation around to explore if and how user agency can be leveraged to mitigate the risks of personalisation. As past research tended to largely reflect back on existing audience research by media practitioners, it cannot fully capture the possibilities awarded by personalisation [50]. To overcome this, we focus our research on enacting a scenario in which personalisation by an ANR is fully awarded, but also where users are provided exaggerated transparency over system decisions and the ability to override and contest decisions made on their behalf.

3.1 Preliminary Survey and Interviews

To understand user perceptions of media and news personalisation and associated comfort levels, we conducted a series of quantitative and qualitative investigations among a total of 211 participants in the UK aged 16-34. Participants were recruited through mailing lists at our University and through the paid recruitment service Prolific to target non-students and non-academics in an effort to recreate a representative population based on the 2021 UK census data on education levels [55].

The first online survey (n=106) aimed to identify which benefits and harms the participants commonly associated with media and news personalisation. Using open text inputs, multiple-choice, and Likert Scales as forms of inquiry, we identified which personal and wider social benefits as well as harms the participants commonly associated with media personalisation. This was followed by ten semi-structured interviews (using the same questionnaire) with participants who took part in the survey, which were used to add further context to their survey responses. Ultimately, we conducted a second online survey (n=105) to quantify sentiments surrounding items related to personalisation reported in the first survey and interviews (e.g., transparency, processing of past user behaviour, customisation, increased agency) by rating how beneficial or comfortable they were perceived on 1-10 scales.

This step of our research highlighted that the respondents desired increased agency, transparency, minimised data collection – specifically demographic data – without decreasing the quality and usefulness of predictions and recommendations. This work has recently been reported in an extended abstract [60]. This complements Monzer’s findings that users wished for more instances in which they can exercise explicit agency (personal curation), and would perceive increased transparency of how a system generates recommendations in terms of news content as a vital trust-enhancing measure [48].

By implementing these features in our provotype – albeit in exaggerated form in specific instances – this allows for an investigation of potential behaviour-intention gaps or paradoxical tensions between the self-reported data and observed user behaviour [39].

4.1 News Personality Types

A set of News Personality Types were constructed for users of the NAIRS provotype. A News Personality consists of four consumption characteristics, each of which takes one of a pair of values. Everyone can, therefore, be allocated to one of 16 distinct categories. This parallels the well-known Myers Briggs (MBTI) psychometric personality types, which is commonly criticised as pseudoscience, but has a wide popularity in social media and pop culture [22, 47, 58]. MBTI enthusiasts deliberately accept and identify with their personality types, potentially by engaging with content related to their respective types to create a sense of identity or even community [22]. This trend was also observed in the popularity of the end of year “Wrapped” feature used by the Spotify music streaming service in 2022, where users were assigned to one of 16 listening profiles, inspired by MBTI but adapted to feature characteristics relevant to their music consumption behaviour in the previous year [57]. Its popularity and engagement amongst Spotify users was demonstrated by widespread sharing by individuals of their listening profiles across social media. The validity users attribute to these profiles may stem from their understanding of the data collection and behaviour analysis that has taken place in order to assign them to an accurate listening profile. This can reduce privacy and data collection to secondary concerns, as users prioritise the perceived accuracy and utility they can get by relying on and attributing value to these profiles [36, 45]. Overreliance on computational assessments has been researched in the past, using a system that outputs personality and value profiles based on past Twitter and Facebook activities. Users reported their generated personality profiles to be uncomfortably accurate (“creepy”), and that they trusted the algorithmically enhanced and automated assessments of themselves more than their own self-assessment [74].

Applying a framework of this kind to News Personality Types allows for a further angle of exploration in our study; such as if users deliberately immerse themselves in a specific news community or epistemic bubble due to the sense of self-discovery associated with it; or if they reject a personality type and desire to change it, or even if they find the fundamental concept of having a news personality problematic. Including News Personality Types, and the capability for these to be altered or contested, could therefore help understand the level of agency users feel they need in being assigned to a particular segment of the news audience, and what their willingness is to engage or reject such approaches to personalisation in the news media landscape.

To design the NAIRS News Personality Types, we first identified a set of characteristics relevant to news media consumption that can be assessed through behavioural signals (Table 1). In order to identify these traits, already established news audience categories were analysed, such as the BBC World News audience segments [8] and the Reuters Institute’s segments of young news audiences [26]. Moreover, attributes of news media consumption from literature were considered, such as appreciation for diversity, wide interests, curiosity, serendipity, and tendency to get bored quickly [10].

The first category pair describes how a user receives the news, ranging from a proactive approach (actively seeking out news and intervening in the curational process, by for instance liking a story) to a more receptive approach of passive consumption. The second characteristic describes if the user consumes news from a variety of news outlets or is loyal to only one or two. Similarly, the third category describes if users have a tendency for exploration of a wide range of news topics or if they remain within what is familiar. And lastly, the fourth characteristic captures whether they are attentive (i.e., pay undivided attention by e.g., opening a news website and reading an article) or distracted when they consume the news (e.g., listening to news on the radio while commuting).

These characteristics often influence how news consumption takes place and can therefore be analysed as part of a behavioural personalisation system. In other words, there are clear signals from which attributes can be inferred (see section User assignment to News Personality Types, below), mimicking practices commonly used on personalised platforms and their interpretations of signals. We acknowledge the occasional mismatch that can occur in deriving attributes through signal analysis [37], however here the objective was to recreate common methods used for audience segmentations, and to emphasise some of this functionality for the purposes of prompting dialogue with participants.

The titles of the NAIRS News Personality Types are acronyms made up of the first letters of their characteristics. These are complemented by descriptive, potentially provocative, names, such as “The Serendipitous Expert” for RVFA or “The Loyal Scroller” for PLED, as well as short descriptions of their news personalities (Figure 1 and Table 2). Once again, these names were included to emphasise the playful and exploratory nature of the NAIRS provotype and mimicked the use of personality types by applications such as Spotify.

Figure 1:

View Figure

4.2 Personalised Newsfeeds

The personalised newsfeeds differ in the formats of the stories, their order, length, and featured news topics to match the NAIRS News Personality Types. For instance, for News Personality Types that have a distracted consumption behaviour (D), the unlocked newsfeed features more short-form content. Types with a preference for thematic exploration (E) are given newsfeeds with a wide variety of news topics (Table 3).

The process of unlocking the newsfeeds is not fully automated, as for users with a preference for familiar topics (F), it is not evident from their News Personality Type which topics they prefer over others, as it only captures the preference for the familiar. Therefore, for users with personality types including familiarity (F), newsfeeds are unlocked manually after inspecting the automatically logged interaction with NAIRS to identify their past preferences in terms of news categories to select the appropriate newsfeed featuring stories of their predicted interest. To avoid the same issue with the News Personality Types featuring loyalty (L) regarding news outlets, the outlet from which the individual stories of the personalised newsfeed are imported is not disclosed; instead, the newsfeed itself is branded with the NAIRS logo. The stories presented are all BBC stories across the genres, which were updated on a weekly basis, or more frequently, if necessary, to ensure timeliness and relevance throughout the duration of the user study.

Figure 2:

View Figure

4.3 Interactive Narrative Design

NAIRS is a fictional news recommender system, created to provoke reflection of different aspects related to news media personalisation, including agency, transparency, and audience segmentation. Due to its branching narrative design, it was built using the interactive narrative prototyping software Twine¹.

Participants’ first interaction with NAIRS (NAIRS Part 1) comprises ten questions about news consumption behaviour and preferences, followed by a sequence of news stories. Each story includes Skip, Like, Dislike, and Next buttons.

The presented purpose of NAIRS is to observe and assess user behaviour and assign News Personality Types and personalised newsfeeds. NAIRS users should understand the system as data collection for personalisation, which occurs once the user is assigned to their News Personality Type and the respective newsfeed is revealed. Through the interactive narrative design chosen (Figure 2) however, recorded preferences can instantly determine which stories are shown next, e.g., if users like a story of the Lifestyle News category, the story they see next may be in the same category. If they skip it, dislike it, or click “next”, they will be taken to a different category, such as Sports. Playthroughs can range from a minimum of six² to maximum of 18 stories³ , depending on the branching of the NAIRS experience based on user behaviour.

4.4 User Assignment to News Personality Types

At the beginning of the first NAIRS interaction, users are asked a series of true or false questions regarding their news intake (Figure 3) to learn more about whether they are “Proactive” or “Receptive” and also “Attentive” or “Distracted”. These news personality factors continue to be influenced by their subsequent behaviour with the NAIRS stories, thereby not solely relying on self-reporting.

Figure 3:

View Figure

After these explicit questions, NAIRS displays news stories sequentially. With each, participants choose to either read, rate, or skip (Figure 4). Stories are categorised into one of five topics (Politics, Lifestyle, Sports, Technology, Health) and are sourced from one of five news outlet from the United Kingdom (the Guardian, the BBC, the Daily Mirror, the Daily Mail/Mail Online, the Sun). The choice of topics and outlets is intended to reflect diversity of news media consumption behaviour by covering a wide range of news (sub)topics as well as editorial stances and tones: providers include a public service outlet (the BBC), popular tabloids with differing political alignments (Daily Mirror, Daily Mail, the Sun), and a broadsheet newspaper known for investigative and independent journalism (the Guardian). The choice of outlets was determined by market data on online usage and whether free access to articles of these outlets was generally granted, to avoid preconceptions or lack of opinion about an outlet due to inaccessibility [59].

Figure 4:

View Figure

NAIRS tracks user interaction with each news item, including which button was pressed to derive interest in the news topic (an interest in a variety of topics results in the “Exploration” trait) and interest in news outlet (a variety of which results in the “Variety” trait). This data can be used to adapt each user’s news personality profile (Table 4). Timing also plays a role, if users remain on a story for longer than five seconds (which is the time required to read the headline and identify the news outlet’s logo) NAIRS registers positive interest in the topic and the news outlet. Proactivity is derived from users’ intentional clicking of “like”, “dislike” or “skip”, whereas clicking “next” is considered more receptive. If users spend more than 15 seconds longer than expected reading time, they are considered distracted.

4.5 Overriding User Agency

NAIRS introduces an element of either algorithmic intervention or editorial intervention. Users can choose an alternative News Personality Type to the one they are assigned if they disagree with the NAIRS’ assessment. Their subsequent interaction with NAIRS (NAIRS Part 2) involves an assessment of their interaction with a sequence of news stories, as in Part 1. However, Part 2 contains an override function acting as a deviation detection. This provocatively displays an “Override” notification if the behaviour of the participant no longer matches their News Personality Type within a predefined threshold⁴. This feature is intended to make users question and reflect on their self-assessment and agency.

Although driven by an identical (hidden) calculation, for some participants this override is presented by NAIRS as being done by the recommender system’s algorithm. For other participants, the override is presented as the work of the fictional curating editor. Both editor and recommender system are understood as part of NAIRS, and free of conflicting interests.

4.6 Pilot and Changes

The NAIRS provotype was piloted with two expert participants with backgrounds in user experience design, to help ensure that its design had the desired provocative effects, whilst its functions were usable and legible. Reported weaknesses were then altered in the version of the NAIRS system used for the user study.

For example, in the pilot, override by the fictional editor was presented using the alert, “the editor in the loop is overriding”, employing a reverse Wizard-of-Oz method [15] in which the system pretends to be a person. This implication that an editor was intervening in real-time made pilot participants uncomfortable, as if spied upon. The alert was changed to, “the rules of the curating editor are overriding”. This suggests a computational calculation directly influenced by curating editors who assign rules to individual stories, placing the responsibility on the editor. In contrast, algorithmic overriding is described to participants as a calculation done by the system’s algorithm, placing the responsibility on the recommender system.

5.1 Participant Recruitment

The target participant group for this study are young adults, aged 18-34 based in the United Kingdom. Past work has noted that people in this age range are used to personalised content consumption, have a low expectation for diversity in their news intake and low appreciation of a shared public sphere compared to their older counterparts [10]. It can also be expected that trends in personalised media consumption among a younger age group may set the tone for the future.

In total, 16 participants were recruited using university mailing lists and flyers. In recruiting university students as participants, we acknowledge the potential limitations associated with focusing on this demographic group. However, this work aimed to investigate attitudes towards agency in personalised news systems, which people tend to use in their private lives, therefore for this specific inquiry, the background of the participants was not a decisive factor for recruitment considerations. Furthermore, university students and staff fall within the broad age-brackets defined by the study, therefore they presented as an easily recruitable sample. Once thematic saturation was reached the user study was concluded.

5.2 Study Protocol

The study protocol involves a primary interaction with NAIRS (Part 1) which assigns the initial News Personality Type to participants. This is followed by an interview which prompts reflection on how the assessment was conducted and allows for participants to voice (dis)agreement with their types. Following this, their respective newsfeeds are shown to them. Participants are asked to imagine this kind of newsfeed delivering their everyday news. Then, they are given the option to make changes to their News Personality Type before the next interaction with NAIRS commences. This provides important consequentiality within the enactment, as participants are able to make meaningful choices during an otherwise speculative activity [21]. By changing their News Personality Types (or not), they are willingly taking the repercussions of their (in)action into account, which would manifest in the second part of the interaction (NAIRS Part 2). Their choice to change their types (or not) is also discussed.

NAIRS Part 2 has two variations: apparently algorithmic and apparently editorial overrides. Half the participants are randomly assigned to each. Participants confirm again the News Personality Type with which they wish to proceed. Then, stories are presented in sequence. Once their behaviour no longer matches the chosen News Personality Type, users are notified that their type has been overridden (by algorithm or rules of the editor), however further details are omitted. To end the interaction, participants are shown the News Personality Type assigned to them after interacting with NAIRS for a second time.

The follow-up discussion explores how participants’ choice of News Personality Type and its overriding affects their sense of agency. Discussion is supported by showing participants the override screen they did not see (or in case no override happened, they are shown both screens). It is explained to them that while editor and algorithm do not have conflicting interests, only one of them determined the need to override their News Personality Type, whereas the other one did not.

5.3 Data Collection and Analysis

Interviews and interactions with NAIRS were audio and video recorded, and subsequently transcribed, to support analysis of participants’ responses. We employed an inductive thematic analysis approach [13, 14] to examine each transcript. In the initial phase, Author 1 identified codes within the data while Author 2 separately coded a subset of the data corpus. This was followed by discussion between both researchers to reach a consensus on the codes. Author 1 proceeded to recode the data before collaboratively identifying themes that brought multiple codes together. These were shared with the wider group of co-authors for discussion around agreements and disagreements and to form a narrative of the data presented through six themes.

6.1 Positive Attitudes towards Agency

Explicit agency (or personal curation) in NAIRS is present as the option for participants to alter their assigned News Personality Types between Part 1 and Part 2 of the interaction. This option to change resonated with the majority of the participants when asked about it. For some, having agency instilled a sense of comfort and trust, “Having more agency over it makes me more comfortable with it” [P5], “I’m like comforted and it makes me maybe trust the system more. More trust and comforted by knowing that I’ve got the choice to change it or not” [P10]. A sense of autonomy was also behind the appreciation for agency: “I think I have more autonomy in choosing the news that I want to intake” [P11], “It definitely makes me feel like I’m a bit in control of what is being shown to me” [P1]. Furthermore, having agency was also considered a “matter of principle, it gives people the opportunity to change the classification they’re given or object to it” [P9]. In their explanations of the value that they saw in being offered the opportunity to change their personality type, participants used the terms control and choice interchangeably with agency.

Some participants had a higher-level understanding of agency, considering themselves in control by making the decision to interact with the system: “I’m continuing so that’s an active decision that I’m making” [P12], “I can exit out of the system. So, for me, it either just gets it right or I don’t interact with the system” [P13], “It is up to the user to actually give a damn [about the system]” [P7]. This highlights that participants felt empowered in their intentional choice to keep or cease the interaction with the system.

Although participants greatly appreciated the opportunity of making changes, it was notable how this did not necessarily lead to alterations made to the News Personality Types by the participants. Only seven of the participants made a change to their type. Participants saw the opportunity of making aspirational customisations that reflect their ideal news consumption when in an ambitious mood in the future: “Altering it towards more what you want to be rather than who you are would be good” [P12], “I would always almost see this as like a kind of an aspiration. I want to strive to be a certain type of media consumer” [P9]. Other reasonings behind not changing the assessed News Personality Types were agreement with the assessment: “If I felt like something was really wrong, then I would want to change it” [P16].

The opportunity to interact with a different newsfeed was also considered widely: “I’m a liberal person politically, but I don’t want to not be exposed to anything that is conservative because I think it’s important to know about both sides, even if I don’t necessarily engage with them, I still want to have the option to read it to keep me more balanced” [P10], “It would be nice to have a way to mitigate you being too isolated. Someday if I just want to look at maybe an alternate page” [P2], “I’d change the way that I’m consuming news. Diversifying the way I receive it, but obviously trying to keep it within an acceptable range” [P5]. Similarly, P14 mentioned to actively browse for news channels they disagree with: “I think it’s good to have viewpoints that you disagree with. Just so you know what’s going on, right?” [P14]

6.2 Negative Attitudes towards Agency

Contrastingly, albeit less common, there were also mixed and negative attitudes towards user agency. These included three participants that criticised yet still exercised their agency by changing personality types [P2, P7, P8]. To them, having agency means having extra work: “Maybe initially I would try a bit but then after a few days… I won’t care, whatever it calls me is fine” [P2], “Ideally I would exert more agency, but realistically, it’s just not something that I have time for, which kind of puts a level of trust in others” [P8]. The other reasoning behind not necessarily wanting more agency was a desire to rely more on recommender systems for exploration, “It makes me not in control, but I’m okay with it because I feel like give me more to read, give me the suspense to see the behaviour changes. Maybe explore a new avenue of interest” [P7].

Reluctance towards exercising agency also stemmed from a worry that actively intervening and making customisations would reveal more about the person and add to the data the system can collect: “I’m just distrustful so I wouldn’t want to feed into it more than it already knows about me” [P15]. P13 also reflected they did not want to “mess up the rest of the experience” [P13] by making changes to the system’s assessment, knowing that their interactions with NAIRS are limited to a one-off occurrence, therefore not allowing for much user experimentation with the system, as with systems of everyday use.

Participants also seemed concerned about detrimental user behaviours such as doom-scrolling: “Reading about racist incidents is going to make me mad and is going to make me want to read more” [P13], “People are more attracted to, like, negative news but then blame themselves for getting depressed” [P6]. Behaviours as such would in their opinion become more frequent if users are given more responsibility over their news curation.

6.3 Knowledge about News Personality Type as Tool towards Self-Discovery

Once participants’ News Personality Types were revealed, a recurrent sentiment was a sense of self-discovery. Participants relied on the system’s assessment of the traits that make up the individual types: “That was based on my choices so it’s not like it’s judging me, it’s just measuring something” [P12], “This is what the system has assessed for me. And I want to trust that more” [P13]. This was largely due to the system’s analysis of user preferences and behaviour which participants considered to paint a more accurate picture about themselves compared to more commonly seen categorisations based on demographic data collection: “If demographic data is used in the algorithm, I think that would especially for me not make sense, because I usually don’t have the normal interests of an average 25-year-old” [P2].

While some argued that the result was similar to their own self-perception: “I think that it is actually consistent with my internal perception with myself” [P14], others used it as a tool towards self-discovery: “This system helps me know more about myself. I don’t know much about myself” [P3], “Maybe it knows you better than yourself” [P7].

Interestingly, after overriding occurred, regardless of whether users had changed their News Personality Type or not, many of them still agreed with the new assessment of the system, “The system doesn’t agree with what I assume as myself and thinks it knows better because it overrode it. I want to understand why. Because maybe sometimes I’m not quite aware of my news reading habits” [P11].

6.4 Temporal Changes as Justifications for System Decisions

Reactions to overriding of News Personality Types were predominantly justifications by participants trying to make sense of what triggered it, especially among those who kept their initial News Personality Types: “I’m not surprised because it had 10 minutes the first time to examine my behaviour. I would be surprised if I’ve got exactly the same result as before” [P10], indicating that the override occurred due to the system having had the chance to learn more about the user. P3 shared a similar sentiment, saying, “I think now this is like my real self” upon seeing the personality type assigned after the override.

Opinions on the overriding were predominantly positive with participants appreciating that the system accounting for temporal changes rather than questioning the validity of the system’s assessment: “Each time it overrides, it shows that it’s breaking down its own biases against me” [P1], “I think my interests vary with time” [P2], “People’s perception of news also varies on a day by day basis” [P5], “Your views can change over time, for me, it’s like in a few minutes” [P7]. This highlights the importance participants felt about systems being adaptive and allowing for fluidity in user preferences and behaviours. Further, the transparency awarded by the overriding’s visual presentation was perceived positively: “I will trust it more because of this information” [P3].

On the other hand, more disagreement with the system’s decisions was expressed upon seeing the unlocked newsfeed. Many users were quick to express that if a story did not resonate with them: “I’m surprised already that the sports story come up because that’s not something I’d have any interest in” [P8], “Tories [Conservative Party politicians], don’t care, skip” [P16]. This means that the agreement with News Personality Types was independent of the newsfeeds – even though they were juxtaposed to establish the relation between them, mostly due to the lack of nuance captured in the News Personality Types (e.g., no mention of specific news topics of interest or political alignments) as opposed to the newsfeeds.

6.5 Transparency to Exercise Agency

Being able to see their assessed News Personality Type and, in Part 2, overriding when it occurred, was mostly positively perceived as the system being transparent: “The overriding was like an awareness of like ‘oh, it’s learning something about me’” [P13], “It’s good because I always want to figure out how things are done. I always watched the magician’s hands” [P15]. These sentiments are particularly interesting, as the system only revealed what was happening, as opposed to how it happened.

Other participants however were more aware of this nuance and commonly expressed frustration over the limitations and selectiveness of the transparency of NAIRS: “It didn’t really say how it was happening or what caused it” [P5], “It to me didn’t make sense when it did the override. It wasn’t giving me any kind of reason why it was changing things” [P8], “It was just a matter of missing transparency. To me, I would have just liked to hear ‘alright in the last two articles you did this, that’s why we changed this’” [P9]. However, these sentiments are contrasted by users expressing no need to be as transparent: “I don’t need to know when it overrides” [P2].

Having transparency over the News Personality Type assigned to participants was perceived as a crucial step to give users agency: “It’s good to see why you’ve been put into that personality type and, maybe to contest it. […] I think for some people they would want to be able to say like no, this doesn’t fit with me” [P14], “Sometimes I’m put into a box that may not be completely accurate. Knowing about the box would be a tool for the first instance of you trying to get out of the box” [P5]. However, transparency was also considered crucial for explainability without the goal to facilitate change: “[It’s] not necessary to change it, but if it could explain why it was like that” [P12]. Consequently, this may indicate users’ wish to see how they are categorised in order to shed more light on the opacity of recommender systems.

6.6 User Role and Responsibility

Similar to the participants who associated having agency with having to do more work, some participants wanted to be less self-reliant: “Using filtering services you have agency but you’re still always relying on an algorithm to do things for you” [P8]. Others pointed more towards wanting to have agency but not shoulder the responsibility of their news curation: “I tend to think of the user as kind of like the receiver, and everyone else is the back end, so I just generally feel someone on the back end is usually more responsible because I’m not creating these tools, someone else is” [P2].

Further, some participants viewed themselves more as consumers who only exercise agency by either interacting with the presented content or not, rather than being involved in the curation: “I believe in the algorithm to do the job for me to personalise what I want to see” [P7]. This is contrasted by the majority of participants who considered themselves as users responsible for their curation: “We are adults. We do have to have some level of criticality” [P13].

Upon reflection of the different types of overrides, there were mixed opinions whether curational responsibility should lie more with human editors or the recommender system. Some participants trusted editors’ expertise, “I’m more comfortable with a human editor choosing my articles and editing my feed rather than the algorithm.” [P10], whilst others were worried about human bias: “The human has a motive and that is something that I don’t necessarily trust” [P12]. Similarly, some participants expressed more trust in recommender systems: “If the computer is doing it, even if at the end of the day the computer was written by a human, it’s still feels to me safer. It’s just data driven” [P15], however others pointed to the importance of retaining professional human judgement: “It should still be decided by the human editor or the user or some kind of a human in the process” [P9].

7.1 Attitudes towards Agency

7.2 Knowledge of News Personality Type as Tool towards Self-Discovery

On a wider scale, it is likely that a relatively high level of media literacy may correspond to a recognition that personalisation reflects a person’s beliefs, world views, and values, as users tend to believe to understand algorithmic feedback loops. [48] It therefore is not solely about the content they receive or do not receive; it is also about what it conveys about their identity. Upon revealing the personalised newsfeeds assigned to them, participants displayed a desire to removing items they disliked rather than adding content they wished to see. This inclination suggests that they associate inaccurate personalisation with encountering unwanted material rather than missing out on their preferred content. This may be because it is not just about the content itself but about the narrative it constructs about them (and to others). The self-reflection prompted by seeing recommendations (or how one was profiled) can make individuals feel sensitive and frustrated if they contradict or don’t align with their ideal self. Consequently, they aim to remove information that could negatively impact how they want to be perceived.

7.3 Temporal Changes as Justification for System Decisions

In each participants’ interaction with NAIRS 2 override occurred. This may be attributed to the deviation detection having been too strict, and therefore a critique to the design of NAIRS. However, participants were more likely to attribute these overrides to temporal changes, which were positively received, as it meant to them that the system was continuously learning about them and adapting to newly gathered data. This positive reception may further be explored as a design opportunity, where transparency can be awarded in ways that show users that the system is actively learning about them and making considerations about temporal changes in their interests, attention span and consequently information needs.

7.4 Transparency to Exercise Agency

While the transparency awarded through NAIRS was intended as part of the system’s provocation, participants did not feel provoked in a way that elicited negative sentiments upon seeing the way they were categorised. On the contrary, seeing their News Personality Types and the personalised newsfeeds that they unlocked as a consequence, helped them understand how they can use their explicit agency. Changing from one type to another was a straightforward process which did not involve the opacity usually associated with recommender systems, thus making participants feel more empowered in exercising their agency.

7.5 User Role and Responsibility in News Curation

Upon reflection on participants’ perceptions of themselves in a personalised news cycle, differing perspectives emerged, ranging from them perceiving themselves as agents to taking on the role of a consumer. Participants identifying as consumers still expressed a sense of empowerment in deciding how to interact with the system or engage with recommended stories, primarily through implicit rather than explicit agency [71], which they understood as passive consumption. This empowerment may stem from an awareness of the reciprocal nature of personalised systems which register implicit user choices such as their interaction behaviour. Here the differentiation lies in refusing to actively intervene and make their own explicit decisions. Further, participants held mixed opinions about the level of responsibility they were comfortable assuming in their personalised curation. Their views varied concerning their willingness to rely on editors or recommender systems, as well as their reasons for and against such trust. The ability to contribute to the process was seen as a factor enhancing system trustworthiness. This diversity of perspectives underscores the intricate nature of user roles and decision-making within personalised news environments.