User-centred quality of UI interventions aiming to influence online news commenting behaviour

ABSTRACT While HCI literature offers general frameworks for understanding user-centred quality, specific application areas may call for more detailed contextualisation of it. This paper focuses on socio-technical context of online news commenting by investigating speculative UI interventions intended to influence users’ emotions and social behaviour. To understand the aspects of quality that matter to users in such UI interventions, we conducted an international online survey (N = 439) and qualitatively analysed respondents’ first impressions of eight different design proposals. The findings describe contextually relevant socio-technical viewpoints and offer actionable considerations for design. For example, the findings imply that designers should be mindful of possible unintentional misuse that may result from the UI reinforcing specific emotional states or affording stigmatisation of individual users. The study advances understanding of which aspects of quality should be considered when designing and deploying UI interventions for digital media services and evaluating them with potential end-users.


Introduction
Understanding perceptions of quality of user interfaces (UI) can be regarded as one of the core agendas in HCI.The breadth of aspects that are seen to influence perceived quality has expanded over time because of new theories, empirical knowledge, and the application of information technology in new areas.For example, the conceptual expansion from usability to user experience in the 2000s (e.g.[Diefenbach, Kolb, and Hassenzahl 2014]) introduced factors like pleasure and playfulness to be considered in the design and evaluation of IT systems.Following this trend, the increasing agency of IT systems and, for example, the recent discussion on the ethical aspects of IT (Shilton 2018) call for continuous revisiting the essence of perceived quality.
This paper analyses potential users' perceptions and articulations of quality of UI intervention designs that aim to influence users' emotions and behaviour in digital media discussions.We focus on speculative, lowfidelity designs and the specific activity of commenting on online news.While online news commenting has been studied with ethnographic and descriptive approaches (e.g.Diakopoulos and Naaman 2011), applying research through design in this area is rarer with only few recent examples (e.g.Grön and Nelimarkka 2020; Kiskola et al. 2021).The proposed intervention designs draw from prior research that suggest the use of 'nudging' mechanisms to influence user behaviour (Fogg 2009;Seering et al. 2019;Thaler and Sunstein 2009;Wang et al. 2014) and in mitigating online incivility (Taylor et al. 2019;Topal, Koyuturk, and Ozsoyoglu 2016).In particular, prior research suggests that increasing online news commenters' reflexivity and emotion regulation could be helpful (Kiskola et al. 2021;Topal, Koyuturk, and Ozsoyoglu 2016).The question of perceived quality becomes apparent as the designs propose to influence a delicate form of social activity where they could simultaneously be seen as both desirable and ethically questionable, depending on the perspective and criteria.Qualitative understanding is important due to the nuanced viewpoints that this application area introduces to nudging UIs.
Knowledge about what makes nudging UIs good in general (e.g.Bovens 2009;Desmet and Hekkert 2007;Fogg 2009;Tidwell, Brewer, and Valencia 2020;Galitz 2007) may not sufficiently inform the design of systems in the specific socio-technical application area of commenting behaviour.Online news commenting features complex social interactions, mediated by a relatively simple digital channel, which may manifest as undesirable phenomena, such as hate speech, intentional trolling, and inconsiderate commenting, which may develop into hateful discussion threads (Chen and Margaret Ng 2017;Cheng et al. 2017;Eberwein 2019).Users, journalists, and other stakeholders have varying views on if and how comment moderation should be implemented in this context (Kiskola et al. 2021;Stroud, Van Duyn, and Peacock 2016).Mindful of this complexity, we suggest that presenting potential users various speculative UI intervention designs and qualitatively analysing their opinions could increase understanding of potential expectations and requirements for such UI intervention designs.The present study focuses on the aspects of quality that potential end-users pay attention to in this context.
We conducted an international online survey (N = 439) in which each respondent evaluated two out of eight speculative UI intervention designs.While the survey featured multiple quantitative questions and items, in this article we focus on qualitative data from two viewpoints.First, we inductively analysed the commenters' first impressions of the designs.We think this offers insight into the aspects users may pay attention to and therefore need to be addressed when designing and deploying such interventions.Second, we inductively analysed respondents' explanations as to why they preferred one of two designs they had viewed.We applied a socio-cognitive lens in the qualitative analysis: we note the respondents choose to select some aspects of the reality and make them more prominent, so that certain problem definitions, causal interpretations, moral evaluations and/or outcomes are favoured and promoted (Entman 1993;Lin and Silva 2005;Orlikowski and Gash 1994).For example, we examined which problem the respondents think the intervention aims to solve.The approach was utilised to form a nuanced understanding of what good quality means to the users.
The contributions of this work are (1) Descriptions of relevant user requirements for UI intervention designs for enhancing discussions in digital media.This includes contextually relevant viewpoints on quality and critical perspectives to deployment of such technology; (2) Preliminary design guidelines for UI interventions in this context.

Theoretical background
The following offers a theoretical background by covering topics like speculative design, user expectations, nudging, and media studies.Situating the work in relation to literature on speculative design and user expectations, Section 2.1 elaborates on the type of knowledge we sought with the analysis.Section 2.2 positions the work in the research on UI interventions and nudges, as well as outlines different ways to consider the quality of such systems.Section 2.3 focuses on the sociotechnical context of online news commenting, shedding light on the communal level requirements for design and arguing for the need for social interaction design.

Using speculative designs to create knowledge on users' expectations
The current design proposals (Section 3.1) can be considered as speculative and discursive by nature (Tharp and Tharp 2019).Speculative design proposals can be useful research tools if they elicit informative reactions from study participants (Baumer, Blythe, and Tanenbaum 2020).This kind of knowledge creation follows the broad approach of research through design where design thinking, processes, and products are used as a method for inquiry (Bardzell, Bardzell, and Hansen 2015;Zimmerman and Forlizzi 2014).Epistemologically, speculative methods and designing speculative solutions can provide insight into social problems (Auger 2013;Baumer, Blythe, and Tanenbaum 2020).Provocative artefacts can be used to elicit users' values for the initial research phase of a project to design acceptable products (Johannessen, Keitsch, and Pettersen 2019).Hence, it can be useful to show people solution proposals that are not designed to be instantly adopted and that are framed as speculative.Further, discursive design aims to encourage critical thinking about design (e.g. about what values and behaviours design embodies), often with the intention of initiating subsequent debate (Tharp and Tharp 2019).Our designs feature this motive, in addition to the problem-solving motive.
The present work investigates the question of what quality means to the potential users in the context of speculative artefacts.Designers often see traditional measurement and evaluative techniques as inappropriate when developing new products that are not yet in existence (Suri 2002).Suri (2002) has argued that measurement, by its nature, forces designers to ignore all but a few selected variables.Hence, using well-established measures like AttrakDiff (Hassenzahl, Burmester, and Koller 2003), System usability Scale (Bangor, Kortum, and Miller 2008), or NASA Task Load Index (Hart 2016) would be misleading if the designers are not confident about which variables are relevant (Hart 2016).At the same time, we acknowledge that users may not accurately recognise their needs or wishes are regarding speculative products (Heikkinen, Olsson, and Väänänen-Vainio-Mattila 2009;Yogasara et al. 2011;Orlikowski and Gash 1994).
In other words, this study seeks to understand the users' assumptions and expectations that affect the acceptance and adoption of technology (Orlikowski and Gash 1994).We inductively analyse respondents' reactions to the artefacts, and their argumentation regarding the quality of the artefacts.Hence, the work is related to studies of anticipated user experience (AUX), while, at the same time it is more speculative and explorative and less about measuring than typical studies of AUX (Olsson et al. 2013;Sánchez-Adame, Urquiza-Yllescas, and Mendoza 2020;Yogasara et al. 2011).We follow a similar approach as Bonino and Corno (Bonino and Corno 2011) who explored user expectations of smart homes of the future in Italy with a qualitative online survey.While the present study focuses on a different application area, the studies are analogous in that the participants know the environment where the technology would be implemented (home environmentonline news commenting environment) and have expectations based on that knowledge.Like Bonino and Corno, we used an online survey to collect a broad sample of data and analysed answers to open-ended questions.

Conceptualising the motivational aspect of our design proposals
The literature features many ways to think about and name designs that aim to influence user behaviour: for example, persuasive design (Fogg 2009), nudging towards certain unconscious selections (Thaler and Sunstein 2009), and friction to hinder certain unwanted behaviours (Cox et al. 2016).In this broad conceptual landscape, there seem to be various interpretations about the termsfor example, what counts as a 'nudge' (Caraban et al. 2019;Zimmermann and Renaud 2021).To avoid terminological clashes and misunderstanding, the designs in the present work are simply termed UI interventions.To elaborate, we believe the term 'nudge' would take too strong a stance on the strength and pervasiveness of the intervention at this stage of design and in this design context.For example, an often-cited example of a 'nudge' is a traffic sign displaying a sad or a happy face depending on whether the driver obeys the speed limit (Weinmann, Schneider, and vom Brocke 2016;Zimmermann and Renaud 2021).In our design context, online news commenting, the user knows there are moderators (c.f.police) behind the intervention.As the user is not isolated from other people, the user might feel more than merely 'nudged'.Further, our designs can be perceived to intervene in naturalistic behaviour in digital media, which is another reason for calling them interventions.
Good persuasive designs match the user's level of motivation and ability to act (Caraban et al. 2019;Fogg 2009) and are transparent (Bovens 2009).They support the user in acting in accordance with their overall preference structure (i.e. with their conception of the right thing to do in a given situation) (Bovens 2009;Sunstein 2018).However, should the user consider what interventions the other users might need, they might accept an intervention that is excessive compared to their personal needs.This further motivates us to study what potential users think about intervening in online commenting behaviour.
Little is known about users' perspectives on intervention designs utilised on online forums, news commenting platforms, and social media in general.While there is knowledge on how users perceive various content moderation strategies across various platforms (Cook, Patel, and Wohn 2021), it does not focus on the UI designs.A few recent studies investigate perceived benefits and drawbacks of guiding social media users or news commenters to stop and think before posting (de Carvalho, Olsson, and Kiskola 2021;Wang et al. 2014).Linhares de Carvalho et al. ( 2021) interviewed 18 university students about their perceptions of four proposed UI mechanisms for guiding users to emotional self-reflection when reading and commenting news articles online.The interviewees commented about the ease of use, usability, usefulness; feeling of control, censorship, intrusion; an unintended consequence of angering users; and level of trust towards the service.The study concluded that users do not want an intervention to interfere with fast-paced interaction in online news commenting.A study by Wang et al. (2014) featured two 'privacy nudges' to Facebook posting.Twenty-eight participants installed them as web-browser add-ons and used them for six weeks.The researchers discussed several perspectives to user-centred quality in the nudges: intrusiveness of the nudge; a sense of being watched or judged; control or customisation of the nudge by users; and usability and reliability of the nudge.The study concluded that 'privacy nudges' have great potential to assist users in avoiding unintended disclosures.Inspired by these two studies, we aim to explore user expectations and perceptions with larger samples of participants and with a larger number of design proposals.

Improving online news commenting as a socio-technical and systemic design problem
Online news commenting can be considered a sociotechnical system (STS) where people communicate with others through technology and their behaviour emerges rather than is dependent on technology alone (Whitworth 2009).For example, individual user's commenting behaviour depends on other users' earlier comments, the semantics and emotional associations related to the news article, their attitudes towards the topic, and the interaction affordances and conditions introduced by the discussion platform.This makes it difficult to predict how even a simple new design would be appropriated.
To further illustrate the complex nature of the problem, we apply the web of system performance model proposed by Whitworth & Zaic (Whitworth 2009;Whitworth and Zaic 2003), which has been used in information systems evaluation (Isaias and Tomayess 2015).Following the model, at the level of software, increasing the rule-based functioning of an intervention to commenting can decrease its ability to respond to environmental changes, and vice versa.At the human level, increasing the intervention's predictability can decrease its flexibility and vice versa.At the corresponding communal level, increasing the amount of order an intervention imposes on commenters can decrease their freedom.Other tensions the model proposes at the communal level are creation of benefit by social interaction (synergy) versus lack of social conflict (morale); respecting the right to be shielded (privacy) versus enabling everyone to easily see what is going on (transparency); and letting new people and ideas enter (openness) versus preventing ideological hijack (identity).To summarise, improving online news commenting can be difficult because it requires accounting for multiple charged perspectives to its quality and at multiple levels (e.g.communal, human, and software [Whitworth 2009]).A narrow focus on a single perspective or level can cause problems to pop up elsewhere (Alexander 1964;Whitworth 2009).For example, even a solution that seems to improve the quality of commenting without incurring any obvious costs might do so at the cost of human connectivity.
The socio-technical level and communal requirements are worth stressing as HCI and Design have long focused on the perspective of the individual (cf.human level [Whitworth 2009]).Designers should take responsibility as 'shapers' of society and not hide behind the needs and wishes of the consumer (Tromp, Hekkert, and Verbeek 2011).Further, existing conditions are often framed as problems and technological systems as solutions (Baumer and Silberman 2011), which is unhelpful when elimination of the problem is unlikely (Baumer and Silberman 2011) or the problem is socio-cultural.For example, as noted by (Sparrow, Gibbs, and Arnold 2021), 'the goal of completely eradicating incivility is unfeasible and unreasonable'.Rather than imagining that a technology design offers solutions to extremely difficult problems, Baumer and Silberman (2011) suggest thinking of design as an intervention in a complex situation.
We found few articles with guidelines or principles for designing for online social behaviours, sociability, or social interaction.Of these, we want to mention Adrian Chan's 175-page explorative essay Principles of Social Interaction Design (2012).While the essay focuses mostly on social networking sites, it also offers general suggestions for social interaction design that seem applicable in this context.For example: anticipate the social practices that will emerge, consider who will be attracted to using the service, and who these users will attract in turn.Overall, according to Chan, good social interaction design accounts for the diversity of user experiences and for the development of a social tool over time.

Methodology
Following an explorative design process, we ran an international online survey to qualitatively analyse perceptions and opinions of people who at least occasionally comment on news on online news sites.The overall setup follows a common methodology where surveys are used to collect qualitative data with openended questions.Similar methodology has been applied, for example, in studies of user perceptions towards data disclosure for cognition-aware e-learning (Herbig, Schuck, and Krüger 2019), towards smart energy consumption metres (Jakobi et al. 2019), and towards augmented reality scenarios at early stages of technology development (Olsson et al. 2012).The use of an online survey allowed us both to invite viewpoints from a diverse sample of potential users and avoid the risks of real-world testing like failure to predict negative consequences (e.g.discouraging diverse discussion and supporting trolling) of intervention designs in the social context (Kiskola et al. 2021).The survey was implemented with LimeSurvey and the participants were recruited via Prolific, a platform for online research participant recruitment (Palan and Schitter 2018).To select a diverse sample of participants, we first conducted a short pre-survey regarding how often the candidate respondents read and commented on online news articles.The actual design survey asked participants about their behaviours and attitudes related to commenting on online news sites and invited them to evaluate two designs selected out of the eight design proposals.In this paper, we focus on the qualitative data from answers to two broad open-ended questions as they were likely the best ones to reflect the respondents' ways of thinking.

Designs and scenarios in brief
The following summarises how the eight UI intervention designs were created and what the related scenarios of use are like, to elaborate what kind of artefacts the analysed perceptions of quality relate to.Only a summary is provided as the designs are not intended as a novel contribution per se in this paper.The designs are intended as propositions of possible future UIs, inviting the reader to assess their meaningfulness and speculate on the possible implications.The design work for this study builds upon our earlier research-through-design exploration (Kiskola et al. 2021), in which we envisioned unconventional solutions to the problem of uncivil commenting.In the study, we unpacked this same problem area and outlined critical perspectives on potential solutions by describing and analysing four designs that aimed to support emotion regulation by facilitating selfreflection.Next, we briefly recap the design process of the earlier study: 1. Existing design conventions were identified by analysing social media platforms and news websites.This was done to find a convention to be tweaked slightly, to avoid reinventing existing solutions, and to reflect on what kind of solutions might fit various news websites.2. Approximately 60 concept ideas were sketched based on several idea generation sessions.Two general strategies mentioned in literature on critical design were used: (1) the designer picks a literary device (e.g.irony, sarcasm, parody, or ambiguity) and attempts to implement it in designs (Johannessen 2017) and ( 2) the designer picks a convention (cultural or UI) and tweaks it slightly, for example, by introducing a foreign concept, and then reflects on the result (Bardzell, Bardzell, and Stolterman 2014).3. 19 of the sketched ideas were subjectively evaluated by the design team as more promising in terms of perceived criticality, novelty, feasibility, and effectiveness.Following this, the first author created UI mock-ups of the 19 ideas.Also, four of the 19 mock-ups were pictured and analysed in depth in the earlier study.
Eight of the ideas that represent a rich breadth of approaches to support self-reflection and emotion regulation in online discussion were chosen for the survey.The ideas were further developed and made more presentable.The eight designs utilise several different 'emotion strategies' that Yoon et al. (2019) propose may be used in designs, such as suppression and avoidance.Also, the interventions are proposed to take place at different moments of use: before reading comments, while reading comments, while writing a comment, and/or after sending a comment.In addition, we subjectively assessed the designs as conceptually different from one another.
The names of the designs are EVALUATE, CREATURE, HIGHLIGHT, SYMBOLS, AUDIENCE, REGRET, PHILOS-OPHY, and WARNING.For a full illustration of the designs and scenarios, see Appendix 1.
In the EVALUATE design (see Figure 1), the user must first indicate how they feel before they can add their comment.This is done by clicking a smiley face that represents their emotional state.The design aims to make comment writers more aware of their emotions.The design is inspired by and applies the theory of affect labelling (i.e., putting one's feelings into words) (Torre and Lieberman 2018).
To illustrate the design scenarios, the EVALUATE scenario was described to the respondents as follows: 'You are reading the comments to an interesting but divisive news article … and wish to add your own comment'.(A couple of comments created by us are shown for illustration purposes).'When you click "Comment", you first need to tell how you feel before adding your comment'.
In the CREATURE design (see Figure 1), an animated pet dog reacts to the emotional tone of a comment while the user is writing the comment.The benefits of using emotional attachment to pets to motivate behaviour change have been documented in previous research (e.g.Dillahunt et al. 2008;Lin et al. 2006).The pet dog is displayed below the text area, and it is described as 'our digital friend'.If the user writes positively, the pet dog appears happy, as if ready to play.If the user is writing neutrally, the pet dog appears neutral (see Figure 1 top right).If the user is writing negatively, the dog communicates submission or fear.The design aims to motivate comment writers to consider their tone by giving feedback about it.
In the HIGHLIGHT design (see Figure 1), the user is offered an option to view an automatic analysis of the emotions in the comments.Negative emotional expressions would be highlighted in red, and comments containing strong negative expressions would be marked with an alert symbol.The design aims to make users more aware of the emotional expressions and to take a more analytical approach to reading comments.This design is also inspired by the theory of affect labelling (Torre and Lieberman 2018).
In the SYMBOLS design (see Figure 1), the user is offered a way to provide anonymous, private feedback to any of the previous commenters.This is intended to decrease the likelihood of written personal attacks toward other commenters.It has been demonstrated that uncivil comments (including replies) promote further incivility (Chen and Lu 2017a; Ziegele et al. 2018), and that ad hominem attacks are a frequent type of incivility online (Coe, Kenski, and Rains 2014; Maia and Rezende 2016).In the design, there are buttons depicting a bomb, a gavel, a smiling face, and a heart next to every comment.The bomb symbolises 'Full of arrogance'; the gavel 'False claim/s'; the smiling face 'Well said'; and the heart 'Love it!'Also, every user's profile contains a prominent section entitled 'Overview of the feedback from other users', which displays the same symbols and the number of times the user has received these feedback types.The design aims to motivate comment writers to consider the quality of their writing and to guide the other users away from writing uncivil replies.
In the AUDIENCE design (see Figure 2), when a user is writing their comment, a virtual audience of expert judges reacts to its tone in real-time and their reaction is displayed below the text area.If the user writes in a moderately positive way, some members of the audience appear glad, and others have a neutral expression.If the user writes in a rather negative way, most members of the audience appear angry or frustrated.The design aims to motivate comment writers to consider their tone and who they are writing for.The audience's appearance in the proposal is also intended to communicate that the audience is ethnically diverse.Previous research has found that showing Facebook users profile pictures of people who will see (c.f.judge) their posts can help some of them avoid regrettable  disclosures (Wang et al. 2013).Also, the AUDIENCE design utilises the concept of being watched to induce self-awareness (e.g.Bradley, Lawrence, and Ferguson 2018;Cañigueral and Hamilton 2019).Previous research implies that designs that induce self-awareness might reduce abusive comments to news (Sohn, Chung, and Park 2019).
In the REGRET design (see Figure 2), users' comments are automatically evaluated immediately after posting.If a comment sounds very angry, the user is notified and offered various follow-up actions below the published comment and by email.The user is offered options to regret the choice of words, to delete the comment, or to edit it.If the user chooses the regret option, a notification is attached to the comment, stating 'username regretted their angry words'.The design aims to motivate commenters to reconsider the emotional quality of their comments and provides a new affordance to show regret.Previous research has found that postings with profanity or obscenity can be a cause of regret for Facebook users (Wang et al. 2011).
In the PHILOSOPHY design (see Figure 2), problematic comments and comment threads are marked with a university icon providing subtle affordance to view analysis of the comment.If the user presses the icon, a box with the emotion score for the comment or comment thread and a quote from Socrates, 'Know thyself!' (Xenophon et al. 1979) is revealed.The emotion score has two dimensions, positivity and calmness.The design aims to motivate comment writers to consider the emotional quality of their comments and to enable other users to skip reading comments or alternatively to analyse the comments' emotional qualities.
In the WARNING design (see Figure 2), a notification is shown above the comment section, indicating a description of the argumentation within the comment section (e.g.'10% Hatefulness').The design aims to make users aware of emotions in comments, to use a more analytical reading approach, and to allow a choice whether they want to read the comments.The design is mainly inspired by the theory of affect labelling (Torre and Lieberman 2018).

Participants and recruitment
The pre-survey deployed in Prolific involved 2000 voluntary participants who met the specified eligibility criteria: fluency in English, normal or corrected-to-normal vision, and a minimum approval rate of 70% in Prolific (percentage of total submitted studies minus returned).
The key criteria for inviting the pre-survey participants to take the design survey included having given complete answers and commenting at least occasionally on online news sites.Furthermore, because we wanted to focus on news sites that have commenting sections, respondents who had mentioned some of the following sites as their main news sites were not invited to take part in the design survey: Facebook, Twitter, Reddit, Quora, YouTube, various blogs, and news aggregators where we could not find comment sections.That is, only responders who mentioned news publishers' sites, sports news sites, gaming news sites, or alternative news sites were invited.Based on these inclusion and exclusion criteria, altogether 480 Prolific users were invited to take the design survey.
Next, we briefly describe how the respondents were introduced to the design survey and the main parts of the survey.The survey study in Prolific was entitled 'Survey on improving discussion around online news articles'.The study description stated that it asks about the behaviours and attitudes related to commenting news on online news sites.Furthermore, respondents were told that two UI proposals will be shown as speculations of how discussion around online news articles could possibly be improved or kept at a good level.
Of the 480 survey responses, 41 were discarded as incomplete (i.e.missing answers), or duplicates (i.e. the same person completing the survey twice), or click-throughs (i.e.response times two standard deviations below the mean, or nonsensical answers to open questions).Of the 439 respondents with valid responses, 45.3% reported being females and 54.7% males.The respondents' age range was 18-75 years (average 33.5 years, SD = 11.98).43.3% of them were from the United Kingdom (UK), 12.1% from Poland, 10% from the United States (US), and the rest 34.6% from altogether 36 other countries.All respondents reported to comment on online news sites at least occasionally.

Survey procedure and questions
Out of the altogether eight speculative UI intervention proposals, each respondent was shown two pseudorandomly selected designs.Pseudo-randomisation was used instead of true randomisation to ensure that all eight designs were presented an approximately equal number of times in the sample.The two designs were then presented to the respondent in a random order.The designs and the associated scenarios of use are described in Section 3.1.Immediately after presenting a design, the respondents filled in a mandatory open-ended question (analysed in this paper) and several other, mostly closed-ended questions (not discussed in this paper).The open-ended question was, 'How would you describe your immediate reaction to this solution?How do you feel about it?'Furthermore, after they had evaluated both designs, another mandatory open-ended question was presented (analysed in this paper): 'Now, consider the two different solutions that you saw: X & Y. Which of them you found as the better solution for improving the commenting culture on online news?Why?' We focus on these two open-ended questions as the answers likely reflect the respondents' own way of thinking about the designs, which is what we are interested in this study.

Data analysis
We qualitatively analysed the responses to the two open-ended questions: first reactions to designs and explanations for the choice of the better design.The average number of characters in the responses were 175 (standard deviation 148) and 100 (st.dev.91), respectively.
We followed a data-driven explorative analysis informed by the socio-cognitive analytical lens of technological frames (users' assumptions, expectations, and knowledge) (Lin and Silva 2005;Orlikowski and Gash 1994).It was kept in mind that people generally choose to emphasise some aspects of reality, so that certain problem definitions, causal interpretations, moral evaluations and/or outcomes are favoured and promoted (Entman 1993;Lin and Silva 2005;Orlikowski and Gash 1994).Open and axial coding was conducted to highlight themes from the data and to build a hierarchy of categories.This and comparable coding methods have previously been extensively utilised to understand user expectations of new technologies or applications (Jakobi et al. 2019;Nicholas et al. 2017;Olsson et al. 2012).The responses were read and coded one at a time (i.e.given short words or phrases that describe the meaning of the responses [Saldaña 2013]).When reading the comments and coding them, the coders paid particular attention to the following aspects of the responses: (a) how the responses described the designs, (b) how the respondents described their reactions to the designs, and (c) what kind of vocabulary was used in the responses (e.g.style, tone, length of the response).
The codes were then further abstracted into categories presented below in Section 4. The categories were generated by abstracting out existing codes and by developing new concepts that encompass several of them.When reasonable, lower-level categories were generated to describe respondents' assumptions and expectations in more detail (e.g. a category of helpfulness could be elaborated by considering who the helpfulness is directed to and for what reason).
In the end, the number of answers matching each category was counted.The quantifications are meant to be inferred merely as indicative; we argue that the contribution of the results lies in the diversity and qualitative descriptions of the identified themes and categories rather than in the quantity of the responses per category.New viewpoints and nuances to quality, and critical perspectives to the deployment of technology are valuable as far as they are meaningful, regardless of how many respondents provide them.
As the questions were open and the answers varied, the first author, who was primarily responsible for creating the designs, collaborated with two researchers to classify and quantify the data.He coded the data using Microsoft Excel and created preliminary classifications, then met with the other researchers to refine the codes and categories.Additionally, the third author rated 50 randomly selected responses twice and the ratings were compared to those given by the first author.While we had some disagreements about the ratings, they primarily resulted from ambiguity of the answers.Overall, we engaged in a highly iterative process where the individual codes and their interrelations were gradually clarified.However, analysis of interrater reliability was not seen useful as the findings do not hinge on frequency counts.The participant quotes presented in the paper are verbatim except for some corrected typos.

Findings
The following reports selected findings on the respondents' impressions of the design proposals.While the survey data features diverse perspectives, we focus on categories we found qualitatively most interesting and specific to the context of online news commenting, hence offering nuanced perspectives to user-centred quality in this area.In other words, the presented categories are not necessarily the most frequently identified in the data.In total, 274 first impressions and 285 arguments for the choice of the better design fell into at least one of the code categories presented in the following sections.The other categories of the first impressions (omitted from this report) contain, for example, short emotional reactions (e.g.'angered', 'I think it's great!') and general comments about good or bad style of design.Those for the choice of the better design contain, for example, expressions of uncertainty, considerations of the ease of use or familiarity, and vague or unclear answers.Each subsection heading represents a relevant high-level category that is represented by several categories identified in the analysis (marked with cursive typeface).

Respecting freedom to comment
Many respondents appeared to feel that some of the designs would restrict users' freedom.In their first impressions of the designs, 32 respondents appeared to refer vaguely to restriction of their freedom.In the choice task, 46 respondents found the chosen design better because it was vaguely perceived to restrict their freedom less than the other design.For example, in their first impression, a respondent wrote that CREA-TURE is vaguely restrictive: It seems quite patronizing.I don't see why comments should be moderated to be positive.(From UK, comments monthly on news sites) Some respondents appeared to think that a design can represent censorship.In their first impressions of the designs, 14 clearly referred to censorship (i.e.intentional suppression of speech).In the choice task, 28 specifically stated the chosen design leaves more space for free expression.For example, a respondent had the following first impression: On the AUDIENCE: 'People should be allowed to express their opinion regardless of what it is.Failing to do so is asserting some sort of control on people's opinion and will be pathetic for democracy'.(US, comments monthly) However, to illustrate how difficult it is to judge what is and is not restrictive at this stage of design, we quote two opposing arguments from the choice task.A respondent argued the PHILOSOPHY to be better than the AUDI-ENCE because they perceived that PHILOSOPHY is neutral: In contrast, another respondent argued the opposite: that the AUDIENCE is better than the PHILOSOPHY because the latter represents censorship: I think it is better to get people to think before they comment than it is to censor it after it has appeared.(US, rarely comments) In other words, the answers in this category imply that high quality is marked by a capability of the intervention design to manage a balance between restriction of speech and promotion of civil discussion.Furthermore, we argue that emphasising free speech might be contrary to the wishes of those users who want more moderation.

Objectivity in assessing comments
In their first impressions, five respondents appeared to ponder the question of who decides whether a comment is problematic.The respondents appeared to indicate that they are doubtful that the proposed evaluators (other users or an algorithm) would evaluate the comments objectively.For example, a respondent asked the following in their first impression of the REGRET: Who on the newspaper is the arbiter of what constitutes anger, and when it is justified?(UK, comments weekly) In the choice task, 13 respondents said the chosen design is better because of the trustworthiness of the actor/s who evaluate the text.For example, a respondent argued that the REGRET is better than SYMBOLS because they perceive machines as incapable of giving biased feedback: It analyses wording of the comment not the meaning of it.In the 'Feedback' solution it's the people who decide what feedback should they give, and they can give you a bad one just because they disagree with you on the topic -not because of your wording.(Poland, comments daily) In sum, this implies that high quality is indicated by having actors that the users can trust as those who judge and moderate the comments.While the responses offered little guidance on what would increase trust, they implied a need for evidence that the intervention appears objective in assessing the tone of the comments.

Helping various users behave better
Most of the respondents who appeared to perceive uncivil commenting as a problem, appeared to think that the interventions are meant to help prevent users from accidentally or unintentionally behaving in a way that can come across as uncivil.More broadly, they thought that news commenting can lead to emotionally stressful situations.In their first impressions, 94 respondents appeared to say that the design would help to improve the quality of commenting of nonspecific users (e.g.respondent referring to 'the writer').Also, 51 indicated that they personally have challenges and need help.A few respondents indicated that they are not the ones needing help.12 respondents argued that a specific user group would need help (e.g.users who are easily 'triggered' or what they referred to as 'troublemakers').In contrast, 35 respondents argued that the design would not stop an irritated user, and some argued it would not stop someone who irritates other users on purpose (9).
To illustrate the point of helping the user to avoid accidentally or unintentionally behaving in a way that can come across as uncivil, a respondent had the following first impression of the REGRET: The solution is actually great.It gives me the opportunity to think about the consequences of my choice of words and be able to make the necessary corrections.(UK, comments daily) Also, to illustrate the closely related point of helping the user to avoid getting into emotionally stressful situations, a respondent had the following first impression of the SYMBOLS: I think it's a good solution to show that I do not agree with this and not to enter into unnecessary discussions with the author.(Poland, comments daily) The respondents' arguments in the choice task coincide with the first impressions.In the choice task, 169 respondents appeared to argue that the chosen design is better because it is more effective or helpful.As an example of this helpfulness argument, a respondent argued that the CREATURE is better than the HIGHLIGHT: Animated creature might be a big help for people to not be misunderstood by using wrong choice of words.It can also make comment section more civilized where people instead of swears might use more cultural way to express their opinion or critics.(Poland, comments monthly) In contrast, 21 respondents appeared to argue that the chosen design is more effective as it is more forceful or restrictive.For example, a respondent speculated that the AUDIENCE would be more effective than the WARNING: Easier to appreciate, the visual effect is more shocking and therefore will be more effective.The arbitrary percentages of the latter just seemed too random.(UK, comments daily) Summarizing these categories, high quality seems to be indicated by the design helping the users to avoid getting themselves into emotionally stressful situations.Still, for some users, good quality means that an intervention design also must be able to deal with those who intend to be offensive.Further, the fact that 94 respondents referred to nonspecific users could be an instance of the 'third-person effect' (Phillips Davison 1983): many believe that they personally would not be influenced, while other people would.

Use of apt metaphors
In their first reaction, 15 respondents appeared to associate the metaphors and manifestations of mimicry (i.e.copying properties of familiar objects, organisms, or environments) in the proposed designs to their usefulness.The answers to the choice task also suggest that users may prefer metaphors and mimicry that matches their personal taste, values, and contextual expectations.15 respondents argued that the chosen design is better because it fits the serious use context better, and 9 argued that it is better because of being more playful.However, as these two criteria could be considered contradictory, they reflect the variety of possible tastes that people can have.For example, the following two responses take contrary views on the CREATURE: It is a very creative and worthy solution, almost everyone feels empathy with dogs so it might be effective.I feel empathy towards the dog, so I'd change my comment if it were sad.(Portugal, comments weekly) This is not relevant to the posting of comments.I feel it downplays the issue of what impact your comments have and is almost more suited to children rather than adults.(UK, comments monthly) The first response above seems to interpret the sadness of the pet dog as a metaphor for human suffering, while the second seems to interpret it more literally as a pet dog.
Further, to illustrate how mimicry in a design was connected to usefulness by some of the respondents, a respondent commented on the AUDIENCE: It makes judgement more human; it seems people are closer to me; I can understand their feelings better.(Italy, comments monthly) To this end, high quality and usefulness appear to be indicated by the applied metaphors and/or concepts matching their personal values.Creative use of metaphors might play an important role in supporting some users' reflective and empathetic thinking.

Avoiding risks of intentional misuse
In their first reactions to a design, 28 respondents spontaneously considered how the design could be purposefully misused to hurt other users.In the choice task, 16 respondents argued the chosen design is better entirely or partly because it is less open to misuse.This means that some respondents not only noted that the design could be intentionally misused, but also argued that it is important that intentional misuse is actively discouraged or prevented.This general category of expected intentional misuse comprised three more specific perspectives explained below: 4.5.1.Some users would seek to receive negative scores This perspective illustrates a downside to giving negative feedback to uncivil commenters: it may encourage further incivility, for example, due to a sense of being provoked or a will to explore the boundaries of the scoring system.This was expected of every design where the system was proposed to explicitly evaluate or grade the user's comment.For example: On the AUDIENCE: 'in general I like the concept of it; however, it is open to interpretation depending upon the article -the article may generate a negative opinion which means people reply with a slightly negative attitude and it may only serve to encourage some people to carry on their comment further if they see it is generating a response that will gain replies by being overly negative'.(UK, comments daily) 4.5.2.The users would show that certain views are not welcome Some respondents were concerned that an option to quickly give anonymous negative feedback to a commenter can be used to send the message that certain views are not welcome, for example: On the SYMBOLS: 'I think it's a good idea, but people could give [the commenter a label of] full of arrogance only because they don't share their opinion'.(Ecuador, comments weekly) On the SYMBOLS: 'Sometimes I comment on articles from other newspapers with very different views (e.g.Daily Mail), even though I know my comments will get downvoted, just to show them that some people think differently.But I would be a bit upset because I know that on that news site, my profile would get a bad rating, purely because my views differ from most of the readers'.(UK, comments weekly) 4.5.3.Bullies would target the users who stand out This concern applied to the designs where individual comments are marked as different from others.For example: On the REGRET: 'I feel like comment readers might start bullying those people who have a label of regret and create even harder conflict'.(Lithuania, rarely comments) These categories imply that high quality is indicated by preventive actions (or assurance thereof) that minimise intentional misuse of the intervention.The design proposals featured indirect suggestions, and it became evident that the users might react to the suggestions in unintended ways.Many expected behaviours like 'gaming the system', which is extensively discussed in the literature (e.g.Petre, Duffy, and Hund 2019).

Avoiding risks of the intervention leading to unintended detrimental behaviour
This category involves unintended, unintentional uses of the designs.In their first reactions to a design, 14 of the 439 respondents spontaneously considered how the design could be used in unproductive or harmful ways without an intent to do so.In the choice task, 5 respondents argued that the chosen design is better entirely or partly because it has less of a risk of unintended use.This category comprised five more specific perspectives, which we explain in what follows: 4.6.1.The user could be misdirected to aim for a positive analysis score for their text This was expected of the designs that evaluate the comment while writing it.This also illustrates a downside to giving positive feedback to civil commenters: it may turn the receiving of positive feedback into a goal, which can distract the original activity of commenting on news.For example: On the AUDIENCE: 'I'd be concerned that it would encourage me to write comments that make the virtual experts happy rather than helping me concentrate on what I'm thinking about the news issue'.(UK, comments daily) 4.6.2.Directing the user's focus on negativity This expectation reveals a belief that online news commenting easily gravitates toward negativity.The expectation came up with the designs that propose to show to the readers whose comments might be problematic: On the PHILOSOPHY: 'It highlights negative comments and hides the more positive ones.I found it unpleasant'.(UK, comments weekly) On the HIGHLIGHT: 'I think this solution would be helpful but wouldn't fix the problem completely.It highlights uncivil comments what leads to us paying attention to them even more and as people tend to react to such strong feelings, it would probably cause even bigger fights because people would focus only on the negativity'.(Poland, comments weekly) 4.6.3.Individual users could be stigmatised over time This concern applied to a scenario where the users give honest and accurate negative feedback to another user who is commenting in an uncivil way, and where the feedback stays on their profile for a long time.This may lead the other users to be overly judgmental toward the one with negative feedback in the future.For example: On the SYMBOLS: 'I don't really like that.You might say something arrogant in one article and 500 people click your 'full of arrogance' and then there is no coming back from that, it will be like a stigma.If you comment next on another article, someone will see your profile and judge you based on one number that may have come from one unpopular comment on another article that had nothing to do with the current article'.(Greece, comments monthly) 4.6.4.Directing the users to comment about the discussion platform rather than the news article Particularly the designs with a provocative communication style were feared to cause this, for example: On the PHILOSOPHY: 'I think this solution is not good.It seems self-indulgent to use Socrates.I don't think the wider public will understand the relevance of this and it won't have the desired effect.It is likely to generate negative comments about the system itself'.(UK, rarely comments) 4.6.5.Reinforcing the commenter's emotion All the comments in this category were about the EVALUATE, where the user must click how they feel before writing a comment.The respondents were concerned that the increased awareness of the emotional state might make one more focused on it, hence reinforcing its negative aspects.For example: I don't think it will work -may encourage people to feel more negative/angry by identifying the feeling.(UK, rarely comments) In other words, the subcategories above imply that high quality would be indicated by explicit features and/or assurance that unproductive and unintended use of the intervention would be prevented.

Discussion
In the following, we discuss the meaning of the identified categories of quality at different levels.We propose preliminary design considerations, many of which introduce the needs for balancing acts between different extremes.The considerations are meant to help creating high-quality UI solutions and appropriately communicating them to users.Finally, we reflect on the validity of the reported study.

Respecting freedom to comment
Considering Whitworth's (2009) STS theory and its communal level, which concerns the exchange of norms, ideas and beliefs, people appear to cherish freedom and active audience participation in journalistic context.At the human level, which concerns personal level exchanges of meaning, the users seem to want the design to remain unnoticed, yet act when needed, in order to allow for appropriate communication between news readers.This aligns with the UI design principles of supporting immersion and compatibility with the user's perspective (Galitz 2007).This requirement is also supported by related work of Wang et al. (2014) who found that a 'privacy nudge' that delays posting on Facebook can both prevent unwanted disclosures and feel intrusive.
Design consideration 1: Seek for a balance between restriction of speech and promotion of civil discussion.
The design could be made feel less restrictive, by letting the user have some degree of control over the intervention design, making the system at least a little bit flexible.For example, we speculate that more users could be satisfied if there were easily accessible settings to influence how often the user is likely to see the intervention.That said, the impact of this kind of customisability on the effectiveness of the intervention ought to be studied case by case.

Objectivity of intervention
Considering the communal level (Whitworth 2009), the users seemingly require the design to be in line with the protection of commenting as a place where different opinions are allowed.At the human level (Whitworth 2009), the users seemingly have a broad requirement of untampered communication.Also, previous research stresses the requirement for objective moderation (e.g.Wang 2021).
We argue objectivity to be important when considering contexts where people of differing opinions take part in commenting.Objectivity is also important in contexts where the users could perceive the discussion platform provider to have an interest in promoting certain types of opinions.In such contexts, the users probably need to know that the system was intended to avoid any bias.
Design consideration 2: Offer reasons for the users to trust that the comments are evaluated by objective actors.
For users who perceive that the intervention is somehow biased or wrongful towards their commenting, it is central to offer ways for them to defend themselves.For example, a new UI proposal could feature a possibility to directly chat with administrators or moderators in problematic situations.

Helping various users behave better
Some respondents seemed to want the designs to target users who are clearly trolling.However, most wanted that the average user is helped by an intervention.The call for help seems to illustrate, at the communal level (Whitworth 2009), that many users think the social interaction (synergy) in commenting should result in more benefits, such as production of information, enjoyment, and understanding.At the human level (Whitworth 2009), the users' need for help suggests that many users think the current commenting systems do not afford enough capability to control one's tone or to empathise with other users when communicating.This is also supported by literature: the current, largely text-based interfaces may limit the ability to control one's emotions or to empathise with other people (Walther 1993).Also, previous research has found that some social media users would like to get help in controlling their tone of writing (Wang et al. 2014).
Unfortunately, our data does not indicate how much help the system should give, in what contexts, and to whom exactly.On one hand, helping when it is not needed could feel patronising.On the other hand, the more the design feels like an intelligent assistant, the higher the risk of 'infantilisation': individuals may come to rely on the guiding interventions and become unable to make decisions on their own (Acquisti et al. 2017;Bovens 2009).
Design consideration 3: Seek for a balance between helping the users too much and helping the users too little.
Design consideration 4: Help the user to avoid getting involved into emotionally upsetting situations.
The designs could be explicitly communicated as attempts to improve social interaction as this could increase the likelihood that the user accepts the design.In the light of the designs presented in this paper, it might be wiser to imply that the users lack the ability to control their tone of writing rather than a motivation to control it (Fogg 2009).

Use of apt metaphors
Considering the communal level (Whitworth 2009), the findings suggest that the style of addressing the commenters should match the commenters' values and contextual expectations.For example, if commenting is considered a serious matter, playful metaphors may be a bad idea.At the same time, at the human level (Whitworth 2009), the findings suggest the design should match user's personal requirements.This seems to call for personalising or customising the design.However, we do not have strong reasons to believe that the users would creatively customise a UI intervention design's appearance.Also, we speculate that a high degree of personalisation of a UI intervention (e.g.highly personalised metaphors) would scare off a large portion of users.
Design consideration 5: Utilise metaphors with caution.
We emphasise the need to try different metaphors (e.g.dog vs. cat vs. abstract creature) as well as basing them on knowledge of the cultural meanings in the target culture.In a great product metaphor, the metaphor's source has high salience (i.e.significance in a person's representation of a 'category') (Cila, Hekkert, and Visch 2014;Ortony et al. 1985).For example, reflecting on our design choices in the CREATURE, a pet dog appearing fearful is not a typical exemplar of the concept of suffering, therefore its salience might not be high.In addition, in a great product metaphor, the 'source' (e.g. a tornado) should have obvious similarity with the 'target' (e.g. a vacuum cleaner) (Cila, Hekkert, and Visch 2014).As the connection between a fearful dog and a negative comment is arguably not that obvious in the CREATURE, it could be seen as a decent metaphor, but not a great one.

Avoiding risks of intentional misuse and unintended detrimental behaviour
Considering the communal level (Whitworth 2009), intentional misuse of commenting UI can be seen to create strong conflicts and exhaust users' morale.The same is true for the other detrimental behaviours that the respondents mentioned, though their effect might be less drastic.At the human level (Whitworth 2009), the unproductive behaviours can harm the perceived ease of use of commenting or one's capability to comment.Previous research indicates that many people avoid commenting because of conflict in comments (Stroud, Van Duyn, and Peacock 2016).Further, we note an earlier work has found that some journalists expect that some users would use automatic notifications about uncivil writing as a guide to write uncivil comments (Kiskola et al. 2021).The expectation of intentional misuse did not, however, come up in an earlier work where 18 university students were interviewed (de Carvalho, Olsson, and Kiskola 2021).
Design consideration 6: To discourage creative misuse, make the design harder to use for unintended purposes.Design consideration 7: Analyse which UI affordances might encourage detrimental behaviour and try to avoid including them in the design.
When designing future UI interventions for social contexts, it could be a beneficial exercise to anticipate and model intended use processes, and then identify unintended forms of use.For example, typical and atypical deviations, and completely aberrant behaviours could be identified, and considered from the perspectives of natural, accidental, and intentional evil (Klein 2007;Merton 1936;Nelson and Stolterman 2012;Van Der Vegte et al. 2004).Also, some crude user personas (e.g. a worrier, a hedonist, a controversialist, or an inconsiderate person) could support the analysis.Moreover, besides this design work, it could be wise to vaguely communicate readiness to address unintended behaviours to potential users.This could help potential users accept the technology despite seeing flaws in it.

Considering quality at multiple levels in design and evaluation
All in all, the analysis implies that the behavioural issues related to uncivil commenting are largely socio-technical by nature.Rather than being caused by either technology or behavioural conventions alone, the issues emerge from the application of technological solutions in complex and socially constructed circumstances (Whitworth 2009).For example, good quality is not only unambiguously linked to the artefact's qualities but also to a belief that everything people might do with the artefact has been considered.This implies that quality also refers to addressing various particularities of the intended socio-technical-cultural context.This idea is strongly in line with Chan's (Chan 2012) normative notion that good social interaction design accounts for the development of a social tool over time.
Next, we reflect the identified user perspectives on quality against common notions of user-centred quality in HCI.The respondents appeared to often evaluate the designs from the perspective of the community or society (e.g.'It can also make comment section more civilized … ', ' … pathetic for democracy … ').Hence, in this context, the concept of user-centred quality also covers communal requirements, such as freedom, order, morale, and synergy, as highlighted by Whitworth ( 2009).
Many qualities commonly focused on in UI design (e.g.ease of use, clarity, desirability) can support communal requirements in this context by, for example, making it easier to comment, understand other users, trust other users, and follow the predefined community rules.However, particularly the adaptability (cf.reliability, [Whitworth 2009]) of the design seems relevant: a person may consider the trouble it would take for a user or a news site to use a design for unintended or unadvertised purposes (adaptability).Hence, perception of adaptability is related to both fears that users will misuse the design and fears that a news site will use the design to censor and manipulate users.While a low cost of adaption does not guarantee use for unfruitful and malicious purposes, a high cost of adaption makes such use impractical.
Reflecting on the prevalence of the expectation of misuse, we found it surprising that as many as approximately ten percent of the respondents raised the possibility of intentional misuse and other behaviours that can cause harm.Perhaps this is connected to a wider social context of online incivility and the public debate about it (Diakopoulos and Naaman 2011; Gillespie 2018).The topic of online incivility has been debated for about a decade (Gillespie 2018; Grön and Nelimarkka 2020), and especially the most actively commenting respondents likely have first-hand experience on it.
From the perspective of design evaluation, the findings can be seen to support the premise that traditional, unavoidably reductionist measurement instruments like specific user experience questionnaires might indeed disregard relevant qualities of UI interventions in this area.As argued by Suri (2002), traditional, reductionist measurement and evaluation techniques are often not helpful to understand how novel products would be perceived and experienced.As they require knowledge about what would be relevant to measure, many aspects of perceived quality will likely be missed.
Further, while high quality may be described using short quality attributes, for quality attributes to offer actionable guidance to design, the design context must be well known, and the attributes contextualised accordingly.For example, recognising that a good motivational intervention to online discussion is effective would leave much contextual nuance unspecified.Accordingly, in this study, rather than reducing the qualities into a list of adjectives, we offered longer qualitative descriptions.

Reflection on the research process and methodology
Considering the methodological approach, the use of Prolific in recruiting participants for the survey resulted in over-representation of participants from the UK and other Western countries.Thus, the findings on how good quality is perceived represent mostly Western viewpoints.The socio-technical nature of the context area would benefit from data from, for example, more collectivistic cultures, and cultures that typically have different views of authority (see e.g.Baggini 2018).Further, we note that the monetary compensation for acceptable survey participation in Prolific might have caused the respondents to give longer answers to make sure their response gets accepted.
Regarding the extensiveness of the findings, they are based on online news commenters' opinions and arguments on eight speculative intervention designs focusing on the tone of commenting and emotional reflection and are therefore limited in both number and type.Opinions on intervention designs focusing on, for example, good argumentation in commenting or socialisation could be different.It would also be interesting to receive additional viewpoints from people who never comment on online news sites.
Despite these shortcomings, we argue that the methodological choices were justifiable vis-à-vis the set goals because: First, the online survey enabled us to reach a large number and relatively broad spectrum of people who actively comment on online news sites.Second, presenting the designs as speculative resulted in meaningful answers.The answers offered an extensive overall picture of the potential end-users' assumptions and expectations.They offered meaningful new viewpoints and nuance to quality, and critical perspectives to the deployment of technology.Also, the speculative interfaces brought forth new insights that would remain latent when using more conventional interfaces: for example, concerning the use of metaphors and the consideration of cultural sensitivity.We note that all the identified requirements for good quality are important to some users and therefore need to be addressed when designing and publishing these kinds of systems and evaluating their quality.Also, as the requirements were spontaneously raised by the respondents, the findings could inform which user-centred qualities are relevant to measure in future studies.

Conclusion
This paper reported a case study on user-centred quality of UI intervention designs intended to influence online discussion in the context of news commenting.We analysed news commenters' first reactions to speculative intervention designs and the arguments they used to justify choosing between two designs.This resulted in several user requirements that relate to the communal, socio-technical perspective to news commenting as a form of social interaction and that are relatively rarely highlighted in the literature.For example, many users think a good intervention design should feature technological and/ or human capability to prevent its intentional misuse.They expect the UI interventions to be objective and to utilise metaphors that are personally relevant and, hence, appropriate, and effective.
All in all, the study advances our understanding of how potential users perceive quality in UI interventions to online discussion.All the identified requirements are important to at least some users and therefore need to be addressed when designing and deploying these kinds of systems and evaluating their quality.To this end, we provide seven design considerations about different facets of user-centred quality, which can help designers make more well-informed decisions.
Less offensive.The second solution [AUDIENCE] appears like you are being judged and tried by 4 other people.The icon solution is a neutral symbol.(UK, comments monthly)