Analysis of sentiment expressions for user-centered design

Highlights

• We translate online customer reviews into attribute-level design insights.

• We develop a methodology for attribute-sentiment expression mapping.

• The algorithms developed utilize NLTK, Word2Vec and Stanford Parser.

• We build on sentiment analysis and information extraction methods.

• We identify customer segments based on the individual reviews analyzed.

Abstract

Devising intelligent systems capable of identifying the idiosyncratic needs of users at scale and translating them into attribute-level design feedback and recommendations is a key prerequisite for successful user-centered design processes. Recent studies show that 49% of design firms lack systems and tools for monitoring external platforms, and only 8% have adopted digital, data-driven approaches for new product development despite acknowledging them as a high priority. The state-of-the-art attribute-level sentiment analysis approaches based on deep learning have achieved promising results; however, these methods pose strict preconditions, require manually labeled data for training and pre-defined attributes by experts, and only classify sentiments intro predefined categories which have limited implications for designers. This article develops a rule-based methodology for extracting and analyzing the sentiment expressions of users on a large scale, from myriad reviews available on social media and e-commerce platforms. The methodology further advances current unsupervised attribute-level sentiment analysis approaches by enabling efficient identification and mapping of sentiment expressions of individual users onto their respective attributes. Experiments on a large dataset scraped from a major e-commerce retail store for apparel and indicate 74.3%–93.8% precision in extracting attribute-level sentiment expressions of users and demonstrate the feasibility and potentials of the developed methodology for large-scale need finding from user reviews.

Introduction

A major barrier to user-centered design is the relative absence of formal mechanisms for translating the “voices” of individual users into the design of differentiated product attributes along which preferences diverge (Salvador, De Holan, & Piller, 2009). Current mechanisms for engaging users in the design process revolve primarily around surveys and focus-group studies (Fogliatto et al., 2008, Griffin and Hauser, 1993) and web-based configurators (Felfernig, 2007, Franke et al., 2010). These solutions merely target an inherently biased fraction of users and product instances, and leave an entire swath of pertinent user behavior, preferences, and opinions not captured. The inability of some users to identify and directly express their true preferences (Franke, Keinz, & Steger, 2009) further exacerbates this gap. This is while the rapidly growing social media and e-commerce platforms provide an unprecedented source of knowledge about the preferences of much larger and more diverse populations of users and on a multitude of product instances within the same class/family. This article presents a novel methodology for the analysis of sentiment expressions (ASE) of users to draw attribute-level design insights and recommendations from unstructured, online user reviews. ASE contributes to the broader field of aspect-based sentiment analysis (ABSA) in natural language processing (NLP) by enabling the extraction of sentiment expressions in a semi-supervised fashion.

The majority of existing ABSA methods classify user sentiments into predefined categories (e.g., “positive”, “negative”, “neutral”) using labeled training data and for some experts pre-defined aspects (García-Pablos et al., 2018, Ma et al., 2017, Peng et al., 2018, Pontiki et al., 2015, Pontiki et al., 2015, Rietzler et al., 2019, Sun et al., 2019, Tamchyna and Veselovská, 2016). The proposed ASE methodology aims at addressing two practical limitations associated with the current state of ABSA literature. First, such sentiment classification approaches may leave important user feedback and insights uncaptured. Second, the supervised nature of most ABSA methods limits their practical use due to the crucial need for laborious labeling and annotation of online review data for training (e.g., Lee and Bradlow, 2011, Pang and Lee, 2006, Ravi and Ravi, 2015, Tang et al., 2009, Zhang et al., 2018a, Suryadi and Kim, 2019, Zhou et al., 2015). The proposed ASE methodology goes beyond identification of sentiment polarity and rather extracts the information about the very same sentiment phrases expressed by users (Fig. 1), without the need for labeled/annotated training data. The developed methodology has been tested and validated on a large dataset of scraped from an online retail store for sneakers and athletic apparel.

Our motivation stems from the growing abundance of user-generated feedback and the lack of advanced computational frameworks and techniques for turning data into new design knowledge and insights. Recent studies show that 49% of design firms lack systems and tools for monitoring external platforms, and only 8% have adopted digital, data-driven approaches for new product development despite acknowledging them as a high priority (Oracle, 2019). The proposed methodology aims at addressing the need for effective mechanisms for translating the preferences of individual users expressed through online reviews into attribute-level design feedback and recommendations (Griffin and Hauser, 1993, Salvador et al., 2009, Wagner and Majchrzak, 2006, Zhang et al., 2018). The foundational capabilities to realize this vision include (Salvador, de Holan, & Piller, 2009): (A) solution space development to gather user preferences, (B) robust process design to enhance manufacturing flexibility, and (C) choice navigation to minimize the burden of choice. Current approaches to mass-personalization predominantly emphasize Capability B to accommodate the uncertainty and diversity of user preferences. Examples include research on reconfigurable manufacturing systems, outsourcing (Xu, 2012), modular product design and postponement (Koren, Hu, Gu, & Shpitalni, 2013), and coordination of operations along supply chains (Salvador, Rungtusanatham, & Forza, 2004).

These approaches are essentially “reactive” in that they treat individual preferences as mostly unknown and intractable, and instead attempt to prepare for the subsequent uncertain behavior at the downstream manufacturing stage. Even more proactive, solution-space development approaches (Capability A) such as focus-groups or web-based configurators (Felfernig, 2007, Franke et al., 2010) are not fully effective due to targeting small groups of individuals and limited product instances that are hard to scale up. These limitations have hindered the practical adoption of mass-personalization due to extensive economic and operational limitations (Fogliatto, Da Silveira, & Borenstein, 2012). Further, neglecting the voices of the majority of users, the gap between what is designed and what is desired lingers notwithstanding the knowledge and expertise of the design team. There is therefore a critical need for a preference modeling mechanism built upon direct feedback from users that leverages the unprecedented wealth of knowledge embedded in freely available online reviews.

Google reports 600% increase, between years 2015–2017, in the amount of time people spend exploring others’ experiences before making a decision on a product or service (Google, 2017). A more recent study (Local Consumer Review Survey | Online Reviews Statistics & Trends,2019) reports that 86% of consumers read reviews for local businesses, 80% of 18–34 year-olds have written online reviews and 91% of 18–34 year-olds trust online reviews as much as personal recommendations. These statistics pinpoint the significant impact of online reviews through e-commerce and social media platforms (1) on users’ preferences and choices, especially millennials, and (2) on the prosperity of small and domestic design and manufacturing startups. The ASE methodology aims at leveraging this unprecedented opportunity to align the design of customizable product attributes with the heterogeneous preferences of users that are increasingly expressed in unstructured text format. Not meeting this need will limit the industrial adoption of mass-personalization due to excessive inventories, financial losses, and poor delivery performances (Fogliatto & da Silveira, 2008), hindering the potential transformative impacts on various other industries such as food, nutrition, orthopedics, electronics, and homebuilding in terms of identifying user segments and tailoring the design, production, and service to the specific needs of each segment.

The sentiments of users about individual attributes of a product are significantly more informative than their overall sentiment about the product (Wang et al., 2014) for gauging their idiosyncratic preferences and choices. The finer-granularity of attribute-level sentiments enables the extraction of more informative design feedback and recommendations, specifically because online reviews typically involve considerable diversity in terms of size, structure, and the way emotions and feelings are expressed (Fogliatto et al., 2012, Tamchyna and Veselovská, 2016). Sentiment analysis is the computational study of people’s opinions, preferences, emotions, or attitudes towards the attributes of a product, service, event, topic, or even individual (Bing, 2015). Since the early 2000′s, sentiment analysis has become a central area of research in NLP, conducted at three different levels: document, sentence, and attribute (García-Pablos et al., 2018, Tamchyna and Veselovská, 2016).

Unlike document- and sentence-level analysis which merely output overall polarity of user sentiment, attribute-level sentiment analysis carries out attribute extraction, entity identification, sentiment description, and attribute-sentiment mapping (Zhang et al., 2018a). Attribute-level sentiment analysis is still a developing field of research. Only recently have studies been conducted on mapping user needs onto design specifications through sentiment analysis (Wang, Mo, & Tseng, 2018); however, the proposed methods are limited to document-level sentiment classification which are unable to capture attribute-level information. Further, existing attribute-level sentiment analysis solutions typically focus on sentiment polarity (e.g., positive/negative/neutral), and leave the detailed description of individuals’ opinions about different attributes uncaptured (García-Pablos et al., 2018, Peng et al., 2018, Pontiki et al., 2016, Sun et al., 2019, Tamchyna and Veselovská, 2016). ASE aims at addressing these gaps by focusing on attribute-level sentiment expressions rather than polarity. ASE is built upon the rationale that true user-centered design can only be achieved if the voices of all (or at least majority of) users are heard across several related product types and classes. A detailed review of state-of-the-art sentiment analysis methods is provided in Section 2.

The overarching goal of this research is to build and test an efficient computational methodology for extracting attribute-level sentiment expressions of individual users from online reviews. Shifting the focus from sentiment polarity to sentiment expression, the methodology can shed light on the diversity and similarity of user preferences at scale: Users may converge into a number of segments, where at least one feasible design alternative exists that would satisfy an entire segment while no such alternative exists for more than one segment. This article builds the foundation for achieving this goal by accomplishing the following research objectives:

• Objective 1: Identify attribute-level sentiment expressions of individual users. This objective revolves around investigating methods for lexicon building and word embedding, mapping sentiment expressions onto differentiated attributes, and extracting customizable attributes along which user preferences diverge. To this end, two rule-based methods for ASE are developed and tested in this article.

• Objective 2: Measure ASE-based user similarities across various product types. This objective is motivated by Arrow’s theorem that proves any social welfare function which satisfies the transitivity, unanimity, and independence of irrelevant alternatives conditions is a dictatorship (Arrow, 1950). The implication of this theorem for this research is that it is impossible to generate a design alternative that maximizes the utility of User A without reducing the utility of User B, if A and B have significantly different preferences. In order to increase the utility of all individual users in terms of satisfying their idiosyncratic needs, it is therefore to necessary to generate at least as many design alternatives as the number of distinct user types or segments. This objective builds on the results of ASE to identify potential patterns in user preferences.

• Objective 3: Build ASE and validate its performance through a large dataset. Objective leverages the knowledge and experience from the authors’ preliminary work and industry collaboration to set up a pilot study on sneakers based on a dataset retrieved from an e-commerce platform.

The remainder of this article is organized as follows. Section 2 presents a detailed background on sentiment analysis, particularly ABSA (readers familiar with sentiment analysis may skip this section). Section 3 elaborates on the developed ASE methodology including the rule-based algorithms and the similarity measurement approach. Section 4 presents the case study and discusses the experimental results, analyses, and limitation. Section 5 concludes the article and provides directions for future research.