“I Prefer Regular Visitors to Answer My Questions”: Users’ Desired Experiential Background of Contributors for Location-based Crowdsourcing Platform

2.1 Mobile Crowdsourcing for Collecting Location-related Information

Crowdsourcing has emerged as a useful method of acquiring data from large groups of individuals [25]. It has been employed in various contexts, including the division of creative labor [63], feedback collection [10, 12, 41], labeling for machine-learning tasks [49, 64], and the provision of objective information about specific objects or places [5, 13, 19, 41, 63]. Mobile crowdsourcing refers both to crowdsourcing tasks performed ‘on the go,’ for which mobile devices are an ideal medium, and to tasks that require contributors to be at a specific location [34, 41]. Various research projects have harnessed distributed crowds to collect data about moving objects [28, 72], specific objects at a particular location [5, 13, 19, 63], or a location itself [10, 12, 41]. Given that much of the information collected pertains to particular locations, such tasks are typically classified as location-related or geographical [27]. A common example is the provision of comments or feedback on online map services such as Google Maps³ and OpenStreetMap⁴, which can include reviews of restaurants [12, 41], stores [12], or attractions [10, 12, 41].

Numerous studies have demonstrated how mobile crowdsourcing can be used for collecting diverse types of location-related information. For example, the fields of citizen science and mobile crowdsensing have explored recruiting individuals to use their phones to collect data about urban places [9], including about noise [43, 53, 60], air quality [26], and the provision of local facilities like biking trails [46] and accessibility features [19, 56]. Prior research, such as Chang et al. [10], have also investigated the use of mobile crowds to contribute up-to-date information about potential attractions and points of interest in a given area. Likewise, Liu [41] developed a social question-answering service whereby users could ask time-sensitive and location-specific questions and receive prompt answers from the crowd, covering topics such as restaurants, travel, and transportation. As the basis for news reporting, Väätäjä et al. [63] recruited reader-reporters as crowd workers who collected information, including other media content. Similarly, Agapie et al.’s [1] hybrid crowdsourcing process for event reporting used a combination of local workers to gather firsthand information and remote ones to curate it and generate event reports. And Huang et al. [28] developed a participatory sensing system that invited transit passengers to share their location data and provide real-time reports, as a means of predicting buses’ arrival times.

While demonstrating the effective use of crowdsourcing to acquire geolocation data, sometimes relying on real-time updates from on-site contributors, none of the studies reviewed above considered the potential influence of the contributors’ experiential background. This could be particularly pertinent when the information being sought is not time-sensitive, but involves more general aspects of a place. Where platform users’ need or preference is for non-real-time data, contributors who have already moved on from the target location could offer information just as valuable as those who are still there – or sometimes perhaps more valuable, thanks to the additional time they have had to reflect on their experience or compile and structure their observations into a detailed account of the location. On the minus side, however, extending the range of potential contributors is likely to increase the challenge of identifying the most pertinent crowdsourced information, given their different experiential backgrounds with the place.

The present study seeks to augment the relevant literature by investigating whether the end users of crowdsourced information prefer the contributors of such information to have had specific experiential backgrounds; and if so, what types, and how those types vary across different information classes.

2.2 Matching Contributors with Tasks

Task assignment and contributor matching is a specific research area directly relevant to the present study. Prior research in this area has proposed various methods for matching tasks to contributors [23]. These methods include utilizing historical worker data [47, 51], analyzing existing answer distributions [35, 71], asking ‘gold-standard’ questions with known answers [30, 40], and reviewing workers’ attributes [21, 33] and behavioral data [55]. In this section, our focus is on the matching of contributors to tasks based on attributes such as contextual factors [34], skills [22, 44], and cognitive abilities [21, 24]. Some prior studies have looked at matching tasks with crowd workers based on their current location. For example, Kazemi and Shahabi [34] implemented centralized task assignment by assigning workers tasks near their reported locations, with the aim of maximizing the overall number of assigned tasks. Similarly, Tran et al. [62] restricted task performance to workers within the spatiotemporal vicinity of the task; Liu et al. [41] allocated temporal and geo-sensitive questions to workers based on their current location by analyzing live streams from public microblogging platforms; and Linnap and Rice [39] assigned geo-sensitive questions based on location tracking and model-based methods. Konomi et al. [38] assigned tasks based on crowd workers’ movement patterns, and reported achieving greater geographical relevance than basic proximity-based methods could. To maximize the coverage of collected data in a given spatio-temporal space, Ji et al. [31] also assigned tasks to crowd workers based on their mobility, as measured by the time-gap between a participant’s arrival and departure, minus the necessary travel time between his/her point of origin and destination.

In addition to tasks’ geographic locations, researchers have explored task matching based on the expertise and skills the crowd needs to effectively perform them. Heimerl et al. [22], for instance, installed a physical kiosk in a university Computer Science building with the aim of attracting individuals who possessed specialized knowledge or skills in that field to participate in grading tasks. Mavridis et al. [44], meanwhile, proposed a skills-based task-assignment model that measures the distance between the skills of a given worker and the skills required for a specific task, and assigns the most specialized tasks first to those workers with fewer skills. Goncalves et al. [21] investigated the matching of tasks and workers based on the latter’s cognitive abilities, including visual perception and fluency, which were measured in a laboratory setting using the well-established Kit of Factor-Referenced Cognitive Test [18]. Hettiachchi et al. [24] also implemented task assignment based on workers’ cognitive abilities, but via a dynamic system that recommended tasks based on workers’ performance on its four types of online cognitive tasks: namely, classification, counting, transcription, and sentiment analysis. Lastly, instead of using a system to match workers to tasks, Wang et al.’s [68] proposed task-recommendation approach helps workers select tasks based on their own progress and resource contexts.

Nevertheless, few if any previous studies have explored the matching of crowd workers to location-related description tasks based on their personal experiential backgrounds with the locations in question. Given the rising utilization of location-based mobile crowdsourcing platforms, it is critical to ensure that the information available on these platforms is relevant and useful to their users. Gaining an understanding of the aspects of contributors’ experiential backgrounds that users perceive as enhancing the utility of contributors’ descriptions can offer important benefits. Therefore, the present study is intended to bridge this research gap.

The remainder of this paper is structured as follows. First, we will describe the design and execution of a semi-structured interview study aimed at obtaining preliminary insights into our three research questions. Next, it describes the survey study we conducted to delve deeper into the types of location-related information commonly sought by crowdsourcing-platform users, along with their perceptions regarding the attributes of this information. Then, we explain how the results from that second stage informed the design of our third: an online scenario-based study, the objective of which was to capture crowdsourcing-platform users’ expectations about the information they were seeking and the aspects of experiential backgrounds they believed would enhance the utility of the descriptions provided.

3.1 Interviewees

Our semi-structured interview study involved a diverse group of 22 interviewees, 12 male and 10 female, aged between 21 and 50. Each interviewee self-reported seeking location-based information on crowdsourcing platforms like online maps or forums at least once a month, and all except one also had experience serving as contributors on such platforms. We recruited them through a variety of Facebook groups specifically designed to bridge the gap between researchers and potential research subjects in our country. In appreciation of their time, each participant was compensated with NT$300 (approximately US$10.75).

3.2 Study Procedure

We informed invited participants about the study’s objective. Once they provided their informed consent by signing the document, we commenced the interview. During the interviews, we sought to understand the kinds of information the participants had procured or wished to procure from location-based crowdsourcing platforms or online forums. We also inquired about their anticipations/preferences regarding the experiences and backgrounds of the contributors to such platforms and forums, and prompted them to reflect on how such anticipations/preferences differed across location-related inquiry types. In doing this, our goal was to identify the central characteristics of the location-related information they commonly pursued; the quality attributes of the descriptions they expected; and the types of experiential background they wanted contributors to have to answer specific type of location-related questions.

To aid the interview process, we prepared a set of cards (as shown in Fig. 1) featuring examples of various location-related experiences and information. These cards functioned as prompts, helping interviewees recall their past experiences of seeking and obtaining similar information. The content of the cards underwent iterative development, with new ones being introduced whenever the researchers identified a novel type of location-related experience or information from the interview data. Ultimately, we formulated 23 cards that covered types of location-based information (shown in blue in Fig. 1) and 11 cards that addressed types of contributors’ experiences (shown in yellow).

Due to COVID-19 restrictions, all interviews were conducted via online video conferencing with screen-sharing capabilities. The cards were exhibited on Conceptboard⁵, an online whiteboard platform, which allowed the researchers and interviewees to view them concurrently. Each interview was video-recorded and transcribed, and lasted from 90 to 120 minutes. The interview study, along with the subsequent survey and online study that will be presented later, all received approval from the Institutional Review Board (IRB) of the authors’ institution.

3.3 Data Analysis

The construction of our codebook was informed by our research questions, and we utilized MAXQDA⁶ for thematic analysis of our interview data. To ensure the reliability of our coding procedure, three researchers independently coded the same three interview transcripts. Then, they jointly reviewed and compared their selected codes, explored similarities and differences in their interpretations, and ultimately reached a consensus on the coding schema. The codebook was then updated accordingly.

3.4 Preliminary Insights from the Interview Study

4.1 Survey Respondents

The online survey was publicized in numerous Facebook groups and pages dedicated to residents of various cities in our country. These venues were selected for advertising due to our assumption that large numbers of their members would be interested in gathering information about their local areas. Before initiating the study, participants were informed about their rights and the withdrawal process. As an incentive to participate, we entered all respondents into a raffle, with every fifth participant chosen at random receiving a reward of NT$200 (approximately US$7.17). Initially, we received a total of 240 responses. Subsequently, we undertook a data cleaning process, which included eliminating duplicate responses, discarding responses that contained incorrect answers to two embedded attention-check questions, and removing entries from participants who took an unusually short time to complete the questionnaire. This resulted in a final dataset of 162 responses that were used for analysis. In this final dataset, 58.6% of respondents identified as female, 39.5% as male, and 1.9% chose not to disclose their gender. The age range of the respondents was 20 to 56 years, with an average age of 29.8 years and a standard deviation of 7.8.

4.2 Results

4.2.1 Most Frequently Sought Types of Location-related Information.

Table 1 illustrates the frequency with which respondents sought location-related information of various types via the mobile crowdsourcing platforms.

Table 1:

	Frequencyscore	Scores of the six key information attributes
		Objectivity	Variability	Location-specificity	Temporal regularity	Time-specificity	Relativity
Food review	5.97	2.83	4.28	4.67	3.52	3.75	4.96
Product supply	4.91	5.14	4.32	4.70	4.79	4.89	4.33
Product price	6.01	5.21	3.70	4.81	3.47	3.67	5.25
Crowdedness	4.69	4.07	5.44	4.66	5.01	5.17	4.41
Event-related information	4.94	4.56	4.74	4.85	4.77	5.78	4.20
Condition of public equipment	3.73	4.53	4.42	4.05	3.43	3.89	4.07
Regional issue	3.67	3.92	4.72	3.17	3.27	3.36	4.46
Parking availability	4.94	5.05	5.19	4.05	4.09	4.25	4.88
Scenery description	5.05	3.06	3.91	3.40	4.34	4.33	4.37
Regional POI recommendations	5.75	2.90	4.19	3.70	3.53	3.77	4.61

View Table

Table 1: Average scores assigned to the frequency and the six key information attributes, by information type

Figure 2:

View Figure

The three categories of information most frequently sought were product price (average score of 6.01 out of 7), food review (5.97), and regional POI recommendations (5.75). These three categories were significantly more sought after than others (χ²(9)=462.25; product price: p<.001; food review: p<.001; regional POI recommendations: p<.001). On the other hand, regional issue (3.67) and the condition of public equipment (3.73) were the significant least sought-after categories (χ²(9)=462.25; regional issue: p<.001; condition of public equipment: p<.001). As such, these two categories were not considered in our subsequent online study.

5.1 Study Design

We conducted an online study using SurveyCake⁸, an online survey tool, with scenarios to prompt participants to imagine the experience of using a real location-based crowdsourcing platform. We first captured their general perceptions regarding the expected quality of location-related information descriptions and expected contributors’ experiential background. Specifically, we asked participants to rate the importance of ten aspects of description quality, as well as how much the presence of each of seven aspects of contributors’ experiential background would enhance the utility of the provided descriptions based on their general experiences of using platforms such as Google Maps, Facebook, and BBS Local Boards to search for/inquire about location-related information.

Subsequently, the participants encountered eight different scenarios, as shown in Table 2, each revolving around one of the eight most sought-after location-related information types identified in our survey results. These were food review, product supply, product price, crowdedness, event-related information, parking availability, scenery description, and regional POI recommendations.

For each scenario, participants responded to a set of three question sections. First, they rated their level of interest in the presented location-related information. Then, they rated the importance of different aspects of description quality on a five-point Likert scale ranging from 1=“not very important” to 5=“very important”, and how much the presence of each of seven aspects of contributors’ experiential background would enhance the utility of the provided descriptions from 1=“no utility increase” to 5=“significant utility increase”. To ensure attentive participation, we included two attention-check questions in the study, both of which simply instructed the participants to select “4” as their answer.

5.2 Participants and Recruitment

We disseminated the recruitment message across various online platforms, including local residential-oriented Facebook groups, Internet forums, Google Local Guides, and travel forums. The message outlined specific eligibility criteria: being aged 20 or older and having prior experience in obtaining location-related information from crowdsourcing platforms such as Google Maps, BBS Local Boards, and Facebook local groups. The recruitment message included a link to the study webpage, allowing participants to directly access the study webpage.

During the four-month period from April 1 to July 31, 2022, we gathered 404 responses. Following data cleaning, as detailed in the upcoming section, a total of 307 responses were deemed valid and included in our subsequent analyses. The respondents of the valid responses included 193 females, 108 males, and six participants who opted not to reveal their gender. Their ages ranged from 20 to 69 years (M=28.9, SD=8.0). In terms of frequency of using location-based crowdsourcing platforms, 71 reported doing so several times a day; 18, once a day; 116, several times a week; 20, once a week; 59, several times a month; 10, once a month; and 13, several times a year. The participants with valid responses were remunerated NT$100 (US$3.19).

5.3 Study Procedure

The study procedure is depicted in Figure 3, which outlines the step-by-step webpage encountered by participants. When participants clicked on the study’s webpage link, they were first presented with an introduction and consent. This form included information on eligibility for participation, the study’s objectives, detailed procedures, participants’ rights, and the process for withdrawing from the study. At this stage, participants could choose to continue by clicking an "Agree" button or opt to exit the page. Upon agreeing, they gained access to an online document explaining key terms they would encounter during the study, including definitions and examples of the ten aspects of description quality for enhanced clarity. Participants were allowed to refer to this document at any point during the study.

Figure 3:

View Figure

Figure 4:

View Figure

Initially, they encountered a brief set of instructions explaining that they would be presented with nine sets of questions. Each set includes a scenario description and three types of questions: their interest in information types, the information quality they value, and the experiences they find helpful. Participants were asked to imagine themselves using a platform similar to Google Maps, Google Local Guide, or a local community on Facebook or BBS forums, where individuals collaborate to answer each other’s local queries. Their task was to seek local information on this platform. The first set of questions was based on a general situation, while the subsequent sets focused on scenarios for eight different types of information, as detailed in Table 2. Upon completing all question sets, participants provided basic background information, such as gender, age, and frequency of viewing location-related information on crowdsourcing platforms. Lastly, they received an identification number for participation in a raffle. The goal of this design was to ensure participant understanding and engagement throughout the study.

5.4 Data Cleaning and Analysis

To ensure the validity of our data, we adopted a multifaceted approach: scrutinizing responses for two attention-check questions, tracking the time participants spent on the survey, and identifying duplicate responses. This led to the removal of 97 responses in total. Among these, 89 were disqualified for failing the attention checks, 5 were eliminated due to being multiple submissions by the same individual, and 3 were excluded for excessively rapid completion times.

For statistical analysis, we ran linear mixed-effects models using the R package lme4 [3]. We opted for a linear mixed-effects model due to the repeated measures from participants in our study. This approach, accounting for individual differences, allowed us to estimate fixed effects on the dependent variable and control for random effects. Participant IDs were used as random effects to manage repeated data from the same individuals. Moreover, the robustness of mixed-effects linear regression against assumption violations underscores its appropriateness for our analysis [58]. Below, we reported test results with p-values using Satterthwaite’s approximation of the effective degree of freedom [57].

5.5 Results

In this section, we first identify which aspects of contributors’ experiential background were regarded as enhancing and not enhancing the utility of descriptions for the eight types of location-related information, respectively. Subsequently, we explore the relationship between description quality aspects and each type of location-related information. We also examine the correlation between contributors’ experiential backgrounds and description quality aspects. Finally, we utilize linear mixed-effects regression models to identify which among the ten aspects of description quality are effective predictors of the value attributed to specific types of experiential background.

Figure 5:

View Figure

5.5.3 Correlation between Desired Contributors’ Experiential Background and Valued Aspects of Description Quality.

We conducted a Spearman correlation analysis to initially explore the relationships between different aspects of contributors’ experiential backgrounds and the aspects of description quality. The results are shown in Figure 6.

Figure 6:

View Figure

It is important to emphasize that such correlations represent overarching relationships, and do not take account of the specific types of location-related information. As indicated earlier, some facets of experiential background or aspects of description quality were perceived as particularly helpful when seeking certain types of information, and the above correlations do not reflect those specific relationships. As such, the results should not be interpreted as fully explaining the relationship between the two elements. Indeed, probably because of the intricate nature of the relationships involved, no pairing of an experience background aspect and a description-quality aspect demonstrated a strong correlation. Even in the seemingly straightforward pairing of recentness of experience and recentness of description, the correlation was only moderate, at 0.43. This suggests that the perceived benefits of a contributors’ experiential background is not dependent on any one aspect, but rather on a combination of various ones.

That being said, the correlation result did indicate that certain correlations were more likely to be seen as beneficial. For example, when participants valued temporal specificity in a description, they were also more likely to value contributors’ longer residence and the quantity, recentness, and regularity of their visiting experiences. In contrast, variety, professional relevance, and engagement in commentary were more likely to be deemed beneficial when participants assigned a high value to the enjoyment they might derive from reading descriptions.

Additionally, some extremely low absolute correlation values in our results can be used to identify which aspects of contributors’ experiential background and which aspects of description quality were considered to be unrelated. For instance, participants’ perception of whether recent experience was beneficial showed no correlation with the importance they assigned to the enjoyability of descriptions. Similarly, the perceived value of relevant professional experience was uncorrelated with the perceived importance of a description being temporally specific and recent.

6.1 Experiential Background Deemed Universally Beneficial vs. Beneficial for Specific Types of Location-related Information

Our findings indicate that certain aspects of experiential background of contributors were perceived as universally beneficial to information descriptions by most of our online study participants, with the top three being recentness, quantity, and regularity. These results suggest that, regardless of the type of information one is seeking, the presence of certain types of experiential background among contributors will generally be considered more beneficial than their absence. These findings suggest that, across the majority of location-related information being sought, location-based crowd-sourcing platforms could usefully prioritize contributors who possess the above-mentioned universally preferred experiential backgrounds.

However, and notably, our findings highlight the additional benefit of task assignments tailored to specific experiential backgrounds. While prior research has delved into matching contributors with tasks based on their expertise and skills in non-location-related contexts [22, 44], our study sheds light on the distinct and sometimes exclusive relationships between experiential backgrounds and particular types of location-related information. This insight supports the adoption of a similar task assignment strategy, which involves pairing contributors with location-related queries based on their relevant experiential backgrounds. This approach represents a notable contribution of our study, enhancing both the methodology of task assignment and the field of location-related crowdsourcing [1, 10, 28, 41, 63].

For example, when temporally specific descriptions are desired – e.g., when querying high-variability information such as about parking availability or crowdedness – participants favored contributions by long-time residents. This suggests that the participants associated the ability to provide time-specific information with extensive observations both across seasons and at multiple times of day.

Conversely, when participants were interested in obtaining highly subjective information, such as assessments of food, recommendations about POIs, or descriptions of scenery, they tended to assign a high value to contributors with a variety of similar experiences. This preference was correlated with a strong desire for objectivity in descriptions of these types of inherently subjective location-related information. The preference for variety of experience was likely informed by the idea that contributors who have visited a diverse range of locations are able to provide nuanced comparisons and contextualize their assessments. Such context-rich comparative analyses, in turn, could lend a degree of objectivity to subjective evaluations, by situating assessments in relation to other contexts rather than attributing them solely to the information items in question. Such an approach enhances the utility of the information provided by enabling information seekers to discern between individual bias and assessments made after thoughtful comparison and consideration of contextual factors.

Also, probably because encountering similar information items in a variety of contexts further implies that the contributor has experienced the items in various situations, participants deemed the variety aspect of experiential background as beneficial to the provision of novel information: presumably, because it suggests the contributor has visited places or observed things that the information seekers have not.

When participants perceived novelty, enjoyment, and objectivity as highly valuable, they perceived contributors’ professionally relevant experience as beneficial, alongside variety. This could have been because their professional backgrounds allow contributors to provide information from an expert perspective, introducing insights and knowledge uncommon among the general public, thus boosting typical platform users’ sense of novelty and enjoyment.

Probably because contributors who engage actively in commentary are often perceived as possessing experience of delivering information in an enjoyable and engaging way, active-commentary experience was deemed helpful by participants seeking descriptions that were enjoyable. Such experience was also deemed useful when participants desired a deeper understanding of information’s context. This could have been due to an expectation that contributors who actively engage in commentary have a better understanding of the specific needs of information seekers, and are able to provide context-specific information to effectively meet those needs. Participants also desired such experience when seeking objective descriptions, since they believed this experience would equip contributors to identify which aspects of their subjective experience would be most beneficial to future visitors to a location.

Due to space consideration, our discussion cannot cover every link in detail. Overall, however, these specific connections suggest that location-based crowdsourcing platforms can do more than match tasks with the crowd’s geolocation and movement, like prior studies [31, 34, 38, 39, 41]. They can also thoughtfully consider the link between the nature of location-related crowdsourcing tasks and the crowd’s experiential background when assigning tasks. This additional consideration could potentially enhance the relevance and applicability of the information provided to the seeker. However, it would also be beneficial for the system to determine the urgency level required by the information seeker, as this would enable the system to tailor its crowd recruitment strategy, striking a balance between prioritizing individuals’ geolocation – essential when immediate receipt of information is critical – and their experiential background, which may be more relevant for other types of inquiries.

6.2 Moving away from Locality and toward Experience in Location-based Crowdsourcing Platforms

Several location-based crowdsourcing platforms, including Google’s Local Guide⁹ and Local Wiki¹⁰, concentrate on using ‘local’ individuals to offer location-specific details, under an assumption that these individuals are best equipped to answer queries about a specific area. However, our findings suggest the importance of broadening the focus from a strict ‘local’ emphasis to encompassing a more diverse range of experiences and interactions with a location. This expansion is necessary for two primary reasons. Firstly, the concept of ‘local’ is subject to varying interpretations, which in turn influences the types of crowdsourced groups and their relative strengths in addressing different types of location-based queries. For instance, in several platforms discussed in the literature, priority is often given to those spatially close to the location, such as recent visitors [10, 41], those about to pass by [31, 38], or those currently nearby [34, 62], are often given priority for task assignments. This approach typically emphasizes the immediacy of the visit experience or observation. However, when the definition of ‘local’ is focused on long-term residents [1], this kind of experiential background is deemed particularly valuable for responding to queries that demand a high degree of specificity. Hence, when prioritizing the recruitment of ‘locals’, it is critical to identify the particular types of location inquiries that benefit from a specific kind of local insight. Essentially, this becomes a question of which experiential backgrounds are most valued for addressing specific types of location-related questions.

Secondly, exclusively depending on ‘locals’ to source contributors may not guarantee the best responses. Our study findings indicate that for certain location queries, traditional ‘local’ perspectives may not be sufficient. For example, objective information was perceived in our study as better provided by individuals who frequently visit the location, rather than those who are merely spatially close or long-term residents. Similarly, regular visitors or even one-time tourists can offer unique viewpoints that may be absent among long-term residents. If these visitors have been to various similar places, they can provide fresh insights and comparative experiences that would be beneficial to platform users in certain scenarios. Therefore, we suggest that platforms broaden their focus from a strict ‘local’ orientation to encompass a more diverse array of experiences and interactions with the location. By placing greater emphasis on ‘experience,’ a platform can refine its task assignment strategy, aligning tasks with contributors based on various aspects of their interaction with and understanding of the location. Additionally, when presenting multiple responses to information seekers on the platform, it can prioritize or suitably display contributions from those whose experiential backgrounds are most relevant to the specific nature of the location queries. Adopting this expanded focus could serve as an effective strategy to attract both contributors and information seekers. This message would signal that the platform welcomes anyone with relevant experience to answer specific location-related queries, and not just those who identify as ‘locals’. Logically, this would tend to increase the pool of potential respondents and result in more accurate and relevant contributions.

6.3 Design Implications

Our research findings have several design implications for platforms operating in the space of location-related mobile crowdsourcing. On that basis, this section proposes several design directions aimed at improving the user experience of those seeking information, and at making more effective use of input from contributors with diverse experience.

Our design recommendations can be divided into two primary strategies. First, our study underlines the potential benefits of customized task assignment and targeted application of contributors’ experiential background to better match platforms’ location-related information requests. Specifically, when task requests do not require real-time information, like those seeking general assessments of POIs or scenery, we propose that platforms take into consideration the characteristics of the information being sought, the specific aspects of description quality that the information seekers are likely to emphasize and care about, as well as the possible experiential background of contributors that could be perceived as beneficial by such seekers. We propose that platforms implement a scoring system to evaluate potential contributors based on how well their experiential backgrounds align with the requested information. In this system, contributors would receive scores reflecting the suitability of their experience, and then be ranked and selected for tasks accordingly. By employing such strategies, platforms can more effectively match contributors to the specific needs of information requests, thereby enhancing the likelihood that requesters receive relevant and applicable answers. The results from our study could serve as a foundational reference for designing such a scoring mechanism. This approach may increase user trust and perception of the platform’s utility. However, for the scoring system to be effective, the platform should keep track of contributors’ visit and mobility histories, or allow contributors to periodically report their experiences when responding to queries. This information can then be integrated into their profiles, ensuring that their scores in future tasks are as accurate and reflective of their real-world experiences as possible. This approach could be particularly advantageous for platforms that already push tasks to consumers’ devices and seek feedback post-exposure to the information, e.g., Google Maps¹¹, Local Guides¹², and rewards-based applications.

Second, beyond the task assignment phase, we recommend that platforms display the relevant aspects of a contributor’s experiential background alongside the information they provide. Specifically, platforms should organize information based on the type of aspect being sought. Using restaurant reviews as an example, the information can be organized into categories such as food quality, ambiance, pricing, crowdedness, product availability, among others. For each piece of content (e.g. a review), the platform could display indicators that emphasize the contributor’s relevant experiences, especially those experiences that are considered valuable for that particular type of content. This organized presentation, along with clear indicators of the contributors’ backgrounds, would assist users who focus on specific aspects of description quality, allowing them to find relevant information more efficiently. Additionally, it would help them to better determine the relevance and applicability of each piece of information in relation to their specific situations and requirements. Platform managers should also consider giving higher visibility and priority to information provided by contributors who have demonstrated relevant experiential background of addressing specific information needs. Such an approach would enable users to quickly find the most helpful descriptions from the most suitable contributors, thereby improving the overall efficiency of platform use.

In summary, our findings can serve as a valuable guide for platforms looking to implement these two strategies. As a reference encapsulating the results of this study, we have provided a summary table (Table  in the supplementary information) that organizes types of location-related information, information attributes, aspects of description quality, and aspects of contributors’ experiential background.

6.4 Limitations

The current study has several limitations that should be acknowledged. First, the range of information items we collected and summarized from prior literature and our exploratory interview study might not fully cover the diversity of information found on real location-related crowdsourcing platforms. This limits the generalizability of our findings. Moreover, all three phases of our study were conducted in our country. This geographic and cultural specificity raises questions about the generalizability of our results to other cultural contexts and regions of the world. Thus, future research should aim to collect a more comprehensive range of requested information from a wider variety of sources to enhance the applicability of the results, and in multiple world regions.

Second, our sample population had a gender imbalance, with a higher proportion of females (58.6% in the survey and 62.9% in the online study). Although this aligns with previous literature’s findings that women seem more motivated than men to seek online reviews [2, 36], it may limit the generalizability of our findings across genders. Finally, during the final phase of our study, we did not collect data about information attributes. This decision was made to avoid extending the length of the study excessively. However, it means that we did not capture the complete interrelationship of information attributes, description quality, and contributors’ experiential background. Future studies should aim to fill these gaps to arrive at a more comprehensive understanding.