Using advanced mixed methods approaches: Combining PLS-SEM and qualitative studies

The PLS-SEM method is a robust approach to researching consumer behavior. However, understanding a phenomenon only through a quantitative approach may not be sufficient in some situations. This work explains how PLS-SEM results can be enhanced by complementing a quantitative approach with other methods proposing an advanced mixed methods design. We provide an actionable guideline, illustrating the approach through three studies in a retail context and using both an exploratory and intervention design. Our results show how exploratory studies help produce relevant contingent concepts for PLS-SEM testing. Building on PLS-SEM findings, a final qualitative study offers fine-grained intervention insights. We present a checklist for researchers on integrating advanced mixed methods and PLS-SEM.


Introduction
The partial least squares structural equation modeling (PLS-SEM) method is now widely used in several disciplines (e.g., business, economics, computer science) to estimate complex models (Sarstedt, Hair, & Ringle, 2022).Over the years, this method has become an "essential tool" for marketing researchers (Sarstedt, Hair, Pick, et al., 2022).Since the rise of this method and the development of software such as SmartPLS (Ringle et al., 2022), researchers have continued to propose new procedures and techniques to demonstrate the method's robustness.The PLSpredict and the heterotrait-monotrait ratio (Franke & Sarstedt, 2019;Henseler et al., 2015;Ringle et al., 2023) are just a few examples that have made techniques like blindfolding or the Fornell-Larcker (1981) criterion for assessing discriminant validity obsolete.
SEM contains a lot of innovation but has also come with debates about the most suitable technique or procedure for testing a model.As a result, researchers can sometimes struggle to choose the right method between PLS-SEM and covariance-based SEM (CB-SEM) or, more specifically, when using PLS-SEM, regarding the use of formative versus reflective measurements or traditional PLS-SEM versus the consistent PLS-SEM algorithm (Dijkstra & Henseler, 2015;Sarstedt et al., 2016).These debates have led to the establishment of clear guidelines that enable researchers to make the best use of the PLS-SEM method (Guenther et al., 2023;Sarstedt, Hair, Pick, et al., 2022;Sarstedt, Hair, & Ringle, 2022).
Enriching the method with other data sources to provide fine-grained evidence and robustness is a new challenge.One answer to this challenge could lie in the use of mixed methods.A mixed methods design differs from a multimethod design because it combines quantitative and qualitative methods, allowing researchers to ask confirmatory and exploratory questions, thus verifying and generating theory in the same research project (Tashakkori & Teddlie, 2010).Following essential quality criteria (Hirose & Creswell, 2023), the mixed methods design can take different forms, such as convergent design, explanatory sequential design, exploratory sequential design, or intervention design (Creswell & Creswell, 2022).
Recent studies have shown the effectiveness of using CB-SEM or PLS-SEM combined with other techniques, such as realist inquiry (Brown et al., 2021), fuzzy-set qualitative comparative analysis (Lee et al., 2022), or necessary condition analysis (Richter et al., 2020), in explaining a phenomenon.
For instance, Lee et al. (2022) demonstrated how mixed methods using PLS-SEM and qualitative comparative analysis facilitate comparing the results.However, scholars must tackle methodological limits when using mixed methods that combine different approaches.For example, Brown et al. (2021) showed how combining the CB-SEM method with realist inquiry can raise epistemological questions because the former is a confirmatory approach, while the latter is based on a constructivist epistemology.
Limitations have mainly been identified regarding the method used to understand a phenomenon.However, even though the two methods are philosophically and methodologically opposed, such as realist inquiry and CB-SEM, they can have many potential applications in the social sciences when properly combined as part of a mixed methods approach (Brown et al., 2021).Yet SEM researchespecially PLS-SEM with mixed methods-is scarce.The situation is even more dire when combining classical qualitative studies with the PLS-SEM method, apart from developing measurement scales following Churchill's (1979) traditional paradigm or the new procedures developed by Rahman et al. (2022).
So far, mixed methods approaches using PLS-SEM in tandem with qualitative comparative analysis have generally used simultaneous or convergent designs (Duarte & Pinho, 2019;Lee et al., 2022).
These designs only enable a comparison of results between studies but do not make it possible for one study to be informed by the results of a previous one because they are based on analyses performed on a single data set.A legitimate question is why scholars need to use mixed methods with PLS-SEM.The main reasons for specifically using the PLS-SEM method have been the following: small sample size (47.70%),theory development and exploratory research (30.54%), and high model complexity (29.29%) (Sarstedt, Hair, Pick, et al., 2022).These reasons can also be seen as issues or limitations to be considered.For example, a model of high complexity tested using PLS-SEM is likely to be more powerful if followed by a qualitative study to enrich the explanation of inter-relationships, given that PLS-SEM is used to validate rather than describe a phenomenon.Another reason for using mixed methods with PLS-SEM is the nature of the data collection.PLS-SEM analyses based on survey data could present limitations regarding ecological validity (i.e., disconnected or not close enough to reality) (van Heerde et al., 2021).Similarly, a qualitative study can enrich and describe the results obtained through a quantitative study.For example, survey questions asking respondents to project themselves in a particular situation cannot provide results as accurately as what would be obtained using a field study.This work illustrates the value of mixed methods when studying a phenomenon with limited field access.Specifically, we examine mobile app integration in retail shopping.Recent studies show that over half of consumers in developed countries use in-store retail apps (Airship, 2023).Retailers want customers to use their apps to create value, increase engagement, and boost sales.At a minimum, apps should offer useful features.However, development requires investments, so retailers may use fictional apps and projective techniques to test effectiveness via PLS-SEM models before launch.Though efficient, fictional apps can lack ecological validity.Thus, we explore how mixed methods with PLS-SEM enable the study of a phenomenon within a projective framework and deliver results approximating a field study.
Our originality is using three successive studies, each overcoming an obstacle.First, an exploratory qualitative study constructs a contingent model to measure perceived value accurately.Second, a quantitative study uses fictional apps and projective techniques to address limited field access and confirm the model.Third, a qualitative study adds ecological value by complementing PLS-SEM results.
In conclusion, we provide an actionable checklist to help researchers enhance PLS-SEM using qualitative approaches through advanced mixed methods.

PLS-SEM method in combination with qualitative research
A mixed methods approach revolves around various combinations of qualitative and quantitative methods.For instance, the convergent design method runs qualitative and quantitative studies simultaneously and then compares the results (see Table 1).Currently, mixed methods approaches involving PLS use a convergent design with fuzzy-set qualitative comparative analysis or necessary condition analysis.These methods are often used in hospitality and tourism research (Lee et al., 2022), in the entrepreneurial domain (Drăgan et al., 2023), mobile services (Duarte & Pinho, 2019), transport services (Sukhov et al., 2023), or technology adoption (Richter et al., 2020).However, it is essential to highlight that both methods that use PLS-SEM are based on a single data set.Nevertheless, they are still identified as mixed methods because the use of two analysis methods allows them to mutually enrich each other through their specific features, which would not be possible using a single method (Morse, 2010).Mixed methods have also often been used to develop various measurement instruments following Churchill's (1979) procedures (Ishaq et al., 2023;Wang et al., 2024) or new procedures, such as the "perceived omnichannel customer experience" scale developed in eight phases by Rahman et al. (2022) with an exploratory sequential design.The exploratory sequential design is a mixed methods approach based on running a qualitative study and then a quantitative one to confirm (or not) the results (Creswell & Creswell, 2022).An intervention design takes a reversed approach by including a qualitative analysis after a quantitative one to provide fine-grained evidence (Creswell & Creswell, 2022).In the PLS-SEM realm, very few studies have used these mixed methods approaches by combining PLS-SEM results with other methods.
Table 2 presents the main papers using mixed methods with PLS-SEM.Most studies using PLS-SEM as part of a mixed methods approach combine this method with fuzzy-set qualitative comparative analysis (fsQCA) (Drăgan et al., 2023;Lee et al., 2022;Santos et al., 2023), necessary condition analysis (NCA) (Richter et al., 2020), or all three together (i.e., PLS-SEM, necessary condition analysis, and fuzzy-set qualitative comparative analysis) (Sukhov et al., 2023).Although this type of mixed methods makes it possible to compare the results obtained through two or three types of methods, they are generally limited to a single database.Moreover, the use of two or three methods on a single database increases the risk of having inconsistent results.For example, in their model of technology adoption, Richter et al.'s (2020) PLS-SEM model showed an insignificant effect of ease of use on the endogenous constructs: adoption intention and technology use.However, the necessary condition analysis results indicate for both endogenous constructs (intention and technology use) ease of use as a necessary condition construct for these two outcomes (Richter et al., 2020).Other approaches, such as sequential mixed methods, allow studies to be completed or enriched in relation to each other.When qualitative studies are first used, they generally help identify and design a quantitative study conducted using the SEM method (Brown et al., 2021;Pinto et al., 2014).When a quantitative study is used first, it helps researchers complement the quantitative results (Goodwin et al., 2013;Janakiraman et al., 2021) by answering research questions not covered by previous quantitative studies.Stathakopoulos et al. (2022) used advanced mixed methods with one qualitative and two quantitative studies.In this case, the qualitative study contributed to the design of the two quantitative studies by identifying the model dimensions in Study 2 and in-depth research required in Study 3 (Stathakopoulos et al., 2022).
The order in which qualitative and quantitative studies are used depends on the topic under study.
However, depending on the order or nature of the studies used, we did not obtain the same results.
For example, in the mixed methods advanced research of Stathakopoulos et al. (2022), the mission of the third study was to investigate a relationship not covered by Study 2, making it more difficult to compare, confirm, or enrich the survey-based results of Study 2. Instead, a qualitative study would have reinforced the results of the quantitative study and enriched them in depth.To overcome these issues, our work proposes an advanced mixed methods approach to show how PLS-SEM results based on one survey can be complemented by additional qualitative studies conducted both before and after the survey.We define this as an advanced mixed methods approach because it involves several types of methods in the same project (Creswell & Creswell, 2022).Our illustrative work in the retail realm combines both an exploratory sequential design and intervention design to show the benefits of each.

The context of investigation: An application to the retail context
The following three original studies primarily demonstrate the added value of our mixed methods approach in retailing: Study 1 is an exploratory study aiming to uncover the potential antecedents of the perceived value associated with using in-shop applications in retail.Study 2 tests a PLS-SEM model that explores the antecedents found in Study 1 and their influence on perceived value.Study 3 uses an intervention design that complements and enhances the results of Study 2 by providing fine-grained qualitative evidence.These studies focus on shopping apps through the prism of hybrid perceived value (Zauner et al., 2015).

Figure 1 The area of investigation for this work -Consumer perspective
This work lies at the intersection of three research areas that raise questions for scholars and practitioners (see Figure 1).First, prior research on mobile applications has examined technology acceptance for mobile shopping (Area 1) (Agrebi & Jallais, 2015;Shen, 2015).Gradually, studies have focused more specifically on relationships between mobile apps and perceived value and engagement (Area 2) (Ho et al., 2022;McLean, 2018;Ozturk et al., 2016).Second, introducing retail mobile apps has connected research in mobile apps (Area 1) and brick-and-mortar retailing (Area 3).
Studies of Areas 1 and 3 reveal mobile apps enable unplanned purchases (Hui et al., 2013), entertain shoppers, prolong store visits, and increase product browsing in stores (Grewal et al., 2018).Studying Areas 1, 2, and 3 together ia key challenge.For retailers, the goal is to develop context-specific mobile apps.This is difficult without field access, so fictional apps are needed to test efficacy.For researchers, fictional apps enable an understanding of how a single app can create customer value and engagement across diverse stores.Examining this complex phenomenon through a hypothetical app/context via surveys and PLS-SEM combines the three unexplored research areas.However, this approach reduces ecological validity.Mixed methods help compensate by enhancing PLS-SEM analyses and addressing ecological validity (i.e., ensuring applicability) (see Table 3).

Study 1
Study 1 represented the first step in the exploratory sequential design of the mixed methods design.
Study 1 aimed to identify benefit and cost dimensions associated with the use of shopping applications in retail and develop scale items for new constructs that might emerge or for existing constructs which items would need to be adapted to the context (i.e., mobile application usage in retail stores).We adopted a qualitative approach using the focus group method (Fern, 2001) to build more interactions between participants.

Data collection, design, and procedure
This first exploratory phase included three focus groups of 18 people overall with various profiles (i.e., students, employees, technicians, supervisors, and managers).Each focus group lasted around 60 minutes.We animated each focus group based on an interview guide, including various themes dealing with the use of mobile applications in stores, such as the opinions of participants on mobile applications already existing (e.g., Ikea), their current usage of mobile applications in stores, and the conditions under which they could use such applications in stores.We analyzed the data using NVivo, following Spiggle's methodology (1994), that is, categorization, dimensionalization, and integration of statements.

Results
Table 4 presents the benefits (i.e., social self, organization, hedonism) and costs (i.e., intrusiveness, ethics, effort) associated with using shopping applications in stores as a result of the qualitative exploratory study.

Implications and Conclusions
Study 1, which was a qualitative exploratory, provided several interesting insights compared with what has been found in the literature.Previous research on mobile applications focused on technological aspects, such as augmented reality (Dacko, 2017;Nikhashemi et al., 2021) and had different usage contexts, with the exception of physical stores (Ho et al., 2022;Le et al., 2022).Those focusing on physical stores dealt with the behavioral aspects (Grewal et al., 2018) and intentions (Kang et al., 2015) related to mobile usage.The aim of the qualitative study was to focus on perceived value and the specifics of the store's physical context.For example, the social self benefit takes another perspective in the retail context by referring to the capability of the mobile app to enhance the user's self-image.Exploratory Study 1 also permitted the adaptation of several dimensions to the retail context, especially regarding hedonism (Holbrook & Hirschman, 1982b;Lai, 1995;Richins, 1994b;Sheth et al., 1991), intrusiveness (Edwards et al., 2002;Truong & Simmons, 2010), effort (Kleijnen et al., 2007), and ethics (Hilton et al., 2013).As an illustration, in this context, the concept of ethics is associated with the negative effect of mobile applications on reducing retail staff.Because we presented several retail contexts where mobile applications can be used (i.e., shopping and convenience stores), Study 1 also highlighted the importance of context usage.For example, intrusiveness is more negative in convenience stores because of their frequency of visit compared with shopping stores.The organizational advantage is stronger in convenience stores because it enables organization prior to shopping, as one individual in the focus group pointed out: "Make a little list where you can check off, filter.This and that interest me, so when I get to the store I know straight away, ah I'm missing this, I've seen it."These preliminary exploratory results could have also been obtained through in-depth interviews until theoretical saturation was reached (see Saunders et al., 2018).
To validate these results, it is crucial to conduct a quantitative phase using a larger sample.By doing so, we can demonstrate the links between the antecedents (i.e., the benefits and costs found in Study 1) and their consequences (related to previous research).Moreover, we can test the differences between two types of stores (i.e., convenience and shopping stores) in terms of benefits and cost perceptions.
As a result, an exploratory sequential design was employed by using the findings of the qualitative phase to inform the research design and scale development in the quantitative phase.Beyond the identification of costs and benefits, Study 1 also enabled us to generate scale items for new constructs such as organization, but also to adapt existing scales, such as the hedonic dimension, to the specific context at hand (i.e., mobile applications used in retail stores).The findings and results have been discussed within their respective phases, given the sequential nature of the data.

Study 2
Study 2 was the second step of the exploratory sequential design (Creswell & Creswell, 2022).The goal here was to quantitatively confirm the exploratory results found in the previous qualitative phase.
Retrospectively, the effort cost was removed from Study 2 because the quantitative results were not in line with standard requirements of the permutation multigroup analysis test (Matthews, 2017).
For Study 2, we relied on a hybrid approach of perceived value, an alternative to both the unidimensional and multidimensional approaches (Zauner et al., 2015), integrating both benefits and costs into the measurement model.For the sake of the study, we used engagement as the final dependent variable and aim to show how the costs and benefits of shopping applications influence the overall perceived value associated with using the applications, which could influence engagement toward the retailer.
We relied on the exploratory qualitative study and existing theory to develop the hypotheses.
Regarding benefits, prior research has shown positive effects of social and hedonic benefits on overall mobile service usage (Cocosila & Trabelsi, 2016;Pura, 2005;Wang et al., 2013).Based on previous work and the qualitative results, we predict all three benefits will positively influence the perceived value of the mobile application.
Specifically, the organizational benefit allows consumers to organize purchases, saving time in-store.
The social benefit conveys a trendy, tech-savvy image, boosting self-esteem.Finally, the hedonic benefit provides fun, playful features.

H1.
The social self, organization, and hedonic benefits positively influence the perceived value associated with shopping applications.
Regarding costs, it is reasonable to assume that frequent in-store interruptions via notifications will inevitably irritate consumers.Furthermore, as was shown in the case of integration of self-service technologies (Hilton et al., 2013), mobile applications can, in their own way, contribute to the reduction of in-store staff.Indeed, if a mobile application allows customers to find products and make purchase decisions without going through a checkout, the retailer may reduce its staff.Intrusion in the sense of Truong and Simmons (2010) (i.e., including interruption and frequency aspects) and ethical issues can destroy value.

H2.
Intrusiveness and ethical costs negatively influence the perceived value of mobile shopping applications.
Prior research shows mobile apps can positively influence relationship commitment, self-brand connections (Kim & Baek, 2018), and attitudes and loyalty toward brands (McLean, 2018).Similar logic may apply in retail stores -if app benefits outweigh costs to create perceived value, consumers will likely continue using the app and shopping at the retailer, strengthening engagement.Therefore, we hypothesize: H3.The perceived value derived from shopping app usage positively influences engagement toward stores.
Finally, building on the qualitative findings and Copeland's (1923) and Bucklin's (1963) research, we compared how perceptions of shopping app usage may differ between convenience stores and shopping stores.Regarding the organizational benefit, customers visit convenience stores more often for smaller, quick purchases (Namin & Dehdashti, 2019).They are more likely to need app features like purchase lists, strengthening perceptions of the organizational benefit.In contrast, shopping at shopping stores involves more time, financial impact, and enjoyment (Namin & Dehdashti, 2019).
The longer time browsing and buying hedonic products with the app could increase perceptions of the hedonic benefit and value.
Convenience store retailers use mass advertising to implement promotion strategies that increase purchase volume (Murphy & Enis, 1986).In contrast, shopping store retailers tend to emphasize personal selling.Accordingly, customers could perceive the discount notifications they receive on their mobile devices while being in shopping stores as being less intrusive than the notifications they receive while shopping in convenience stores because a) they will receive notifications less often and b) the offers could reduce the monetary risk, which is higher in shopping stores.
Consumers' need for product information may be greater in shopping stores than in convenience stores, where they make regular, repeated purchases.In shopping stores, asking sellers for advice on choosing the right product is common.However, shopping applications can supply this advice through consumers' comments, product ratings, and product sheets.Shopping applications may have a harder time replacing salespeople's advice in convenience stores because there are fewer service needs than in shopping stores.
Consumers visit convenience stores more regularly than shopping stores, so their visit frequency is higher.Frequency is a dimension of store engagement; the more customers visit retail stores, the more likely they are to engage with them (Brodie et al., 2011).Accordingly, if consumers use shopping applications each time they visit convenience stores, their frequency of use should increase their engagement with the stores.Based on our qualitative findings and the above theorizing, we propose the following: H4.The context of shopping app usage (convenience vs. shopping stores) moderates all the relationships such that we can expect the following: -(H4a) a weaker effect of the organization benefit on perceived value in shopping stores than in convenience stores.
-(H4b) stronger effects of social self benefit on perceived value in shopping stores than in convenience stores.
-(H4c) stronger effects of hedonic benefit on perceived value in shopping stores than in convenience stores.
-(H4d) stronger effects for intrusiveness cost on perceived value in convenience stores than in shopping stores.
-(H4e) a stronger effect for ethical cost on perceived value in shopping stores than in convenience stores.
-(H4f) a stronger effect of perceived value on engagement toward the store in convenience stores than in shopping stores.

Data collection, design, and procedure
We created two shopping applications that could be used in two different scenarios, that is, one in a shopping store and the other in a convenience store.We described the scenarios created for each store/context in two 5-minute videos, which we wanted to be as immersive and realistic as possible.
To assess the perceived value associated with such applications, we used projective techniques traditionally used for psychological assessment (Weiner & Kuehnle, 1998).The participants were in undergraduate and postgraduate programs at universities and business schools, in accordance with what Bello et al. ( 2009) recommended (i.e., the use of student samples in "proximal" research is only problematic if they have no experience in the area).For the sample size, we relied on the inverse square root method (Kock & Hadaya, 2018) based on the path coefficient.Because this is a retrospective method, considering our smallest path coefficient, that is, 0.122 (for the effect of intrusiveness on perceived value), the method indicates a sample size required of 415 (Hair et al., 2022;Kock & Hadaya, 2018).Our actual sample size was 454 for a significance level of 5% at a power level of 80% (Hair et al., 2022).Regarding the multigroup analysis, despite employing equal sample sizes (i.e., 223 participants for shopping stores and 231 for convenience stores), the use of the same technique based on the path coefficient for sample size raises certain concerns.In the shopping store context, both the social self (β.103) and intrusiveness (β-.022) paths exhibited notably low values, even when considering a significance level of 10% (Hair et al., 2022).As a result, we must be cautious in interpreting the results relating to the effects of these two constructs on perceived value.
We followed Churchill's (1979) paradigm to develop and adapt a multi-item measurement for all constructs in the model.The organization dimension was new, but other dimensions such as hedonic were adapted from previous scales.Because the scales were adapted for specific tool usage (i.e., mobile applications) in two specific contexts, we followed the process of a traditional development scale (Churchill, 1979).We used SPSS 28 to develop and assess the scales' robustness.The scale was initially composed of 30 items, of which 19 were retained.We ran factor analyses using the Kaiser-Meyer-Olkin test and Bartlett's test of sphericity (Bartlett, 1954), score commonalities, VARIMAX rotation (Gerbing & Hamilton, 1996), and loadings above .6as the selection criteria.We tested the reliability of the scale using Cronbach's alpha.
We employed partial least squares structural equation modeling (PLS-SEM) to test our hypotheses using SmartPLS version 4.0 (Ringle et al., 2022).We used the path scheme (Henseler, 2010) and preferred the PLS-SEM algorithm to the consistent PLS-SEM version to avoid any bias (Sarstedt et al., 2016).We evaluated the reliability and validity of the constructs using Cronbach's alpha (α), composite reliability (rho_a and rho_c), and average variance extracted (AVE).For all measures, the α value was greater than .70(Hair et al., 2019;Nunnally & Bernstein, 1994), the CR exceeded .70(J.Hair et al., 2019), and the AVE was greater than .50(Hair et al., 2019).All loadings were greater than .70(Hulland, 1999), and we could confirm the discriminant validity of all constructs with the heterotrait-monotrait ratio (Franke & Sarstedt, 2019;Ringle et al., 2023) (see Table 5) and confidence intervals (Franke & Sarstedt, 2019) (see Table 6).We ran PLSpredict to test the predictive model quality (Shmueli et al., 2016(Shmueli et al., , 2019)).The model showed good predictive quality, which aligns with the recommendations (Shmueli et al., 2016(Shmueli et al., , 2019) ) (see appendices 1 and 2).Furthermore, we conducted a second assessment of predictive modeling: the cross-validated predictive ability test (CVPAT) on the overall model (Sharma et al., 2022).The PLS-SEM predictions significantly outperformed the naïve indicator-averages (IA) prediction benchmark for both endogenous constructs: perceived value and engagement/store.Regarding the linear model (LM) prediction benchmark, the PLS-SEM predictions demonstrated robust predictive validity for the perceived value construct and predictive validity for the engagement/store construct (refer to appendices 3 and 4 for details).Moreover, to test the statistical significance of the PLS-SEM results, we ran a bootstrapping procedure with 10,000 subsamples, the percentile bootstrap confidence interval method, a two-tailed test, and a significance level of 0.05 (Ringle et al., 2022) (see Table 7).Finally, to test our hypothesis on the differences between the two types of stores (H4), we ran a permutation multigroup analysis to determine whether the two data groups (convenience vs. shopping stores) exhibited significant differences in their group-specific parameter estimates (e.g., outer weights, outer loadings, and path coefficients) (see Table 8).We followed the procedure by starting with the generation of data groups, the three-step analysis of the measurement invariance of composite models (MICOM) (Henseler et al., 2016), and the analysis and interpretation of permutation results.Regarding the three-step MICOM procedure, all constructs passed step 2 and at least one of steps 3a or 3b, as recommended (Matthews, 2017) (see appendices 5 to 7).

Results
Hedonic, organizational, and social self-benefits significantly influenced perceived value, supporting H1.The hedonic benefit had the strongest effect.As predicted in H2, intrusiveness, and ethical costs also significantly affected perceived value.Perceived value significantly impacted store engagement, confirming H3.
The permutation multigroup analysis showed path differences were only significant for certain dimensions.Specifically, the organization, hedonic, and ethics paths did not differ significantly between groups, rejecting H4a, H4c, and H4e.However, the social self path strongly influenced the perceived value for convenience stores versus shopping stores, supporting H4b.Similarly, intrusiveness more strongly impacted the perceived value for convenience stores, confirming H4d.
Finally, perceived value had a greater effect on engagement for convenience stores, supporting H4f.
Finally, it is important to carefully read the results of H4b and H4d because the sample size for shopping stores is insufficient for the path coefficient to yield significant results at the 5% or 10% significance levels.

Implications and Conclusions
All the benefits extracted from Study 1 positively influenced the perceived value associated with the use of shopping applications, complementing previous research in the area conducted in online contexts (Cocosila & Trabelsi, 2016;McLean et al., 2018;Revels et al., 2010;Tojib & Tsarenko, 2012).The negative impact of costs on the perceived value associated with the use of shopping applications was in line with previous research (Hilton et al., 2013;Kleijnen et al., 2007;Truong & Simmons, 2010).Our results have also supported perceived value as influencing engagement with the store, which is in line with our qualitative exploratory evidence.Finally, the permutation multigroup analysis supported context or type of stores as affecting-and more specifically moderating-some of the links between the dimensions of perceived value and engagement with the store.
Although the PLS-SEM results were insightful and robust in terms of sample size, Study 2 had several limitations.First, while it was homogenous, a sample composed of students is not fully representative of the general population.Another limitation concerns the projective technique with the scenarios used.Following the methodological approach suggested in the present paper, we conducted an additional explanatory study to enhance the interpretation of the quantitative findings.

Study 3
The aim of Study 3 was to explore at a fine-grained level what can be done to implement effective shopping applications and answer consumers' needs and expectations in this area.The goal was to offer robustness in describing the phenomenon from a different lens by drawing meta-inferences.We adopted a qualitative approach by drawing on the focus group method (Fern, 2001) as an effective means of interpretation of the survey outcomes (see Creswell & Creswell, 2022).The focus group enabled us to observe group interactions, which is the hallmark of this method (Belzile & Öberg, 2012).As for Study 1, these explanatory results could have also been obtained through in-depth interviews until theoretical saturation was reached (see Saunders et al., 2018).

Data collection, design, and procedure
Eleven participants participated in a focus group with a common discussion protocol (see Cyr, 2019).
First, we formulated the general research questions based on the main goal of the research.The moderator followed a topic guide, aiming to elicit a discussion of the benefits and drawbacks of using shopping applications in a retail context.Second, shoppers were recruited through referrals (Aiello et al., 2020) with a screening question.Ultimately, the participants consisted of referred adults with different backgrounds, educational levels, occupations, and ages.
The focus group lasted for 2 hours, until we reached theoretical saturation.We began by exploring the main dimensions that emerged from the PLS-SEM investigation before then moving to the differences between shopping and convenience stores.Before the focus group began, the moderator explained the procedure and purpose of the session, ensuring that all the participants were aware of shopping applications.The session was transcribed into a word processing package to allow for NVivo content analysis (Bazeley & Jackson, 2013).The transcripts were independently read, analyzed, and compared, leading to interrater reliability measured with Cohen's kappa coefficient (0.81).Using a categorization process suggested by Brocato et al. (2012), recurring themes in the data were identified by listing items reflecting similar characteristics.We first open-coded all the data, which provided the basis for developing the coding framework.As we progressed through the analysis, our codes (i.e., planning, privacy concerns, indulgence, engagement, and informativeness) became more specific.

Results
Overall, the qualitative findings revealed that consumers saw shopping applications as potentially useful.Nonetheless, they highlighted the importance of seamless experiences.Customers were concerned about overspending, so the app can serve as a better (i.e., more diligent) persona: "It helps me stick with my initial plans without indulging in temptations.It is my perfect persona" (GR).
Accordingly, the participants were open to the app for organizational purposes.Indeed, as reported by one of the participants, "I can plan everything with a mobile shopping assistant, without wasting my time in the shop" (FV).This confirmed similar results from Studies 1 and 2 while also expending on them.
Regarding the specific contexts on where to use mobile shopping assistants, the participants saw some important privacy conditions: "I feel the app is keeping an eye on me.I do not want to be bombarded with pop-ups, and it is scary that-often the app tells me what to do or buy based on what I previously searched" (MM); another participant stated, "I hate receiving too many notifications.I want to get the info when I am the one who searches for it" (FP).Thus, consumers indeed felt shopping applications were intrusive, as already found in both Studies 1 and 2; however, at the same time, they saw their value when searching for information on the spot.While in Study 1 the intrusiveness dimension was already recognized, the privacy issue emerged mostly in Study 3.
Interestingly, only hedonism appeared to be specifically related to the case of shopping stores.Some participants were guided by what Namin and Dehdashti (2019) defined as the hedonism benefit: "For me, recharging batteries means spending good quality time in shopping stores.I love indulging in those sorts of purchasing experiences" (GB); "I want to make a great use of my time when I am in a convenience store, so I want to avoid the distraction of dealing also with a mobile assistant" (SC).
Although the participants in Study 3 were not concerned about workers who could be replaced by shopping applications, they did raise effort considerations: "I want a seamless experience.At present, those sorts of applications are not smooth enough, making it complicated or cumbersome to use them.
If they improve, then this value will make me more engaged" (RP).Similarly, "I care both about the design-pleasurable to use-and at the informativeness-the detail on the listed products-of the application.If those two aspects are there, then I am really engaged" (AN).This supports the notion that perceived value is positively related to future engagement.

Implications and Conclusions
Study 3 offered fine-grained details on the feelings that consumers feel toward shopping applications.
By offering an effective means of interpretation of the survey outcomes, we suggest that consumers focus on the ideal, looking for an app that helps them stick to their initial plan.There was an important concern about privacy and intrusiveness because consumers feel that pieces of information collected about them can also be used for further marketing purposes, such as subsequent notifications.
Interestingly, the only dimension that emerged as clearly different between shopping and convenience stores was hedonism, which was clearly higher in shopping stores.Finally, consumers were still unhappy about the average quality of shopping applications and called for more seamless and pleasurable experiences.In general, if the above hiccups were addressed by brands, the perceived value of the assistant application could translate into higher engagement.

Discussion and Conclusion
In the light of inductive, abductive, and deductive research design approaches, we observed four contributions of the advanced mixed methods involving the PLS-SEM method: exploration, confirmation, disconfirmation, and addition.These four contributions are elaborated in the subsequent three paragraphs that cover the following points: induction, deduction, and abduction.
Induction -Qualitative Study 1 employed an inductive approach to explore and identify specific concepts involved in using mobile applications in the retail context.As Bonoma (1985) stated, the inductive approach is "theory building" and is generally based on individual observations.Focus group techniques are a way of finding concepts to build theory through a limited number of people's observations or discourse analysis.Study 1 found contingent concepts, such as the social self, which take a different perspective in the retail context, or ethical issues related to staff reduction, which only arise in this context.
Deduction -Quantitative Study 2 was a deductive approach with the objective of testing the theory (Bonoma, 1985).Several hypotheses have been derived from previous research studies.In the deductive approach, we sought to confirm the findings of Study 1.For instance, Study 1 raised ethical concerns, particularly regarding the adverse impact of shopping applications on the reduction of retail store staff.The confirmation of this result can be found in Study 2, where the ethical cost had a negative effect on perceived value.
Abduction -Qualitative study 3 adopted an abductive approach.Abduction is "the middle ground between induction and deduction" (Patokorpi & Ahvenainen, 2009, p. 126).As a result, Study 3 undertook an exploration of the subject, guided by the prior results of Studies 1 and 2. This made the task more complex, splitting the aim into three objectives: to confirm, disconfirm, and enrich.Specifically, the results encourage developing retail apps that create customer value and engagement.
However, demonstrating this benefit required context-specific measurement tools, not a generic model.The contingent value approach shows how individuals evaluate a tool in context.For example, excluding social and ethical dimensions overlooks their potential impact.
In summary, the methodology allows lower-risk app testing and development.The context-specific results then show apps can increase customer value and engagement.However, these actionable findings required an innovative methodology combining projective techniques, quantitative surveys, and qualitative confirmation.The methodological approach was key to producing tangible, contextualized learnings for managers.
Based on previous research and our use of a mixed methods approach, including PLS-SEM, we highlight several methodological implications in the form of questions: How do you conduct a mixed method approach with PLS-SEM?How many studies are needed?What factors need to be taken into account when using mixed methods?How can the results be put into perspective?
Before addressing the questions specific to using mixed methods with PLS-SEM, researchers should ask themselves whether they really need to use a mixed methods approach.Table 9 presents a mixed methods design checklist to help answer these and other questions.
The question of how to conduct a mixed methods with PLS-SEM refers to the beginning of the project.Should we start with a qualitative or quantitative study?The researcher must know whether the subject has been extensively researched or the context calls for a specific study (see Figure 2).If this is the case, a qualitative study would be the first step before using the PLS-SEM method because it would play a crucial role in the conduct of the quantitative study.Study 1 was necessary to highlight specific elements concerning the dimensions (i.e., specific benefits and costs) and contexts of mobile app use in physical stores.The same applies to Pinto et al. (2014), who had to identify the dimensions to conduct their quantitative study.Hence, when there is a requirement to explore the topic, an inductive approach becomes necessary.Other studies adopted an abductive approach such as that by Janakiraman et al. (2021), which started directly with a quantitative study using the PLS-SEM method and then completed part of the research questions using a qualitative study.Based on this, the approach should be progressive, taking the researcher step by step into discovering and explaining the phenomenon.
The question of how many studies should be carried out is not easy.According to Table 2, with the exception of Stathakopoulos et al. (2022), who conducted three studies, most mixed methods applications consist of two studies, and usually, there is a comparison between both methods using the same data set.Although comparative methods can have many advantages, an additional study always provides new elements to address the limitations of the previous one.However, the number is not always the answer.Stathakopoulos et al. (2022) conducted three studies using mixed methods, but the last two studies were purely quantitative and complementary.In our case, with the approach being progressive, our third study had to be qualitative to enrich the quantitative Study 2. Finally, having three studies increases the triangulation of all the data collected and the likelihood of having solid results to offer.
When carrying out mixed methods research with SEM, some aspects need to be considered.For example, Brown et al. (2021) showed that epistemological issues need to be considered when conducting qualitative research with a SEM combination.Two opposing visions can produce the opposite results in studying the same phenomenon.Our Studies 1 and 2 revealed that, when conducting traditional qualitative research without a specific approach, such as realist inquiry, there is no problem in terms of epistemology.In addition, for PLS-SEM results, several parameters must be considered to evaluate the model (see the guidelines in Sarstedt et al., 2022).
Finally, to put the results into perspective, the findings of all the studies must be taken into account, along with their specific features.For example, we should not expect in-depth explanations of quantitative results or representative results from qualitative studies.The execution of an exploratory sequential plan, complemented by an intervention plan, is a prerequisite for confirming, refuting, and improving research findings.We encourage researchers to put their results into perspective with the circle of confirmation, disconfirmation, and addition.This approach has effectively highlighted what each study brings to the other and how their interrelationships feed each other.In our case, the inherent limitations of the projective techniques used in Study 2 were overcome in Studies 1 and 3 so that the combination of all studies can reduce possible biases.In this work, we have demonstrated the significant value of utilizing PLS-SEM in conjunction with qualitative studies.We hope this will help researchers make better use of mixed methods approaches with PLS-SEM in the future.When using PLS-SEM, you should verify the validity parameters of your model (see Sarstedt, Hair, Pick, et al., 2022).A qualitative study routed onto the previous quantitative research findings increases the explanatory power, providing fine-grained detail for your findings.

Put in perspective your studies
What did each study add to your research questions?How do the studies, taken together, strengthen your overall empirical package?
The researcher must show how each study responds to the research questions and what it adds (i.e., confirming, disconfirming, or adding new results).Each additional study must be justified by clarifying its need.

Figure 2 .
Figure 2. Using a mixed methods approach with PLS-SEM.A decision tree.

Table 1
Advantages and drawbacks of the main mixed methods approaches

Table 4
Costs and benefits found in the exploratory study(Study 1)
Primarily, the organization, intrusiveness, perceived value, and engagement effects were all strongly confirmed by Study 3. Second, Study 3's findings did not reveal any ethical concerns, thereby disconfirming the results of Studies 1 and 2. Study 3 further disconfirmed the findings of Study 2 concerning the effort cost while corroborating the results of Study 1 regarding the same construct.Third, the " exploration " aspect in Study 3 centered on "enrichment" because it capitalized on the groundwork laid by Studies 1 and 2. As a result, Study 3 deepened comprehension of the phenomenon by employing an abductive approach; this was done by advancing the understanding of shopping applications' roles and utility in retail stores by offering supplementary insights into concerns related to overspending, highlighting how these concerns can be alleviated through mobile app usage.Regarding managerial implications, we must distinguish between what the methodology provides andwhat the results show.The mixed-method approach enables retailers to projectively test a fictional mobile app's effectiveness, reducing launch risks.The early qualitative study allows testing in a realistic context.Quantitative results are more robust when confirmed and enriched by an additional qualitative study.

Table 9 .
Mixed methods design checklist.If the risk of finding the same results in both studies is similar, prefer successive methods (explanatory, exploratory, and intervention designs).If one study has a weakness that can be addressed through multiple studies (e.g., not enough theory to derive hypotheses, issues with sample size, and no fine-grained detail), you should address this point with another study.Sometimes, a topic needs to be conceptualized first.Therefore, a quantitative PLS-SEM study cannot be conducted first.A qualitative study (either a focus group or in-depth interviews) allows you to explore the topic first.In this case, you first adopt an inductive approach and then deductive in the quantitative phase.