Investigating Mixed Methods Research in Applied Linguistics: Methodological Avoidance and Possible Barriers in the Field

The field of applied linguistics is becoming increasingly transdisciplinary as recognition for the need to approach empirical questions from a variety of epistemological and theoretical perspectives grows (Douglas Fir Group, 2016). One methodological approach that holds promise for advancing sophisticated inquiry into complex issues of applied linguistics is mixed methods research (MMR); however, studies adopting MMR to its fullest potential remain infrequent. Employing an exploratory sequential mixed methods design that includes a focus group and survey questionnaire, this empirical study investigates the internal and external factors that may lead applied linguistics researchers to avoid conducting and/or publishing MMR. Integrated analyses revealed that participants’ methodological and publishing decisions were influenced by factors such as their socialization into research practices in graduate school, the pressure to publish, and the considerations of the research journal industry. Implications for future applied linguistics researcher education programs and the impact of the publishing industry on research agendas are discussed.


INTRODUCTION
The field of applied linguistics is becoming increasingly transdisciplinary as recognition for the need to approach empirical questions from a variety of epistemological and theoretical perspectives grows (Douglas Fir Group, 2016).In turn, the research method "paradigm wars" of the 1990s has given way to a more productive understanding of methodological diversity and sophistication (King & Mackey, 2016).One methodological approach that holds promise for advancing inquiry into complex issues is mixed methods research (MMR) because, fundamentally, MMR involves an intellectual and practical synthesis of qualitative and quantitative methods to examine an issue beyond what can be accomplished with either approach individually (Johnson et al., 2007).As a research methodology, MMR integrates qualitative and quantitative approaches at every level of inquiry, including forming research questions, collecting data, and interpreting findings (Hashemi & Babaii, 2013;Riazi, 2017;Tashakkori & Creswell, 2007).However, despite recent interest in MMR in applied linguistics (e.g., Brown, 2014;Farsani et al., 2021a;Riazi & Candlin, 2014) and applied linguists' general knowledge about MMR research (Farsani et al., 2021b), studies employing MMR to its fullest potential remain infrequent (Hashemi & Babaii, 2013;Mathieu & Gopalakrishnan, 2018;Riazi, 2017).As a field, this leads us to ask: What factors might contribute to applied linguistics researchers' hesitation in adopting MMR?
The present study investigates the internal and external factors that may lead applied linguistics researchers to avoid conducting MMR.To the best of our knowledge, this study is the first of its kind to systematically explore researchers' methodological decisions while conducting MMR or publishing it.As such, this study greatly informs our understanding of the realities of conducting MMR in the field and has important implications for considering structural and systematic barriers that might constrain researchers' abilities to approach questions from a methodologically robust perspective.Importantly, our intention is not to argue that MMR is a better or more productive research approach than others; however, MMR does have potential to add nuance to our understanding of applied linguistics phenomena.As background to the study, we review how several applied linguistics researchers have discussed "bridging the gap" in the study of second language (L2) teaching and learning, proposing MMR as one possible solution.We then define MMR itself and outline the previous literature on MMR in applied linguistics.

"Bridging the Gap" and Defining MMR
Research in sociology has shown that researchers' methodological preferences are developed through a variety of sociocultural and sociohistorical forces (Bryman, 2007).Moreover, as publications on the paradigm wars remind us (see Magnan, 2007), the ways in which applied linguistics research has historically approached questions of interest continue to impact how researchers are socialized into "habits of mind" that naturally mediate how they approach research and knowledge creation (Young, 2018).Young (2018) explains that what we attend to during research is grounded in "… our 'habits of mind' or our personal preferences as researchers and the early training we received" (p.32).Researchers with similar preferences, through mutual interaction, form part of what he calls a "thought-collective" and different thought-collectives result in different methodologies.Novice researchers are assimilated to the rhetoric, epistemology, and incommensurability of every thought-collective through early socialization.And, whereas these so-called paradigm wars may have expanded epistemological perspectives, thus giving way to a more methodologically balanced field, this debate may have also led to what some applied linguistics researchers (see Hulstijn et al., 2014) perceived to be a gap in the field.It is therefore not sufficient to simply "call" for more methodological mixing or layering (King & Mackey, 2016).Instead, structural changes in researcher preparation and research support and dissemination may be warranted if the field of applied linguistics truly intends to support greater methodological diversity.
In their multi-authored article on "bridging the gap," Hulstijn et al. (2014) discuss the different paths that researchers, themselves included, have taken in pursuit of their research questions and the extent to which they acknowledge a gap in the field.Instead of swearing fealty to a particular theory or methodological approach, which can polarize the field and limit the questions researchers ask, Husltijn et al. encourage readers to draw upon knowledge across disciplinary boundaries to answer real-world issues related to teaching and learning.One recommendation made by contributors to this article (see contributions by DeKeyser and Bigelow) is the use of a mixed methods approach.
MMR is a problem-centered approach (Leavy, 2017) which draws upon quantitative and qualitative approaches and integrates the datasets in a purposeful manner.However, despite its putative ability to "bridge the gap," this research approach remains underutilized in applied linguistics research and is often misunderstood and/or overly simplified in its definition.King and Mackey (2016) reflect on a common misconception about MMR in their acknowledgement that in applied linguistics "mixed methods has tended to refer to utilization of distinct research techniques in a single study," such as employing a survey coupled with follow-up interviews (p.210).In fact, mixed methods are frequently employed solely as a tool for triangulation, by which a phenomenon is studied via several methods to seek corroboration among results and eliminate potential bias (Riazi & Candlin, 2014).This approach has been subject to critique, however, for two main reasons.First, historically, very few MMR studies in applied linguistics have attempted an integration of results across quantitative and qualitative datasets and analyses, instead relying primarily on one dataset or analysis for results (Hashemi & Babaii, 2013).Second, it is challenging for researchers to convincingly argue that a construct can be similarly defined in both quantitative and qualitative strands in order for true triangulation to occur (Denzin, 2012).
Quality mixed methods studies are grounded in integration, or mixing, at several levels.Tashakkori and Creswell (2007) define MMR as "research in which the investigator collects and analyzes data, integrates the findings, and draws inferences using both qualitative and quantitative approaches in a single study…" (p.4).This is to say that whether conducted concurrently or sequentially4 , MMR studies require mixing of qualitative and quantitative approaches and methodologies, from forming research questions, to collecting and analyzing data, to interpretation (Hashemi & Babaii, 2013;Riazi, 2017).As such, King and Mackey's (2016) proposal of a "layered approach" that "demands the explicit consideration of research problems from a range of distinct epistemological perspectives" (p.210) is not so divergent from this robust understanding of MMR.

Previous MMR Literature
Although MMR has existed in applied linguistics research for decades, recent surveys of journals in the field have found that quality MMR studies remain infrequently published.For example, in an analysis of research articles published in seven international peer-reviewed applied linguistics journals from 1995-2008, Hashemi and Babaii (2013) found only 205 articles that utilized some type of mixing of qualitative and quantitative approaches.Moreover, a detailed content analysis revealed that the majority of those articles would best be described as quasi-mixed design (Tashakkori & Creswell, 2007), meaning that the quantitative and qualitative components were not interrelated in conceptualization nor execution at all levels of the study.Riazi and Candlin (2014) expanded this study by reviewing 40 papers published in 30 journals between 2002-2011 that included either "mixed methods" (n=18) or "quantitative and qualitative" (n=22) in the title, abstract, or methodology section.The authors found that most of these studies used mixed methods purely for triangulation.These studies did not "engag[e] closely with the theoretical basis" for such mixing of methods and merely "[brought] together different kinds of data and analysis in an additive way, without any sound or coherent conceptualization of the object of the study" (p.159).Mathieu and Gopalakrishnan (2018) built on these previous studies in analyzing the number of published MMR studies between 2012-2018.Ten leading journals were surveyed using the keywords "mixed method(s)" in the title, abstract, or methodology sections.This study found a mere 23 articles in these ten journals over a period of seven years.
Even though the above cited studies found an overall dearth in mixed methods publications, one study showed a rise in MMR in a small subdomain of applied linguistics.Cheng and Fox (2013) analyzed 24 doctoral dissertations produced in Canada from 2006-2011 within the field of language assessment.They found that 16 studies had adopted a pragmatic view by combining qualitative and quantitative methods.They concluded that these dissertation projects demonstrated "distinct strength" through their use of mixed methods and multiple stage studies (Cheng & Fox, 2013, p. 539).
Most recently, Riazi et al. (2020) analyzed 416 empirical articles from the Journal of English for Academic Purposes from 2002-2019.In their analysis, they differentiated between studies that merely combined qualitative and quantitative methods for the purpose of triangulation and MMR studies that provided a rationale for how the layering of different methods led to a better understanding of the research phenomenon.The results showed that while 51% of studies analyzed in the 18-year period used "combined methods," only 3% used MMR.
Surveying past literature on MMR brings to light a noticeable dissonance between movement toward interpretivist or transdisciplinary epistemological stances (Douglas Fir Group, 2016;Hulstijn et al., 2014) and methodological approaches in the field.On the one hand, we find several authors rejecting strict allegiance to a single methodological approach, touting the mixing of methods, adopting a pragmatic approach to research, and being led by research questions rather than methods.On the other hand, journal survey studies reveal that the number of studies adopting MMR is still sparse.The two studies that found a large proportion of MMR in its survey were Cheng and Fox (2013) and Riazi et al. (2020).However, the former study reviewed dissertations, not journal publications, and restricted its search to a very specific subdomain and geographical location.The latter mentioned that most studies merely combined methods, but did not show "how the data and results of the two methods [were] mixed to answer more complicated questions and to provide a more comprehensive understanding of the issue under investigation" (Riazi et al., 2020, p.15).The larger picture still suggests that MMR has not gained traction among applied linguistics researchers.
Those writing about MMR in applied linguistics have anecdotally proposed several challenges faced by researchers.Most common among these seem to be a lack of knowledge about the philosophical and methodological principles underlying MMR, a need to follow strict timelines for research projects, a limited skill set when it comes to research methods, difficulty bridging ontological divides, and publication issues (e.g., Mackey & Bryfonski, 2018;Mirhosseini, 2018;Riazi & Candlin, 2014).These proposed challenges may be well-conceived from scholars' experiences; however, there currently exists little empirical evidence to support them.The current study provides additional empirical evidence by directly investigating applied linguistics researchers' knowledge of, experience with, and perspectives on MMR.The question that guided our research was: what factors might limit applied linguistics researchers from conducting and/or publishing MMR?

METHODOLOGY
This study followed an exploratory sequential design to identify what factors influence whether applied linguistics researchers conduct and/or publish MMR.Exploratory sequential design is a methodological approach in which qualitative and quantitative data collection methods are incorporated in the study in a sequential manner (Riazi, 2017).Qualitative data is first gathered with the intent of exploring the topic.These data are then analyzed to identify the variables to be measured, which in turn inform the quantitative data collection methods.Finally, data from all sources are brought together for interpretation.This MMR design was appropriate for this study because little previous research had explored this topic, thus warranting an initial data-driven exploratory phase in order to determine initial themes.These themes then informed the subsequent quantitative phase of the study, with a first point of interface between the qualitative and quantitative components occurring there.The second point of interface occurred when the results of both the qualitative and quantitative data analyses were integrated during final interpretation.The following two sections describe the study methodology of these two phases in greater detail.

Phase One -Focus Group Discussion: Data Collection & Analysis
The first phase consisted of a focus group with six participants in the field of applied linguistics.A purposive sampling method (Davis, 2015) was used to recruit focus group participants who would represent various facets of the field of applied linguistics, such as language teacher education, classroom-based second language acquisition, and language policy.Criteria for purposeful sampling included diversity, current research interests, and current methodological approaches used in research.However, the majority of the participants attended graduate school at least 20 years ago, and more than half of the participants had received initial research training in quantitative methods.Table 1 provides information regarding the six participants who agreed to participate.

Yes
The goal of the focus group discussion was two-fold: (1) to gain better insight into the participants' reasons and rationale behind their methodological choices, and (2) to identify what factors encouraged or prevented them from adopting and/or publishing MMR.To this effect, the discussion began with their own training in research as doctoral students, how research methodologies had been taught during their graduate years, what methodologies they preferred today, if and how their methodological preferences had changed over the years, and the reasons for these choices.Participants' definitions of MMR were elicited, with a wide variety of responses.Then, specific questions about the use of MMR were also posed.For example, the group discussed if they had considered conducting MMR, and when, why, and how often they adopted MMR in their own research.Factors that have made the implementation and publishing of MMR challenging were also discussed.The discussion was audiotaped with the consent of the participants.
The discussion lasted for about one hour.It was transcribed and analyzed using MAXQDA (Version 2018.1.0).The thematic analysis focused on exploring factors that encouraged or limited researchers from adopting MMR and publishing it.We did not begin the coding process with predetermined codes, but rather adopted an inductive coding approach.The analysis aimed at understanding the larger themes that emerged from the conversation.All authors analyzed and coded the focus group transcript separately.We then discussed our individual codes, categorized similar codes, and merged them into one set of categories.A second round of coding was performed to identify larger patterns in the conversation.These patterns were then grouped to identify four themes: researcher education; research process; pursuit of tenure and career advancement; and publishing considerations.These themes informed the survey in Phase Two and were later re-interpreted holistically alongside the findings from the second phase.

Phase Two -Questionnaire: Data Collection & Analysis
An anonymous survey intentionally integrating etic and emic perspectives was developed after the focus group.Specifically, the barriers to conducting MMR found in Bryman's (2007) study and the themes that arose from the focus group discussion informed the creation of the survey.The survey consisted of multiple choice, yes/no/maybe, and open-ended questions and was separated into four different sections: respondents' graduate education experience (including questions related to researcher education and research process), research beliefs and practices (including questions related to researcher education and research process), researcher experiences (including questions related to research process, pursuit of tenure/career advancement, publishing considerations, time constraints, and grant applications), and demographics.It should be mentioned here that the participants were not asked to define MMR in the survey.Given that the focus group participants seemed hesitant in offering different definitions of MMR, we deliberately decided to not ask survey participants to define MMR.
The survey was sent out via email to 45 prominent applied linguistics departments at universities, colleges, and other research institutions in the United States and Canada.It was also shared on applied linguistics email listservs that may have reached an international audience.A total of 25 researchers responded, most of whom were experienced researchers with tenure at R1 doctoral universities.Table 2 provides a summary of the demographic data reported by these respondents.Quantitative analysis was generated using Qualtrics software (Version 12.2021), and descriptive statistical analysis is reported using proportions, also referred to as relative frequencies (Lock et al., 2015), allowing us to make comparisons relative to the proportion of our sample population, denoted as a percentage (%).In our tables, we include both the frequency counts (out of 25 survey respondents) and percentages.

RESULTS
The results from each of the data sources are first presented sequentially as this is how they were analyzed.Importantly, the data were not analyzed for triangulation.In other words, the survey results were analyzed independently from the focus group rather than as a confirmation or disconfirmation of the focus group results.The integrated interpretation in relation to the research questions is presented in the subsequent discussion section.

Findings from the Focus Group Discussion
As noted previously, the qualitative analysis of the focus group discussion yielded four salient themes: researcher education; research process; pursuit of tenure and career advancement; and publishing considerations.Each theme is briefly described below with supporting examples from the focus group transcript.

Researcher Education
Focus group participants highlighted the role their graduate education had played on their methodological socialization.Several noted being "pushed" toward qualitative or quantitative work while others recalled an "Alice in Wonderland approach" in which graduate students took one course each in a variety of methodologies.None of them mentioned MMR being part of the methodological training in their graduate programs.In addition to the methodology courses that had been offered in their graduate programs, methodological preferences of professors, advisors, and peers also influenced the participants' initial training as doctoral students.Three of the focus group members reported a stronger quantitative orientation in their graduate programs, and two of these referenced the shift in methodological trends in applied linguistics in the 1980s.Participant F said: "Yeah, I think that qualitative methods … were relatively new.This was in the 80's, the late 80's and um I had an advisor who just, still to this day, doesn't have any regard whatsoever for qualitative work so I was sort of pushed into the quantitative track and did six stats courses." In contrast, the focus group members who now primarily conduct qualitative research mentioned lacking the requisite statistical skills to conduct quantitative analyses without extensive collaboration.In turn, this lack of comfort with quantitative analysis subsequently affects their current graduate students.Participant C shared: "I have graduate students coming to me all the time and say … I want to do a mixed method study, and I'm very honest … and say you know, quantitative work is not my expertise.I can do my best to advise you and many of them do move forward with that.I don't feel very confident in my advising them around quantitative methods." Many participants indicated that their initial education in their graduate programs had been limited and depended on the research culture that had existed in their departments.Several reported that this initial training influenced how they advised their current students, inevitably and inadvertently socializing the novice researchers into their own "thought-collectives" (Young, 2018).

Research Process
Despite having narrow, or focused, graduate training in research methods, focus group participants reported having a change in methodological perspective from their graduate days to their current research.The focus group members who reported a more quantitative orientation in graduate school, studying cognitive second language acquisition or psycholinguistics, all experienced a major shift in their methodologies as they became interested in questions of language teacher education and language pedagogy.Several researchers agreed that while they started their career asking research questions that aligned with their methodological training, they later developed more methodological diversity as their research process became question driven.By centering their work around research questions and choosing a methodology that corresponded with these questions, some participants reported having adopted MMR in their recent studies.In this process, they had taught themselves new methodological approaches, thus broadening their research knowledge and practice.

Pursuit of Tenure and Career Advancement
Securing tenure was another factor that was reported as having an influence on how the participants conducted their research.Several researchers in the focus group explained that the pursuit of tenure comes with its own mandates on how much time a certain research project can take, how often publications are produced, and how much data gets reported.The requirement of having a certain number of publications and demonstrating research progress within a limited amount of time sometimes led them to truncate their research process.In the initial years of their career, participants did not always have time to teach themselves new methodologies and were under constant pressure to convert research projects into publications.
Conducting research, data analysis, and writing can all consume a lot of time on their own.When combined with the pressure of work, career advancement, and publishing, participants expressed getting frustrated with the time commitment of robust mixed methods studies.
"I find myself getting in the midst of it, I'm like oh my God this is taking so long.It's like when am I going to get a paper out of this right you know?That's the frustrating part.I feel like we just keep dragging out this analysis you know.It's just like when are we going to write the damn thing."(Participant F) Research projects, especially those that espouse different epistemological stances like MMR, require extensive time for data collection, analysis, and interpretation.Researchers may not have the luxury of time and can resort to modifying their research accordingly.

Publishing Considerations
The pressure to publish and the requirements of journals seemed to influence the methodological decisions that the focus group members made.Participants from research universities reported feeling more pressure to maintain a good publishing record than those from liberal arts colleges and mid-sized universities.This meant that researchers often did not have the time for multi-study projects and extensive data analyses that are characteristic of MMR.They reported constantly having to strike a balance between conducting meaningful research and maintaining their publishing record.Furthermore, publishing guidelines such as word limitations also determine how much data and analysis get published.Some participants reported that the ever-shrinking word count on journal articles and the pressure of publishing often forced them to divide a complex mixed methods study into two simple independent studies.Such reports from the research group members seemed to indicate that external factors such as journal mandates and publishing requirements of one's position exerted a strong influence on the kind of research that they conducted and disseminated.
The themes from the focus group highlighted various internal and external factors that may lead applied linguistics researchers to avoid conducting and/or publishing MMR.An individual's socialization into research practices in graduate school may point to one methodological path, but the focus group participants made clear that methodological preferences can change over a career, especially as researchers gain confidence in their work and the questions that they pose.Furthermore, the experiences of the focus group participants mirrored the broader epistemological and methodological trends in the applied linguistics field.Those participants who attended graduate school in the 1980's and 1990's reported more siloed methodological training, whereas those who more recently received their doctoral degrees had received more education on methodological diversity.Beyond their training, the factors that most seemed to influence the focus group participants' methodological decisions were those imposed externally by the pressure to publish and the constraints of the research journal industry.

Findings from the Survey Questionnaire
Recall that the second source of data was gathered using a survey questionnaire that was created based on the themes that arose from the focus group discussion and Bryam's (2007) study on barriers to conducting MMR.The findings from this survey are presented in the following order: graduate education experience, research beliefs and practices, and researcher experiences.As a reminder the abovementioned themes were integrated throughout the three sections of the survey.

Graduate Education Experience
Analysis of participants' responses regarding their graduate education training and experiences revealed that very few graduate programs offered a course or courses in MMR, even though research methodology courses were a required component of most respondents' graduate programs.In fact, most participants reported either not having had or not knowing that they had a faculty member who even had experience conducting MMR.Table 3 presents a relative frequency table depicting these findings.Proportions of our sample population, also known as relative frequencies (Lock et al., 2015), are represented as percentages; frequency counts are also included in parentheses beneath.When asked to further describe the research culture of their graduate program, participants were generally split between describing their training as strongly quantitative (48%) and moderately qualitative (48%), or conversely as strongly qualitative (48%) and moderately quantitative (48%).This trend was further corroborated by responses to survey questions related to how often participants' peers conducted quantitative and/or qualitative research during their graduate studies.A large majority of participants' peers (76%) either always or often conducted quantitative research, with the remaining number (24%) reported as only sometimes conducting such studies.Qualitative research appeared to be less frequent, with fewer of respondents' peers always or often (44%) conducting qualitative research and even more (48%) only sometimes conducting such studies.A few respondents reported that their peers never conducted qualitative research (8%).
With regard to MMR, a large majority (68%) of respondents characterized the culture of their graduate program as being moderately mixed methods, despite not having had MMR course offerings or faculty with expertise in that area (see Table 3 above).Very few participants (12%) chose to describe the research culture of their graduate program as being strongly mixed methods; in fact, more (20%) elected to describe it as being not at all important in their programs.

Research Beliefs and Practices
In relation to researchers' beliefs and practices, analysis of responses to epistemological statements using Likert scales revealed some common trends in agreement from our sample population.Table 4 presents these results under the umbrella categories of the research paradigms often associated with these views.Likert scale categories included strongly agree, agree, disagree, and strongly disagree.Unlike the first set of epistemological statements, there was far greater agreement among participants in response to statements targeting viewpoints often associated with an interpretivist/constructivist research paradigm.For each of these statements, there was a large or vast majority of participants who agreed or strongly agreed.For example, 60% agreed and 20% strongly agreed that the purpose of research is exploratory; 52% agreed and 36% strongly agreed that research is always a subjective enterprise; 44% agreed and 44% strongly agreed that their research is usually designed to provide a detailed understanding about a phenomenon; finally, 52% agreed and 20% strongly agreed that research aims at uncovering 'multiple realities' that are intersubjectively negotiated and constructed.
Overall, the strongest trends in agreement were demonstrated in response to epistemological statements targeting a more pragmatic research paradigm, which is often associated with MMR.All participants agreed (32%) or strongly agreed (68%) that research methodologies should be chosen in order to best solve the problem at hand; in fact, a large majority of participants strongly agreed with this statement.Another insightful trend in the data indicates that a vast majority of participants agreed (20%) or strongly agreed (68%) that research methodologies originating from different epistemological stances can be integrated.

Researcher Experiences
Analysis of researchers' responses to Likert scale survey questions regarding their research skills and experiences demonstrates an overall confidence level in conducting, analyzing, and/or critiquing MMR studies.Table 5 presents participants' responses to these questions.Likert scale categories included varying degrees of confidence: extreme, moderate, slight, and not confident.As is shown in Table 5, a large majority of participants reported moderate (40-52% of participants) to extreme degrees of confidence (24% of participants) in relation to conducting, analyzing, and/or critiquing MMR.If we recall from earlier, this confidence is in spite of the fact that a vast majority of participants (88%; see Table 3) had reported not ever having the opportunity to take a course on MMR during their graduate studies.Instead, they reported teaching themselves.When asked, Have you ever taught yourself a new methodology in order to answer a research question?, most participants (84%) reported yes, illustrating one way in which researchers continue to grow in their scholarship beyond their graduate school experiences.
Besides teaching themselves new methodologies, researchers may have also gained this confidence through their collaborative work with colleagues across departments and programs.Many respondents reported having either very frequently (16%) or sometimes (72%) collaborated with other researchers who held different methodological expertise, or reported very frequently (16%) or sometimes (52%) asking a colleague for support in analyzing data about which they were unsure.Survey results highlighting this collaborative disposition of participants are shown in Table 6.In spite of researchers' self-reported levels of confidence with MMR and their collaborative dispositions, other factors related to research design appear to possibly constrain participants and their choices of research methodologies.Table 7 displays participants' responses to questions targeting their approach to research design.As shown here, a vast majority of participants report always (24%) or often (64%) asking research questions that align with their preferred methodology, thereby allowing them to stay in their lane, so to speak.In fact, far fewer participants reported that their methodological approaches varied depending on their research questions.When further prompted about how often participants have changed their methodologies while in the midst of conducting a study to better answer their original research question, most reported rarely (40%) or never (44%) having done this.Other factors that appear to possibly constrain participants and their choices of research methodologies relate to publishing considerations, time constraints, and funding opportunities.These results are reported in Table 8.A vast majority of survey respondents reported that maintaining a consistent publishing record was extremely (80%) important in their job description.In light of this, it is not surprising that several participants reported sometimes (20%) or often (12%) needing to eliminate either the quantitative or qualitative portion of a mixed methods study in order to publish within a journal word limit.In addition to journal limits, several participants also recognized the timeconsuming nature of MMR work and reported that this is an extremely (16%) or moderately (24%) important consideration for them as they engage in such projects.A few participants also reported that funding and grant applications very frequently (8%) or sometimes (12%) played a role in determining their research methodologies.

DISCUSSION
Integrated data analysis indicates that a host of internal and external factors contribute to researchers' hesitation in conducting and publishing MMR.Individual factors include researchers' epistemological stance, research education and socialization, and preferred research practices.Very few participants reported MMR being part of their initial training.Participants seem to have been socialized into particular epistemological stances and research practices during their early training years, as suggested by Young (2018).However, they also seem to have taken a pragmatic approach to answering research questions in their recent work.Such changes in our participants' research trajectories reveal how these researchers "position themselves within the social processes of investigation and publication" (Young, 2018, p. 48) and substantiate Young's claim that habits of mind are not immutable.
Despite distancing themselves from their initial training and espousing MMR in their recent work, external factors such as prerequisites for securing tenure and considerations while publishing research work seemed to influence if and how frequently MMR got published.These external factors exerted pressure on researchers maintaining a good publishing record, on their path to securing tenure, and achieving the above two within a limited amount of time.Researchers often struck a balance between these external influences and individual methodological preferences in various ways.Some reported that when they found themselves lacking or less confident in certain research methods, they collaborated with peers with different methodological strengths.Others indicated having taught themselves new research methods through self-study or again, through peer collaboration.Participants of the focus group also made clear that other external factors such as journal constraints and grant applications at times influenced their decision to not conduct MMR.The survey results for these factors were more difficult to interpret, as participants did not indicate journal constraints or grant funding as impacting their study designs.
Interestingly, despite low indication that survey or focus group participants conducted MMR themselves, both groups reported being confident in identifying good MMR studies in their fields, thus corroborating Farsani et al.'s (2021b) recent claims of applied linguists' general knowledge about MMR research.Moreover, both focus group participants and survey respondents indicated that they taught themselves new research methods, probably in part because their methodological training in their graduate studies was narrowed to mostly qualitative or quantitative methods.Also, several researchers in both data sources agreed that research should be led by research questions and should aim at solving problems-an epistemological stance that undergirds MMR (Riazi, 2017).In other words, while researchers' confidence, methodological knowledge, and epistemological stance seemed to suggest that they might be in favor of implementing MMR, their actual research practice suggested the opposite.
One plausible explanation for this dissonance that surfaced from the data is the heavy influence of the external factors discussed above.Given the pressure of tenure, publishing, and limited time, researchers may hesitate to adopt methodological approaches that are long drawn out, that might not adhere with the requirements of journals, or that might not meet the preferences of funding organizations.Participants in Bryman's study (2007) reported similar hurdles in pursuing MMR.MMR often takes a long time as it involves gathering both qualitative and quantitative data from multiple sources, conducting different kinds of analyses, integrating them, and collecting additional data if needed.Researchers often are not in a position to spend such extended periods of time on studies, as they are constantly under pressure to bring out publications consistently.
Furthermore, focus group participants mentioned that many journals have methodological and epistemological leanings and that few journals are open to publishing studies that bring together different paradigms within the same study.This further affects how and what kind of studies are reported.Some focus group participants also mentioned that they have had to leave out part of their findings in a multi-methodological study.When journals are willing to publish mixed methods studies, word count limitations pose an additional challenge.Researchers may be forced to eliminate some findings and only report those that would fit within the word limit, or otherwise limit their descriptions of data integration and mixed analysis to demonstrate merely "combined methods" (cf.Riazi et al., 2021, p. 5).
A second interpretation of the reported confidence in identifying quality mixed methods studies by survey respondents-despite the lack of training and personal experience in the methodological approach-is that even experienced scholars in the field may hold misconceptions of what MMR actually entails.As argued previously by Riazi and Candlin (2014) and Hashemi and Babaii (2013), a majority of the research published as "mixed methods" is at best "quasimixed methods" with little theoretical conceptualization for integrating the two data sources.It seems possible that the field of applied linguistics has perpetuated a cyclical misunderstanding of MMR in which graduate programs prepare scholars with only a limited conceptualization of MMR, and then those scholars interpret the quality of-and recommend publication of-empirical articles in the field based on that conceptualization.
The findings from this study highlight that both internal and external factors affect how applied linguistics researchers approach and carry out MMR studies.While this may appear evident to those working in the field, this study is the first to empirically parse out specific factors in relation to MMR itself.We believe this is an important point to underscore, as without actual data to support what has been hitherto anecdotally shared and discussed, we do not believe that real change will occur in our field.For example, the lack of methods courses in graduate programs that discuss mixed methods or methodological diversity is a key factor that needs to be addressed if the field hopes to move toward more transdisciplinary work.Future studies should further investigate how applied linguistics graduate programs approach methodological socialization and in what ways future researchers are provided models of and opportunities to engage in quality MMR.Second, and related, it seems that more professional development opportunities for current applied linguistics researchers to understand the complexity of quality MMR that integrates methods theoretically and in practice are warranted.One example is Jang's (2018) pre-conference workshop at the American Association for Applied Linguistics Conference on integrative mixed methods research and analysis.In order for the field to take up MMR, more opportunities such as this will be required.Lastly, more research is needed to understand how the publishing process and pursuit of tenure is affecting the types of research pursued in applied linguistics.As long as journal considerations and publication pressures dictate research output, it seems unlikely that great movement toward more robust methodological approaches will occur.

LIMITATIONS OF THE STUDY
This study offers empirical data on the factors that might limit applied linguistics researchers from conducting or publishing MMR.However, the study is not without limitations, and future research is necessary to further understand how the field might need to adapt in order to respond to calls for transdisciplinary and layered approaches to applied linguistics inquiry (Douglas Fir Group, 2016;King & Mackey, 2016).First and foremost, the small sample size of the survey limits the interpretations that can be made.Second, the survey intentionally did not directly ask participants (1) how they define MMR and (2) whether they conduct MMR (this second question was intentionally embedded within other questions as one of the Likert scale options).These survey design decisions were made for several reasons.First, participants in the focus group appeared uncomfortable when asked to define MMR, and we were hesitant to alienate survey participants who may not feel confident offering a definition.Second, due to this expected lack of confidence and the abovementioned varying definitions for what MMR actually is, we determined that asking participants whether they conducted MMR directly would not yield useful data.However, upon analyzing survey responses, it was clear that both of these questions would have proved helpful for better interpreting participants' responses to other survey questions.

CONCLUSION
This study arose in light of the recent discussion in applied linguistics regarding methodological diversity and the need for more transdisciplinary approaches to pressing questions in the field (e.g., Douglas Fir Group, 2016;Hulstijn et al., 2014;King & Mackey, 2016).While MMR is certainly not a panacea to the issues raised in this discussion, it does offer potential for robust analysis of research questions by mixing epistemological and methodological approaches.Despite this potential, quality MMR remains relatively rare in applied linguistics research.To that end, this study was interested in better understanding the factors that might influence whether or not researchers adopted and/or published MMR.As the findings show, it is not one but a combination of individual and systemic factors that act as barriers in adopting MMR studies more frequently.These include lack of methodological training in MMR in graduate programs, limited professional development opportunities, complexities in the publishing process, and the pursuit of tenure.In conducting this study, it is also our hope that the graduate programs and research organizations in the field of applied linguistics start addressing these issues.Without them, novice and experienced researchers will continue facing barriers in adopting MMR.
statements targeting the epistemological viewpoints often associated with a post-positivist research paradigm, participants' responses appear to have varied more in terms of their agreement-especially in relation to the first two questions regarding hypothesis testing and generalization of test results to larger populations.Stronger trends of agreement are shown with regard to the objective nature of research (60% agreed; 16% strongly agreed), as are strong trends of disagreement with regard to uncovering the "absolute truth" (52% disagreed; 32% strongly disagreed).