Application of Mixed Methods in Transdisciplinary Research Projects on Sustainable Mobility

Abstract

Urban mobility systems are facing far-reaching structural transformations: There is the challenge of managing the growing volume of traffic and the associated environmental and social problems. On the one hand, novel micro-mobility services such as light electric vehicles (LEVs) show high potential for reducing emissions, e.g., through increased energy efficiency. On the other hand, they require change of urban mobility systems, e.g., through increasing shared concepts. The mixed methods approach is considered a good methodology for exploring the transformation of urban mobility systems since it can help to answer both technical and societal questions. Two transdisciplinary projects using the mixed method design were evaluated to improve the research for future mobility. To provide a critical reflection of the projects, a catalog of quality criteria was used to evaluate the research. This catalog was evaluated using structured interviews with project participants from research, practice and civil society. The results show that the combination of applied methods enables a comprehensive multi-perspective sustainability evaluation of urban micro-mobility concepts. The need for an early participatory research design was also evaluated. The result is an exemplary research design and a methodological guideline for the successful application of mixed methods for transformation research in the field of urban mobility systems.

1. Introduction

Urban transportation is facing major challenges due to the increasing volume of traffic and the dominance of private motorized transport. In Germany, the greenhouse gas emissions (GHG) of the transport sector have remained high since 1990 and thus are contributing to climate change despite efficiency gains. In addition, air pollutant emissions and noise pollution are a burden to health [1]. The high volume of urban traffic causes congestion and high space demands [2]. To address these challenges, research in the field of micro-mobility, with solutions such as sharing systems for light electric vehicles (LEVs), has become increasingly important [3]. Sharing and logistic services with LEVs based on digital platforms are a promising solution for urban transport transformation since they can be more environmentally friendly compared to other transport modes [4] and helpful for individuals, especially in times of traffic congestion [5]. According to the German Advisory Council on Global Change (German: Wissenschaftlicher Beirat der Bundesregierung Globale Umweltveränderungen, WBGU), there is a gap in sustainability-oriented research on digital platforms as well as criteria and schemes for their evaluation [6]. The positive impact of e-scooters is especially questioned in the public debate. They are criticized as a safety risk and for their lack of environmental sustainability due to a short service life and inefficient operating concepts. In particular, inefficient energy supply systems such as the so-called milk run, i.e., battery swapping with the help of vans, as well as charging with electricity of unknown origin have a negative impact on the life cycle assessment as well as economic efficiency [7,8,9]. Simultaneously, access to mobility services and acceptance of LEVs is still limited as people see electric vehicles in a negative light because of the costs, limited range and security concerns. LEVs are mostly used for entertainment or private use as a substitute for walking and not for transport. However, there is an indication that LEVs such as e-scooters have a high potential for individuals to change their mobility patterns [10,11,12,13,14,15,16]. According to Brost et al. [17], LEVs have the potential to reduce emissions from car passenger trips by around 44%. Around 57 mil. tCO₂eq per year can be avoided by substituting LEVs for heavy duty vehicles with internal combustion engines.

There are many operators in the field of mobility and energy services with different vehicles, batteries and energy supply systems as well as business operating models. Thus, on the one hand, there are scientific questions on the topics of decarbonization, and on the other hand, there are societal challenges, especially regarding social acceptance. Therefore, a transdisciplinary approach towards the analysis of LEV sharing, in which the scientific questions and societal problems can be synthesized and answered across the disciplines, is warranted [18,19,20].

1.1. Transdisciplinary Research

Transdisciplinary research can solve problems which cannot be solved in one discipline as the approach is able to connect different disciplines and stakeholders and guarantees a sufficient and efficient consideration of all relevant factors for real-world problems [21,22,23]. Transdisciplinary research projects combine the knowledge of researchers, practice partners and civil actors and involves them as early as possible in the research process [24]. Due to the collaboration between societal stakeholders and researchers, it is possible to obtain results that are more sustainable, real-world concerning and long-lasting [25] and therefore to develop robust solutions for sustainable transitions [26]. However, this transdisciplinary approach is difficult to implement methodologically due to the different disciplines and complexity of the sustainability questions. Therefore, it is important to define criteria for the implementation of those projects [27]. Thus, the application of a transdisciplinary approach in the field of sustainable mobility solutions is very challenging [24].

1.2. Mixed Methods

Methodological approaches can come from both quantitative and qualitative disciplines. Quantitative data are used to explain certain theoretical aspects and to develop measures for standardization and generalization of research results [28]. Qualitative research designs are characterized by circularity, which means that research design, questions, surveys, evaluations and interpretations are adapted during the research process and cannot be dealt with sequentially [29]. Furthermore, qualitative research designs arise from the limitations of quantitative research methods [28]. They are mostly used to study the structures and the functioning of social systems [30].

The mixed methods approach involves the combination of both quantitative and qualitative data in the research design [31,32,33,34]. The structure of mixed methods studies differs depending on the target and how the methods are combined. They are distinguished in various aspects—e.g., if methods are combined during research or only during analysis, and how data are related [35]. There are different types of designs which are either sequential or concurrent depending on how data are used [36]. If qualitative data are used to explain quantitative data, it is called explanatory design [36,37]. It is a complementary application of methods in which they enhance each other and allow a more comprehensive picture of the research subject [35]. This is in contrast to the multimethod approach, in which data are merely collected simultaneously and not correlated [38]. These systematic interrelationships of partial studies should make it possible to use the weaknesses and strengths of the different methods and to analyze the research object comprehensively. Nevertheless, a fixed definition and an associated research paradigm by means of methodological models for carrying out the research do not yet exist [35]. Mixed method studies should consider the associated quality criteria from quantitative and qualitative studies as well as the criteria for the mixed methods approach [39]. Additionally, the mixed methods design poses a special challenge for research projects with regard to the quality of results [40].

1.3. Transdisciplinary and Mixed Methods Framework

Due to the challenge of answering the sustainability questions in the transport sector, it is not possible to use only one methodology. Therefore, it is appropriate to use mixed methods to combine the different methods within a rule-guided approach. The different methods arise because, on the one hand, there are questions to be answered concerning the environmental impact of technologies, and on the other hand, social issues such as user acceptance must be considered. However, there is a difference in proceeding either to analyze environmental impacts or social issues (See Section 2). Mixed methods therefore are an option for strengthening the informative value of data regarding the changes of the transport sector. For example, movement data and user intentions can be linked together [41,42]. Travel behavior, social interactions and support systems can be linked for the improvement of sustainable transport systems through understanding of intrinsic complexities of people’s mobility strategies. This information can be used to develop new methods to transform urban mobility [43]. In addition, mixed methods can help to analyze the sustainable and successful management of business models [44] and enable insights into the organizational structures and cooperation necessities in business projects [45,46]. As a result, studies that use research and business model analysis can ensure that the main criteria and barriers for an overarching transformation of the mobility sector are identified and that resource-efficient, open, innovative business models that link the various stakeholders are developed [47,48]. Through this, products can be designed and implemented in such a way that as many people as possible benefit from the results and the negative effects are minimized through social assessment [49]. Additionally, decarbonization, including the demand for more electricity from renewable sources [50,51] and social questions regarding the need to change mobility patterns [12,15], can be linked and usable solutions can be developed.

The transdisciplinary approach, on the one hand, and the application of different methods within a mixed methods framework, on the other hand, are very challenging due to the high number of stakeholders and methods involved. This leads to the research question regarding how mixed methods in transdisciplinary research can be implemented and general rules for the application be found. For transdisciplinary projects, there are existing guidelines to define the research process [23,52,53,54,55]. For the mixed methods research, there are several guidelines to assess the quality of mixed methods papers within the review process [39,56,57]. Guidelines are available to review research projects in information systems [58] or to evaluate research projects with the validation framework to validate the outcome of those projects from an outside perspective [59]. To find a guideline for transdisciplinary mobility projects, two case studies are analyzed which used the mixed method research design to carry out measures and a methodological guideline for the successful application of mixed methods in transformation research in the field of urban mobility.

2. Case Studies

The Sustainable Technologies Laboratory (STL) at Bochum University of Applied Sciences (HSBO) is researching the challenges of urban transportation outlined above as well as the necessary transformation process in various transdisciplinary research projects. This case study focuses on the projects “BaaS for LEV-Sharing” (BaaS-LEV) and “SciSusMob—Smart City Sustainable Mobility” (SciSusMob). BaaS-LEV is an exploratory pre-study, while the main study, SciSusMob, is currently still in progress.

The projects aim to research how sustainability of urban mobility can be increased by linking different personal mobility and logistical services with LEVs and novel concepts for regenerative energy supply into a holistic product service system (PSS). This PSS should combine different LEVs (e.g., e-scooters, mopeds, cargo bikes, e-bikes), charging infrastructure (e.g., solar charging stations (SCS) and battery swapping stations (BSS) as well as different users (e.g., private users, logistics) and providers (e.g., sharing provider, energy supplier). The PSS is referred to as the LEV sharing system below.

In line with the transdisciplinary approach, various stakeholders from the field of urban mobility and energy supply are involved in the projects, as shown in Figure 1. Primarily, these partners support and participate in the real-world laboratory envisaged in the project. Furthermore, the goal is to involve the partners in the development of the research design. During the project, their requirements are collected based on workshops, interviews and surveys, and the project results may be communicated and reflected on regularly.

The projects of the case study follow the mixed methods approach. Figure 2 shows the exact implementation of the methods within the two projects.

To achieve this implementation, different qualitative and quantitative research methods that might be suitable for the challenges addressed in the research projects were analyzed, as shown in Figure 3. The justification and application of the methods are subsequently shown in

Table 1. Methods in the research projects.

Justification and Application
Overview of the state of the art is crucial to have basic information about the LEV sharing systems, and an overview of the stakeholders for analysis and design of expert interviews. Therefore, relevant search engines and journals were used to obtain the overview of the challenges in the transport sector. Results are assessed according to the criteria for sustainable business models and initial findings on aspects that need to be considered as requirements for LEV sharing services as well as knowledge about relevant stakeholders and necessary integration of citizens.
2. The expert interviews serve to enable a basis for the development of sustainable business models in this area. They are particularly used for obtaining information about current and previously unexplored questions when there are no theoretical assumptions or reliable empirical data available [53]. They were conducted to supplement the information from the literature review and helped to gain data about framework conditions which can be used in a real-world laboratory. Therefore, interviews with relevant stakeholders such as sharing services and energy suppliers are used within SciSusMob.
3. Quantitative surveys are used to enable statistical analysis of many cases [54]. This methodology was therefore chosen to obtain an initial overview, since focus groups are more suitable for information that is already available or for more detailed questions. The user surveys were conducted at the HSBO. Users were asked about their attitudes towards swapping batteries in BSS and SCS as well as wireless charging solutions. The results show that most users would prefer SCS due to the simplicity of charging via cables, but non-users indicate that they would more likely use BSS systems than SCS. In general, most people are willing to use alternative energy supply concepts.
4. Scenario analyses show the long-term development of different scenarios. Thus, different trends can be presented which help to foresee future developments and to consider the variety and different dynamics [60]. Sustainability issues dealing with global warming and decarbonization are a main topic for the application of scenario analyses. For example, there are publications on energy-related topics such as the world energy outlook (WEO) [61] or research focusing on decarbonization in the transport sector [62,63] or on the impact of decarbonization on human health [64]. With the data from the user surveys, possible future developments of LEV sharing systems within urban mobility in Germany were analyzed. They considered the aspects of social acceptance as well as aspects from the expert interviews.
5. Prototypes are used to obtain and test the first model of a product. As certain requirements result from the expert interviews (2), the user surveys and the scenario analysis (3) and (4) a prototype must be created which fits those. These include the design of the optimized charging infrastructure and operational concepts for LEV sharing systems which are tested in a real-world laboratory (6). Within BaaS-LEV, two prototypes for energy supply concepts were installed at the HSBO and evaluated through an application study with users at the SLT.
6. A real world-laboratory is a short-term testing space where innovative products and processes can be tested under real-world conditions. It is an experimental space where the legal framework as well as conditions can be tested without any outside pressure. They are part of transdisciplinary processes and help to test, evaluate and observe new technologies and societal solutions [65,66]. The real-world laboratory in the above-mentioned research projects takes on several roles. On the one hand, it serves to test the concepts developed in the scenario analysis. On the other hand, the data on user acceptance and technical solutions can be collected. These data help to increase the sustainability of the whole sharing system as data are provided for the LCA and the Sustainability Assessment. The collection of usage patterns can be evaluated to make sure that LEV sharing services meet the requirements of the environment. In contrast to the scenario analysis, real data and behavioral patterns of users can be analyzed. In addition, a long-term evaluation that is not within a test or an experiment is also possible.
7. LCA is a methodology to quantify the environmental impacts of products, processes or services [67]. For the decarbonization of the transport sector, analysis of the actual environmental impacts is important for developing products and services that are as environmentally friendly as possible. A literature review can help to gain an initial overview. However, to make the environmental impacts of the prototypes concrete and to consider all aspects of sustainability, new surveys are necessary. This is important for the LEV-sharing services as well as for the charging infrastructure. Therefore, several LCA studies were conducted at the SLT. These studies analyze the environmental impact of electric moped sharing services [4] as well as e-scooter sharing services [68]. In addition, a study about the environmental impacts of SCS was conducted [69]. The result show a high potential to reduce emissions if charging infrastructure and substitution are raised.
8. In addition to the ecological sustainability dimension, which is covered by the LCA, sustainability assessment deals with the economic and social aspects of products, processes and services which are also required for sustainable development [70]. Thus, it is the synthesis of the preceding methodologies to enable an all-encompassing analysis of the sharing services and energy supply concepts in the sense of sustainable development. It is based on the results from the literature review and the data collected in the expert interviews, the user surveys and the real-world-laboratory

.

3. Materials and Methods

This study was conducted as an explanatory and qualitative design to find out how successful the transdisciplinary approach and mixed methods design that were used in the projects were and which possibilities and general measurements could improve further projects. Figure 4 gives an overview of the study design. The qualitative approach was used for the data analysis as the contextual knowledge of the project participants was important. For this purpose, eight semi-structured expert interviews with researchers and practice partners of the project were conducted between May and July 2021.

3.1. Preparation

To evaluate the research projects, a criteria catalog was prepared using guidelines that help evaluate transdisciplinary research [21,53,54,55], resulting in three transdisciplinary categories: “real-world problems and relevance”, “transdisciplinary cooperation” and “transfer” as well as one methodological category: “methodological evaluation”. The interview questions have been developed based on these categories.

3.2. Sample Selection and Recruitment

For the selection of the sample, a stakeholder analysis was used to figure out the relevant stakeholders that were involved in the project (Figure 1). The analysis resulted in 10 practice partners and five researchers and research assistants as possible interview partners. A total of five requests were sent via E-mail to the practice partners. In each of these E-mails, an accompanying set of material was received, which should give an overview of the projects and of some of the transdisciplinary terms and the mixed methods approach. This should have helped the recipients to answer the questions in the interview. The questions were not included but could be sent along per request. None of the practice partners asked for that. The participation was voluntary. Only one of the contacted five practice partners said he did not know how to contribute to the evaluation and declined to be interviewed. Researchers and research assistants were contacted personally. They also received the accompanying interview materials and, if wanted, the interview questions to prepare. In total, eight persons were interviewed (

Table 2. Overview of the interviewed stakeholders.

Code	Stakeholder	Specification
B1	Practice partner	Energy infrastructure
B2	Practice partner	Energy infrastructure
B3	Researcher	Project Management
B4	Practice partner	Infrastructure
B5	Researcher	Project Management
B6	Researcher	Research Assistant
B7	Practice partner	Manufacturer
B8	Researcher	Research Assistant

).

3.3. Preparation and Implementation of the Interviews

After the agreement to participate in the interviews, a data protection declaration was sent that described the further use of the data and the guarantee that data will be treated anonymously to ensure that the interviewees could answer freely and honestly without any concerns. The declarations were returned signed or verbally acknowledged at the beginning of the interviews and repeated in the recording. Seven of the eight interviews were conducted online. The methodological approach involved semi-structured interviews designed to evaluate the project and review the methodological approach. At the beginning of the interviews, an introduction to the project and an explanation of the terms transdisciplinary and mixed methods was given. Then, stakeholders were asked whether they perceived the projects as contributing to real-world problems and as transdisciplinary and how they would evaluate the mixed methods approach and how successful the project and transfer have been (questions are listed in

Table 3. Guideline of the Interviews.

Category	Questions 1
Real-world problems and relevance	Is the problem addressed in the research projects a real-world problem?
	Does the research question contribute to the solution of a real-world problem?
Transdisciplinary cooperation	Were practice partner involved and can you name some?
	Were criteria of success defined in the beginning and were praxis partner involved in defining these criteria?
	Have the success criteria been met?
	How did you saw the prospects of success before starting the process?
	Could you see governance and decision-making structures and how functional were they?
	Can you tell us which of those following criteria were successful?
	Was there an additional benefit from the transdisciplinary approach?
Methodological evaluation	Are the methodologies adapted to the solution strategy?
	Which methods were used?
	Were the methods appropriate for the research question, form of practice involvement trough the practice partner and type of outcomes of the products?
	Was there a stringent implementation of the methods and of the mixed methods approach?
	Are the data significant?
	Have the data been meaningfully related to each other?
Transfer	Has the research goal been achieved/Is the research questions answered?
	Are the results generalizable and transferable?
	Are the results published?
	Are the results being used?

¹ Questions are translated to English.

). If it was necessary for the interview, the accompanying material was shared to explain topics during the interview, e.g., the defined success criteria from the application phase were shown in order to identify the degree of fulfillment from the experts’ point of view.

3.4. Data Analysis

To analyze the interview results, a qualitative content analysis approach was used with the transcripts [71,72]. The data were deductively predetermined by the criteria catalog with the four categories and some subcategories. Interviews were coded three times. All interview data were categorized in the final coding scheme with some data accruing in more than one category. Subcategories were adjusted with the data. In the last coding process, measures for the exemplary research design were conducted and determined in an iteratively analysis process. The results have been paraphrased and summarized. Categories and subcategories are shown in Figure 5.

Additionally, an overview was created to classify the interview results from researchers and practice partners according to categories and the degree of fulfilment. To do this, keywords were used to assign the previously categorized and coded data to four different levels of fulfillment. The keywords are presented in

Table 4. Keywords and examples for the assignment of data towards different levels of fulfillment.

Level of Fulfillment	Keywords 1	Data Examples
Criteria are fulfilled	Yes; definitely; Yes, I think so; in any case	“…This is a real-world problem…” (B5, l. 17)
		“Yes, I think so, definitely...” (B6, l. 64)
Criteria are mostly fulfilled	Yes, but…; rather yes; only; not always; conditionally; too much, some missing aspects; not all	“...With data logging, I would say it’s been done conceptually, but […] it’s still missing field data...” (B5, ll. 195 f.)
		“So a structure, yes conditionally.” (B6, l. 234)
Criteria are partially fulfilled	Could have been better; rather not; tend to say no; limited; difficult; incomplete; many missing aspects	“...that the practice partners could have been even better involved in any case...” (B8, l. 90)
		“...I would tend to say no actually now or I don’t know about that.” (B6, ll. 130 f.)
Criteria are not fulfilled	No; not; never	“From my point of view, we have not directly involved anybody…” (B3, l. 86)
		“..., but maybe not meaningful if you want to completely generalize that now [...] to Germany [...] or to a whole city” (B2, ll. 211–215)

¹ Translated to English.

. Afterwards, the content analysis recommendations for action for further projects were deduced in the discussion.

4. Results

The results show how the practice partners and researchers evaluated the projects within the individual categories. An overview is shown in

Table 5. Quantitative summary of the results.

Category	Researchers	Practice Partners
1. Real World problems and relevance	Fulfilled	Fulfilled
2. Transdisciplinary cooperation	Mostly fulfilled	Fulfilled
2.1 Necessity of transdisciplinarity	Fulfilled	Fulfilled
2.2 Transdisciplinarity of project conception	Partially fulfilled	Mostly fulfilled
2.4 Definition and fulfillment of success criteria	Partially fulfilled	Mostly fulfilled
2.5 Prospects of project success before start	Fulfilled	Fulfilled
2.6 Management/decision making structure	Partially fulfilled	Partially fulfilled
3. Methodological evaluation	Mostly fulfilled	Fulfilled
3.1 Implementation of methods	Mostly fulfilled	Fulfilled
3.2 Significance of data	Not fulfilled	Partially fulfilled
3.3 Interpretation of data	Partially fulfilled	Fulfilled
4. Transfer	Partially fulfilled	Mostly fulfilled
4.1 Achievement of the research goal	Partially fulfilled	Mostly fulfilled
4.2 Generalizability and transferability	Partially fulfilled	Partially fulfilled
4.3 Usage of research results	Fulfilled	Fulfilled

. Most of the answers are for the BaaS-LEV project as SciSusMob had just started at the time of these interviews. Sample answers are shown in Appendix A.

4.1. Real-World Problems and Relevance

The research object of charging infrastructure for LEVs is a real-world problem because the research results affect many people. This can be seen through the outcome of the projects that focus on researching PSS solutions for a large-scale city such as Bochum with around 360,000 citizens. Accordingly, the real-world problems consist in everyone’s need for mobility and the necessity to change individual mobility to avoid negative impacts such as greenhouse gas emissions, noise emissions and land use conflicts. In addition, people are directly affected by the changes in the mobility sector even if they do not clearly care about the impacts of future mobility. Solutions such as BSS are changing the cityscape and people must accept those products because they are the ones who use them. Furthermore, results can be used for the innovation of new products which affect the companies and the people’s consumption.

4.2. Transdisciplinary Cooperation

The collaboration between practice partners and researchers is necessary for the reflection of scientific results in societal practice. However, it is very challenging to connect partners and make sure everyone knows each other, especially because stakeholders do not always come from the city where the implementation is planned. In BaaS-LEV and SciSusMob, companies are involved from different federal states such as Berlin or North-Rhine-Westphalia. Digital workshop formats help to connect the stakeholders.

4.2.1. Added Value of the Transdisciplinary Approach

The added value in the transdisciplinary approach arises largely from the following aspects:

• The transdisciplinary approach is necessary for solving real-world problems which are defined as problems that affect many stakeholders and therefore cannot merely be solved within the scientific community. There is a broad context that could be covered by the various partners and the involvement of many stakeholders.
• Sustainability problems are not just about the economy, environment or social acceptance, but they are characterized by the fact that all dimensions must be considered. This in turn leads to the involvement of many stakeholders.
• The knowledge gained through this approach is greater since different parameters could be investigated, and different methodologies and research objects could be used to analyze the reality. The analysis of different vehicle types is helpful and necessary to show the complexity of the results.

4.2.2. Involvement of Practice Partners and Joint Definition of Success Criteria

Research projects can be divided into three phases: The application phase where success criteria are defined, the implementation phase where research is conducted and the final phase in which research results are collected and processed. The involvement of the practice partners in all stages of the project is important, but difficult in practice. To have a successful transdisciplinary process, the following points must be considered:

• In the evaluated projects, the practice partners specified the goals together with the researchers in advance. The integration took place through the definition of their own products and services. The success criteria are mainly important for the researchers and could not be named by all practice partners. Therefore, a transparent communication could have helped to ensure that all involved stakeholders can name the goals.
• During implementation, stakeholders must be involved at any time. Involvement can be accomplished by carrying out the work packages and buildup of the energy supply and sharing concepts.
• To integrate the stakeholders into the research process and the results, workshop formats are very helpful.

4.2.3. Fulfillment of Success Criteria

The success criteria defined in the beginning of the project have been met in part. The practice partners were not able to name the success criteria independently, which is why they gave an assessment of the degree of fulfillment of the criteria based on a visual list.

• Energy supply concepts of LEV sharing have been recorded, analyzed and evaluated. Due to the dynamic market, there are still further research topics to examine. For example, the use of LEVs is not yet as widespread in Germany as in other countries, which limits the research due to the missing infrastructure.
• The criteria for social acceptance of sustainability for LEVs, Sharing and Energy-as-a-Service have been identified and individual components have been evaluated. Those surveys were conducted in Düsseldorf and Bochum. As a result, services could be developed from this, but there is still some potential to sharpen the results.
• The modular off-grid SCS has been evaluated with an LCA. A prototype is at the university campus in Bochum and has been tested. Tests are ongoing. There are only a few people using the station, so there are some valid assumptions missing which should be determined in the real-world laboratory to gain more detailed results. Social acceptance for those kinds of charging stations is still being tested and surveys are ongoing.
• This also applies to the surveys concerning the social acceptance of the BSS, which are still ongoing. The results of the LCA of these stations are not known by the practice partners. An assessment of the station has been limited due to the COVID-19 pandemic. Therefore, it was not possible to test it in a real-world laboratory. There are initial valid data which are not yet broadly meaningful.
• The business, technical and user requirements for business models for a sustainable energy supply for LEV sharing have been collected through expert interviews and user surveys. The results are not yet fully known as the survey on user acceptance is still pending.
• A network of relevant stakeholders has been created and outreach has been conducted. The main reference is to a trade fair as well as a workshop where the practice partners have been able to become acquainted with other project participants. Publications have been produced as well. The public relations work is still expandable and will be conducted in the ongoing SciSusMob project.

4.2.4. Prospects for Success and Cooperation

In the beginning of the project, prospects of success were assessed as good. One of the reasons for this was previous cooperation. Another reason was the opportunity for the practice partners to develop products and explore their business models. A benchmark for the successful cooperation and the assessment of prospects of success is seen in the fact that further transdisciplinary projects are ongoing.

4.2.5. Management, Decision-Making Structure and Communication

There are differences in the interviewees’ perceptions of management and decision-making structures. On the one hand, such structures were perceived and on the other hand, communication was flexible via short communication channels if it was needed. The management was exclusively in the hands of the researchers. Due to the university’s administrative processes, decisions were delayed or difficult to coordinate. For some, this approach worked well, while for others, this structure was not very functional. Tasks and structures were not always clearly distributed, so the progress was not always sufficiently communicated. It is indicated that regular meetings or newsletters with a project update would help to provide better communication and to clear the insufficient process.

4.3. Methodological Evaluation

In the projects, qualitative and quantitative methods were used. Qualitative methods which were used are expert interviews and workshops. Quantitative methods were surveys, LCA and prototyping for the real-world laboratory. Not all methods applied have always been known to all interviewees. Methods were used to analyze ecological, economic and social aspects. It is indicated that there were too many methods.

The methods applied to analyze the ecological impact of the energy supply concepts are mentioned as suitable for the solution strategy. In the field of social and economic research, there is potential for optimization, e.g., with an observation of user behavior for the battery swapping station and research on the financial implementation of business models.

4.3.1. Realization of the Methods

The implementation of the methods themselves was stringent. The mixed methods approach was not stringently applied. The methods were run parallel and mixed in between. The methods were the right ones, but results based on the relations between them are still missing. The statistical implementation of some methods is described as not valid. Sometimes, depth in implementation was missing.

4.3.2. Significance of the Data

The data are described as being of limited value. Their additional value consists in the fact that they can be used for further research. There is a lack of representativeness in the data as, for example, surveys were only conducted in specific cities. A distinction is made between the significance of the quantitative and the qualitative data. For the quantitative side, data which were collected for the ecological impact of the energy supply concepts and the sharing vehicles are seen as meaningful for the specific cases. The data for the prototypes are valid, but the cases are described as too small for general implementation, as there were not enough partners involved in the transdisciplinary approach and not enough users testing the devices. Statistics about the usage of the battery swapping system, for example, were not made. A transfer to real environments is not possible with these data, and more research needs to be performed in this field and more data must be collected. The qualitative data are described as not meaningful. There are too few data. Among other things, this is related to the COVID-19 pandemic, because of which data in terms of usage and user acceptance, in particular, could not be collected.

4.3.3. Interpretation of the Data

Practice partners state that the interpretation of the data is carried out according to the mixed methods approach or that this procedure is reasonable for the data. Interpretation within individual methods is stringent for them as well as for the researchers. Researchers say that the interpretation is not yet completed as not all data are collected. They say that the data have not yet been related or linked to each other.

4.4. Transfer

The transfer has taken place in different ways. Workshop formats and trade fairs are mentioned as physical places where stakeholders had an exchange. Results are not known to all stakeholders. Publications serve to transfer the knowledge, but they are not yet completed. Transferability and usage of the results are partly achieved.

4.4.1. Achievement of the Research Objective

The research objective has been partially achieved. Since the BaaS-LEV project is a preliminary study, no final results are known yet, but for this part of the project the goals have been achieved. Sometimes, the output from the research objective is not entirely clear. As the goal is so extensive, there may be a need for further research with energy supply concepts, but this remains uncertain.

4.4.2. Generalizability and Transferability of Results

The results are not seen as generalizable, which is mainly because of aspects regarding the socio-cultural background of potential users and users’ acceptance. The results are conceptually helpful but not yet transferable to practice and further research needs to take place. The sample size and the case have been too small to make generally valid statements from them, so it is not possible to run a sustainable sharing business model based on these results. There is still a lack of usage of data and general measures for market research as usage depends on local conditions. Different locations and manufacturers were considered, but the scope was not fully exhausted.

On the technical side, data for the individual devices are helpful to avoid technical errors and useful for the end products which are generalizable because they are based on physical parameters such as solar radiation. This is the same for the results of the LCA, which are easier to transfer to other cases.

4.4.3. Usage of Results

The results are used by the researchers to conduct further research with them and to make teaching at the University more practical. The results are applied in the follow-up project. The network is expanded through the results as new stakeholders appear.

The practice partners can use the results to optimize their products and services as well as the knowledge that these products are not suitable for them.

5. Discussion and Measurements

The evaluation results highlight the importance of evaluating transdisciplinary projects to manage the challenges which arise from transdisciplinary approaches. As sustainability projects are long-term oriented, it is important to evaluate their contribution and large-scale effects.

For the analyzed projects, the research results show the difficulty of developing and researching a sustainable and economically sufficient solution for LEV sharing, especially with a focus on novel energy supply concepts. The evaluation outlines the limitations of mixed methods in term of significancy, quality assurance and transferability of research results. However, those limitations might be overcome by a more structured approach using a strict guideline of methodological implementation.

In the following discussion, the potential and limitations of the transdisciplinary mixed methods approach for research in sustainable mobility are reflected based on the evaluation results of this paper. Afterwards, we will discuss possible measurements to overcome the identified limitations.

5.1. Potential and Limitations of the Transdiscplinary Mixed Methods Approach

The results highlight the importance of the transdisciplinary approach for sustainable transformation and change of the transport sector due to the connection between mobility and usage of public space [73,74,75]. The projects are described as contributing to real-world problems and as significant for the development of future mobility. The results emphasize the benefit from a transdisciplinary approach using case studies to solve those real-word problems, which helps bridge the gap between society and science [76,77]. It is necessary to adapt the results to reality and find solutions that meet the requirements of citizens, practice partners and other stakeholders [78]. An informative cooperation supports this process. This is consistent with Gebhardt et al. [24], who found that documentation and easy information are important for developing knowledge of individual actors and finding the most effective workaround. Practice partners must be actively involved in the methodological approach to enable scalability and useful interpretation of data.

The mixed methods approach is helpful to address all sustainability aspects but brings in difficulties in implementation. Since data are sequential, the importance of data availability has become increasingly essential. The mixed methods approach can strengthen the availability of useful data and interpretation, but due to unforeseen circumstances such as the pandemic, data collection can be inhibited. In addition, a lack of knowledge can result in reduced use of data. In the projects, few data about usage and user acceptance could be collected and therefore scalability is not achieved. This raises further questions about the actual implementation in the real-world, which should be answered with further research within the framework of SciSusMob. Further research can ensure that missing data are collected through implementation of a real-world laboratory, which is one of the methods that had been envisaged but could not be implemented.

Real-world laboratories are helpful for analyzing the process in the environment in which they must persist because data are collected in real settings [79]. Then, scientific results can be translated into social practice and results that persist beyond scientific understanding [80]. However, unforeseen circumstances may inhibit implementation.

It is clear that transdisciplinary research exists through real-world settings, and methods are not adaptable in any way that replaces real conditions of usage, especially with focus on long-term effects. However, the consideration of long-term effects is important for the integration of users in battery swapping or usage of LEVs in general. Due to the circumstances of the projects and the close cooperation with the practice partners, another important aspect—namely, the cooperation with citizens—has been neglected. Since parts of the research have not been participative, it may be that environmental impacts are difficult to reduce as potential users were not involved. With user involvement, more solutions may have been identified to avoid typical user behavior that are difficult to change [81]. In addition, the involvement of citizens is very conducive to accepting and changing user behavior [24].

5.2. Measurements

In order to exploit the potential of the transdisciplinary and mixed methods approach, researchers need to evaluate their projects regularly and question the suitability of methods and results. To avoid any conflicts of interest in the evaluation, it is important that researchers are not involved in the project and are neutral. The best option is a completely external evaluation. The evaluator should first receive an initial overview of the project itself according to the work packages and envisaged project results as well as an overview of the methodologies used. Afterwards, the evaluator should define a suitable methodology to conduct the assessment. Depending on the project size and duration, recurring questionnaires can be suitable as well as expert interviews. These should be recurring to allow comparability during the project and assessment of the results of the evaluation. Interviews were used in this paper as they are more appropriate for smaller projects. The interviews and surveys can be designed with the existing guidelines [23,52,53,54,55] as well as the following measurements. Through the background knowledge about the work packages and project goals, specific questions can be identified and added. Furthermore, a mixed methods design can also be used during evaluation to obtain more neutral data, e.g., by conducting observations. The simultaneous realization of the project and the evaluation results in the possibility of using results by back reflection with the interviewed or surveyed persons. A back reflection can be performed during a small internal workshop. Thereafter, the results can be used instantly to strengthen the outcome of the projects. The visualization of results and information of all stakeholders is necessary as well as the definition of new criteria and KPIs to further improve and assess the project based on the results.

• For the project application itself, there are different points which must be considered depending on the time of the project (1) application phase, (2) duration and (3) conclusion and follow-up [53]. For the application phase, it is important to define the real-world problem based on information about concrete political needs and goals which are focusing on decarbonization and electrification of the transport sector. This helps to justify the problem as necessary for a sustainable transformation [52]. Since everyone has a need for mobility, the design of mobility projects requires collaboration with citizens in order not to bring technical solutions into the market which are not accepted and used but are responsible for a further waste of resources. Therefore, the reflection of the topics with citizens is important to ensure the significance and relevance of the research project for the stakeholders [80]. The relevant stakeholders for mobility projects are energy suppliers, public utilities, vehicle manufacturers, mobility service providers, software developers, users, citizens and transport companies. Practice partners and the transdisciplinary team should be defined and involved in the research process as early as possible [55]. This can ensure that, during the project, field data can be collected by guaranteeing that public space can be used and users are available. It is equally important to consider communication, organization and evaluation in the application phase and to set regular exchange and evaluation dates [76]. The joint definition of success criteria can help to avoid conflicts of objectives and to identify goals for each of the partners. These criteria help to obtain a rule-based procedure and evaluate the success later [55]. For the field of mobility, there is a large portfolio of methods which should already be adapted to the real-world problem and transferred to the mixed method design. The methodologies can be quantitative, such as life cycle assessment, vehicle data collection, prototype development, scenario analysis and user surveys. Especially for the social research and the involvement of citizens’ perceptions, qualitative data from observations, expert interviews and workshops can be used. It is important to translate these into a detailed work plan with milestones and handover criteria between the methods to ensure validity. The openness of the methods should be enabled by adaptive project scenarios, and the methods should be condensed to the extent that they correspond to the project scope in terms of time. This means that it must be ensured that methods are applied according to the information flow between practice partners and researchers. However, the main implementation of the mixed methods should take place independently within the transdisciplinary research. Of course, the results from work packages are interdependent, but the process should be defined by the structure, so that practice partners are involved step by step.
• During the project, it is important to ensure the collection of field data. The collection depends strongly on the involvement of users and availability of vehicles and public environment. Due to the early involvement in the application phase, contact persons should be available here. Dynamic planning and fixed decision-making structures and collaborative leadership are key [78]. If it is not possible to obtain field data, the dynamic adaptation of methods can be useful to ensure the collection of data to answer the research questions. Continuous evaluation of results can help to identify such areas of adaptation [53,54,55]. Praxis partners should be actively involved in all project stages, but they do not need to be involved as much in the mixed methods framework, which helps to reduce the complexity of the process.
• If during the research period vehicles are not used, it might be helpful to conduct a survey to determine the reasons and respond to the dynamic project and market events. This requires continuous transparency and cooperation between the stakeholders and observance within the rules from the application phase. During implementation, it should be ensured that the results are relevant beyond the use case to reduce environmental impacts and strengthen societal acceptance. Transdisciplinary collaboration should be continuously kept in mind [55]. This is especially relevant when there is no financial benefit for stakeholders because interim results show that the mobility project is not economically viable, but the generation of results depends on the stakeholder’s participation.
• Researchers are responsible for the implementation of the mixed methods framework. They should specify when they need which information to link the results. Delays in data collection should be communicated. The transdisciplinary approach can be used as a value to adapt to dynamics.
• At the end of the project, it is very important to transfer the information and evaluate the results and interpretations together with the stakeholders. The results are essential for further research questions as the transformation of the mobility sector is a long-term process. The data should be made available, and results compared with other projects and use cases to ensure the scalability beyond the special case of one city. The involvement of relevant users is once again important in this phase because they are the ones who will later use the applications for their daily life.

6. Conclusions

There are great challenges going ahead with the decarbonization of the transport sector. Projects addressing these challenges need both a high level of methodological diversity and a transdisciplinary approach. The collaboration with partners requires a structured project management and flexible organization and coordination as well as communication aspects, while the different methods require stringent implementation. At the same time, it is necessary to always react flexibly to external circumstances.

The aim of this paper was to present the evaluation of the BaaS-LEV and SciSusMob projects to create a guideline for the use of mixed methods in transdisciplinary research. Both projects have a real-world-concerning question that deals with the linkage of LEVs and regenerative energy supply with SCS and BSS. The BaaS-LEV project had already been completed at the time of the evaluation, which was very helpful for obtaining information about the challenges and limitations of the research process. These can be used in the follow-up SciSusMob project. From both projects, it could be deduced how difficult transdisciplinary mobility projects within a mixed methods framework are and how the implementation should be planned. A particular challenge in this context is the dynamic application of methods and the dynamic project process, which is difficult to foresee in the application phase. The stringent use of methods is important but not always easy to implement if circumstances change. However, it is not important that the practice partners understand what is being done methodically as they need less methodological rigor for the implementation of products in practice. This allows an easier integration of the mixed methods framework as it is possible to conduct the methodological part separately. This means that practice partners are only involved if necessary and during data collection but not within the research framework. In the end, it became clear that the mixed methods approach in the projects is necessary to address many aspects of sustainability and to evaluate the ecological and economical as well as the societal aspects.

A special challenge additionally arises from the fact that many stakeholders are involved which can trigger a further delay. Transparent communication and smart stakeholder management can reduce negative impacts and the risk of delay. This paper has shown the possibilities that exist in transdisciplinary mobility projects and thus demonstrates the importance of evaluation with already existing evaluation guidelines. It concluded that it is important to deal with real-world problems in real-world environments to ensure the maximum possible benefit.

This research will be continued in the aforementioned follow-up-project “SCiSusMob”. The SciSusMob project will continue the research results of the Baas-LEV project with a focus on applicability, scaling, quality assurance and transferability. A main component will be the implementation of a real-world-laboratory in collaboration with a local sharing provider, which also allows further interdisciplinary cooperation and involvement of stakeholders. The SCiSusMob project will include a detailed analysis of the applicability of the previous research for widespread implementation in micro-mobility services beyond Bochum within a three-year-period. Therefore, the authors will further apply mixed methods. To build up the concrete business models, the authors plan to create further research business models using the Business Model Canvas and interviews with micro-mobility service providers.

This paper has not yet resulted in a concrete implementation of the presented guideline. Therefore, we recommend implementation and evaluation in further research. The SciSusMob project will be evaluated and improved using this guideline. To further involve citizens, user surveys and observations will be conducted as well as a workshop-based reflection on the evaluation results with users in the research projects.