Jointly experimenting for transformation?: Shaping real-world laboratories by comparing them

Real-world laboratories (RwLs, German Reallabore) belong to a family of increasingly popular experimental and transdisciplinary research approaches at the science-society interface. As these approaches in general, and RwLs in particular, often lack clear definitions of key characteristics and their operationalization, we make two contributions in this article. First, we identify five core characteristics of RwLs: contribution to transformation, experimental methods, transdisciplinary research mode, scalability and transferability of results, as well as scientific and societal learning and reflexivity. Second, we compare RwLs to similar research approaches according to the five characteristics. In this way, we provide an orientation on experimental and transdisciplin ary research for societal transformations, and reveal the contributions of this type of research in supporting societal change. Our findings enable learning across the different approaches and highlight their complementarities, with a particular focus on RwLs.

olution-oriented (Miller et al. 2014) and pragmatist approaches to research (Popa et al. 2015) have given rise to a new generation of laboratory and experimental settings of research with and for society.In fields such as urban development (Evans et al. 2016), environmental politics, and climate mitigation (Bulkeley and Castán Broto 2013), these approaches attempt to accelerate transformations towards more sustainable societies and to transfer existing knowledge into action (Fazey et al. 2018).They combine the production of evidence on solutions to societal challenges (Cani g lia et al. 2017) with the mission of supporting transformation (Voy tenko et al. 2016).
New forms of real-world experimentation, such as(sustainability) living labs (SLLs) (e. g., Liedtke et al. 2015), urban transi tion labs (UTLs) (e.g., Nevens et al. 2013), transformation labs (T-Labs) (e.g., Olsson 2016), and real-world laboratories (RwLs) (e.g., Wagner and Grunwald 2015), attempt to merge the strengths of laboratory settings with the advantages of conducting research in the real world (Caniglia et al. 2017).Yet, setting up a laboratory within society requires the adaptation of methods and procedures to specific contexts, actors, and issues.Knowledge produced in this way is highly contextual, making it challenging to understand how results obtained and lessons learnt might be transferred or generalized, if at all (Krohn et al. 2017).
In the German context, RwLs (German: Reallabore) have rapidly emerged as a leading approach in transformative research and sustainability governance (e. g., Schneidewind 2014, Wagner andGrunwald 2015).Often motivated by rationales at the intersection of political and scientific agendas, experiments in RwLs are designed to create knowledge related to potential solutions for sustainability challenges (MWK 2013,WBGU 2016).Overarching conceptualizations of the RwL approach are under development (e.g., Wanner et al. forthcoming).However, we still lack clarity about the main features and added value they produce, as well as about how RwLs compare to other real-world experimentation approaches in sustainability science (Schäpke et al. 2015).
Building on previous exploratory work (Schäpke et al. 2017a) as well as on the experience of some of the authors as accompa-

RESEARCH | REVIEW
Niko Schäpke et al. nying researchers in RwLs, we first highlight the main characteristics of RwLs in this article.Second, we briefly describe the related SLL, UTL, and T-Lab approaches mentioned above.Third, we compare RwLs with these approaches according to the characteristics.This results fourth in a synergistic comparison putting RwLs center stage.Finally, we discuss opportunities for the further development of RwLs.

Methods
This article is based on accompanying research in 14 RwLs (BaWü Labs) of two funding lines (2015 to 2017 and 2016 to 2018) in the state of Baden-Württemberg, Germany (Schäpke et al. 2015).This research included: 1. a literature review to examine the state of the art of the discourse revolving around RwLs (Schäpke et al. 2017a); 2. an initial workshop of seven BaWü Labs in 2015 as well as a survey and supporting interviews in 2017, to exam ine characteristics and success factors of RwLs and to ground cri teria from the literature in empirical experiences; 3. discussions with researchers from BaWü Labs and sustainability scholars during two symposia and a conference 1 , for example, identifying challenges when manifesting characteristics (Schäpke et al. 2017b, Wagner et al. 2016), and 4. a review of conceptual articles to capture core information on RwL-related approaches.
RwLs have developed dynamically through increasing funding opportunities, research projects and publications, while remaining conceptually and methodologically vague.This lack of clarity requires further conceptual and methodological development.To this end we decided to place RwLs center stage in our comparison, that is, we used the identified characteristics of RwLs as criteria of differentiation.Taking a different starting point might have revealed different characteristics.Thus, learnings primarily apply to the RwL approach.However, given that experimental approaches at the science-society interface pursue similar aims, results can to some degree enable us to characterize experimental research for sustainability more broadly.
For the comparison, we identified related approaches that focus on experimentation in transdisciplinary settings, with a rela -tion to sustainability and that are sufficiently described.We then selected those that revealed a certain difference in fulfilling the characteristics of RwLs and that cover a wide range of topics.

Core Characteristics of Real-World Laboratories
From the literature, we distilled five main characteristics of RwLs 2 : contribution to transformation; experiments as core research method; transdisciplinarity as core research mode; long-term orientation, scalability, and transferability of results; learning and reflexivity (table 1, the literature base for the derived characteristics is far more comprehensive, for an extended list please see the online supplement 3 ).Below we briefly describe the characteristics and outline ways for their implementation in RwL research and related challenges.Furthermore, where possible, we contextualize RwL characteristics in related discourses, mainly within the broader sustainability research debate.

Contribution to Transformation
Research aiming to contribute to sustainability transformation can be differentiated into transformation and transformative research (WBGU 2011).The first is largely descriptive and analyzes transformation dynamics and processes of change.The second fosters transformation by developing and applying solutions to sustainability challenges and generating actionable knowledge.This knowledge "provides instructions on strategies that can solve (or mitigate) certain problems (or its effects)" (Caniglia et al. 2017, p. 42).As such, transformative research shows parallels with the tradition of action research (e. g., Reason and Bradbury 2001).

Experiments as Core Research Method
Experiments are scientific practices that rely on an intervention and aim at producing empirical evidence (Caniglia et al. 2017 RwLs involve experiments designed to generate evidence related to action fostering sustainability transformations.Experimenting in real-world settings raises methodological questions around the participation of stakeholders, as well as ethical questions on the responsibility and legitimacy of interventions.A major challenge regarding research quality concerns the generation of generic and transferable insights from experiments in specific contexts, with many factors that are difficult to control (cp.Gross et al. 2005, chapter 1).

Transdisciplinarity as Core Research Mode
Transdisciplinary research concerns tackling real-world problems in collaborations between researchers from different disciplines and societal actors (Lang et al. 2012).Knowledge from various sourc es (scientific disciplines and nonscientific sources) is generat ed, differentiated, and integrated to foster socially robust knowledge.Transdisciplinary research therefore goes beyond multiand in terdisciplinarity that combine knowledge from different sci entific disciplines.Ideal-type transdisciplinary processes differentiate three phases of collaboration: co-design, co-production, and re-integration (Bergmann et al. 2012).The intensity of involvement of societal actors proceeds from information transfer through consultation, cooperation, collaboration, to empowerment.Phases > and intensities can be combined into a functional-dynamic model of collaboration (Stauffacher et al. 2012).Thus, depending on the process phases and respective aims, different intensities of collaboration can be dynamically combined to best serve aims (see Menny et al. 2018, in this issue).
RwLs aim to realize transdisciplinary research in order to differ entiate and integrate scientific and societal knowledge, related to a real-world problem.The intensity of collaboration may differ -in general, a meaningful involvement of societal actors is emphasized.Although RwLs do include many aspects considered typical for transdisciplinary research, they have a particular focus: real-world experiments on solutions.Thus, labs can build on previous transdisciplinary processes of, for instance, co-designing a shared problem understanding and related vision (Jahn and Keil 2016, see also Rogga et al. 2018, in this issue), or they can include these steps (Wanner et al. forthcoming).Undertaking collaborative, real-world experimentation, however, often raises particular challenges regarding ownership, transparency, knowledge integration, and conflict management.

Long-Term Orientation, Scalability and Transferability of Results
Transformations here are understood as long-term and large-scale processes of societal change.Thus, respective research should allow for a long-term perspective (e.g., 25 years, Loorbach 2007), and evidence generated on solutions should provide insights with regard to their transfer or upscaling (e.g., Luederitz et al. 2017).Transferability concerns transferring insights to other contexts, via generalization of insights as well as gaining knowledge on contextual factors.This can mean transferring insights from one topical area to related ones, such as, for instance, in the field of campus sustainability, to move from energy efficiency to waste reduction.Scaling concerns increasing the reach of solutions in the original context and depends on insights on scalable features of solutions.An example could be to increase the geographical reach by scaling from households, to districts, to cities, and beyond.Replications of (simplistic) solutions, such as innovative products, often do not sufficiently account for the complexity of sustainability problems.Rather integrated solu tion strategies including processes, contexts, and outputs to solve a problem, should be developed and replicated (Forrest andWiek 2015, Heiskanen et al. 2018, in this issue).nRwLs aim at contributing to transformations.Large-scale impacts do not necessarily depend on the long-term existence of RwLs, but rather on the uptake of solutions.Thus, the solution options created should have a long-term horizon, potentially going beyond the existence of the lab.Transfer and scaling depend on generalized insights and anticipation of negative side-effects.Feasibility studies, comparisons of experiments within one lab or between different labs, as well as involvement of key actors from different scales, can contribute in this respect.However, this requires adequate project architectures and longer-term funding, allowing for continuous experimentation and longitudinal evaluation.Transferability and scalability are particularly challenging and potentially limited due to the situatedness of RwLs, which constitutes a major challenge in doing RwL research.

Learning and Reflexivity
The discourses on sustainability transformation research refer to learning in various forms.Barth and Michelsen (2013) propose three levels to structure the understanding of learning: firstly, individual competency development; secondly, social learning processes amongst the collaborating actors and beyond, for example, on sustainability problems or their solutions; thirdly, learning with regard to transdisciplinary collaboration (how to collaborate).This also entails reflecting on the influence that actors' values, norms and epistemologies have on the collaboration (reflexivity).Thereby, reflexivity supports the transdisciplinary collaboration and can be understood as a social learning process itself (Popa et al. 2015).
Also, in RwLs learning and reflexivity are particularly relevant processes.Contributions on all mentioned levels may occur (Singer-Brodowski et al. 2018, in this issue): individual competency development can be facilitated by offering learning space (e.g., integrated seminars).RwLs may enable experimental learning cycles, and focus on the interplay of knowledge exchange, action, and reflection.They may enable social learning in facilitating the discourse between participants and offering a protected space to build trust, allowing for mistakes and iterations, and mediating conflicts.While negotiating different perspectives, participants can join a process of collective meaning making which nurtures ownership of, and participation in, the lab.Transdisciplinary collaborations bring togeth er scientific and nonscientific actors who seek to intervene in real-world settings, following their different agendas.Reflexivity is therefore crucial and includes confronting, interrelating, and integrating different epistemic cultures, values, or goals.In principle, learning contributes to realizing the four other RwL characteristics and may be considered a cross-cutting characteristic.

Related Approaches in a Nutshell
In this section, we provide a brief overview of three approaches that we compare to RwLs (please see online supplement for an extended list of relevant literature 3 ).

(Sustainable) Living Labs
A living lab (LL) is an experimental research setting embedded in a real-world context.In LLs, researchers, users, and other stakeholders along the value chain co-create innovative products and services (Liedtke et al. 2015).LLs are used to explore social practices and consumption patterns.The approach builds on participatory innovation studies, individual and organizational learning, and open innovation: "(A) Living Lab is an open innovation environment in real-life settings in which user-driven innovation is the cocreation process for new services, products and societal infrastructures.Living Labs encompass societal and technological dimensions simultaneously in a business-citizens-government-academia partnership."(Bergvall-Kåreborn and Ståhlbröst 2009, p.357) LLs combine different methods of user integration into the innovation process.Real-world elements (e. g., specific cultural and social settings) are configured as context within the LLs so that realistic usage patterns can be addressed.Current developments include elaborations on "sustainable living labs" (SLLs) (Liedtke et al. 2015, Baedeker et al. 2017), including the sustainability aspect in the analysis of products and services as well as of routinebased lifestyles.SLLs are the focus of the comparison in the following chapter.
(Urban) Transition Labs UTLs (Nevens et al. 2013) have been developed based on the transition management approach.Transition management aims to facilitate societal change towards sustainability (Loorbach 2007).A central mechanism in UTLs is the development of alternative ideas, practices, and structures in transition arenas (Loorbach et al. 2016), involving selected change agents.The participants are chosen in order to cover a variety of perspectives and roles (Loorbach 2007).Based on shared understanding of the transition challenge, guiding principles and future visions are developed, that are translated into transition experiments through a process of back-casting.Thus, experiments are embedded into transition pathways that relate the current situation to the envisioned future.In doing so, emerging transitions can be guided and accelerated.Consequently, UTLs are designed to influence everyday practices of participants, rather than to produce direct tangible outputs within the lab.Fields of application are diverse, including energy, health care, food, and city planning.

Transformation Labs
T-Labs create interactive spaces that allow for experimenting with potential solutions that take into account social, technological, economic, and ecological aspects (Olsson 2016).The concept builds on the social innovation lab approach (Westley and Laban 2015), and was developed as a response to the neglect of human-environmental relationships in existing approaches to transitions towards sustainability.T-Labs address the risk such approaches may run in supporting transformations without improving the overall capacity of a system to learn from and manage environmental feedback.Thus, T-Labs build on the notion that humans are dependent on ecosystems, and that humans and the ecosystems they are embedded in should be treated as an integrated whole.They are platforms for multi-stakeholder collaboration with an aim to generate innovations that can contribute to concrete, large-scale, systemic transformations to sustainability.Situations in which this approach has been applied include sustainable agriculture and food systems, low carbon energy transitions that serve the needs of the poor, and sustainable cities (Ely and Marin 2016).

Manifestation of Characteristics in Approaches Related to Real-World Laboratories
This section compares how the approaches described above realize RwL characteristics, based on key publications describing these approaches.This allows us to flesh out basic differences, while acknowledging variations in concrete contexts and situa-> RESEARCH | REVIEW tions.Each section ends with a brief reflection on challenges of RwLs in operationalizing the characteristic in question, and potential learning opportunities from related approaches.

Contribution to Transformation
SLLs aim at reducing unintended and unsustainable uses of products and services and related rebound effects.Thus, areas of high resources consumption (living, nutrition, mobility) are the focus of this approach (Liedtke et al. 2015).We understand SLLs as settings for transformative research when developing and testing innovations for solving societal challenges (such as climate change and resource scarcity).SLLs contribute actionable, applicationoriented knowledge with a focus on changing products and services and related consumer behavior and production processes.How far this contributes to more radical change needs to be further elaborated.
UTLs take persistent unsustainability and observed transition dynamics as a starting point.In doing so they explore why desired transitions to, for example, renewable energies develop slowly, and look at how to accelerate them.Labs touch upon diverse forms of innovation, including social, socio-technical, and economic.Experiments are embedded in a specific theory of change, that is, a broader, reflexive governance approach providing an understanding of how different methods and processes engaging particular actors facilitate change in specific contexts.UTLs follow a prescriptive, action-oriented logic introducing a transition frame to develop a new, shared discourse among participants who, in their daily lives, play an important role in a transition.The process is designed in a way that the participants develop and internalize a transition perspective that helps to guide their decisions.The process facilitation provides input and synthesizes output of discussions, based on the framing of the transition challenge.Thus, rather than prescribing a particular solution, the process develops an understanding of the problem and the wider process of (desired) societal transformation.
The aim of the T-Lab is to collaboratively explore specific sustainability problems and generate social-ecological innovations which can fundamentally change the system conditions that created the problems in the first place.Thus, it can initiate processes that lead to systemic transformations.Rooted in complex systems theory, transformative research is drawn upon through every phase of the lab process, and adapted towards the focused problem domain.For instance, the first step involves participants engag ing in sense-making to co-produce knowledge and a shared understanding of the complex systems in question and related uncertainty.This collaborative and facilitated process can help explore system dynamics and the implications to be considered when moving into the stage of designing innovative sustainability solutions.It also provides methods for monitoring and evaluation.
Related RwL challenges include balancing both, understanding and facilitating change, combining a descriptive-analytical and a prescriptive research focus.An underlying theory of change, however, is missing.For further development, RwLs can provide room for plurality and systematic exploration -explicitly relating RwLs to different theories of change, and related research practices (see Heiskanen et al. 2018, in this issue).

Experiments
SLLs apply experiments as part of co-design and co-production processes.Products and services are designed for the highest sustainability performance (Baedeker et al. 2017).Experiments include the re-combination of existing technologies and testing of newly developed prototypes and products and services systems, in order to produce generalizable and transferable knowledge and solutions.Thus, SLLs aim for full, external control of experiments and their contexts, at the same time involving mixed methods for user integration.Thus, methods from natural and social scienc es are combined to generate a combination of quantitative and qualitative data oriented towards a boundary object as a shared unit of analysis (a product and service system).Initially, behavioral routines of users are observed, using, for instance, diaries and observations as well as related resource flows are assessed.Secondly, scenarios and prototypes are co-created in design workshops.Finally, prototypes are tested and evaluated, combining sensoring, diaries, workshops, and actor analysis for diffusion (von Geibler et al. 2013, p. 28).Tested constructs (e.g., prototypes) are reflected on, and amended.Thus, experiments can reinforce one another (Van den Bosch 2010).Comparative studies building on numerous standardized experiments play an important role.In result, SLLs generate evidence-based actionable knowledge.
In UTLs, experiments are part of the wider transition agenda to explore socio-technical transition pathways.Often the experiments build upon existing ideas or actions of participants, but add a broader systemic orientation, learning goals, and a connection to other experiments.Transition experiments are defined as system-innovation projects that help to visualize and explore alternative futures, empower transformative agency, and create opportunities for social learning (Van den Bosch 2010).They take diverse forms according to the openness of the approach and the diversity of application contexts.A UTL may facilitate a portfolio of exper -RwLs do not exist in a vacuum, nor are they unique or completely new in what they pursue and in the ways they proceed.They belong to a family of increasingly popular transdisciplinary and experimental approaches to transformative research.
GAIA 27/S1(2018): 85 -96 iments to increase learning, and interventions are controlled in a participatory manner.Thus, experiments are decided upon in the transition arena, carried out by participants and reflected upon and modified again in transdisciplinary collaboration.
A T-Lab is a method to jointly experiment with prototype solutions that can create new social-ecological system configurations, as well as explore new relationships between people and the planet in real-world problem domains, and in real-time (i.e., participatory controlled settings).Before a gathering for a T-Lab is convened, the problem domain is investigated using in-depth interviews and system analyses.Then, decisions need to be made about whether a lab is well-suited to the problems identified.If a lab seems appropriate, multiple workshops are hosted to generate, test, evaluate, and refine prototypes.The testing method may differ depending on the problem and solution: for example, piloting governance models may entail different approaches than testing and refining algorithms that enhance the consideration of ecosystems in financial markets.The workshops tend to involve a blend of formal expertise and informal participatory inputs.
Learning possibilities include the generation of generalizable insights from experiments controlled by forms of participation limited in intensity, combined with scientifically rigorous mixed methods analysis (SLL).In addition, the orchestration of experiments as well as their integration in larger governance activities (UTL) and methodic embedding in socio-ecological systems (T-Lab) can increase contributions to societal change and evidence creation.

Transdisciplinarity
SLLs generate knowledge on sustainable products and services.Knowledge gains for society and business include application-oriented insights on user-behavior and related products and services systems.Evidence and generalizable knowledge is developed in a joint learning process.Core participating groups are users and business actors coming together with engineers, social and natural scientists, and designers in transdisciplinary collaboration.SLLs provide an innovation system that combines different forms and intensities of user engagement.The most intense forms of transdis ciplinary collaboration appear in the prototyping stage (consultation and collaboration).User observation and field-testing are limited in stakeholder involvement, orienting collaboration forms and intensities towards fulfilling goals.
UTLs are driven by a small transition team including researchers and societal actors, for example, local government officials.An UTL is a transdisciplinary research environment.It includes the iteration between interdisciplinary analysis and interpretation of transition challenges, and enriches this by social change agents: the team undertakes an initial analysis that is then further developed and validated through in-depth conversations with transformative actors in the specific area.During this process, potential participants are also scouted, as the conversations not only seek to gather information but also explore the actors' drive for societal transformation and ability to engage with new perspectives.The actual transition arena process, as well as the experiments themselves, are then typically a process of joint knowledge production and sense-making, focusing intense forms of collaboration aimed at empowering participants.As such, traditional research does not play a pivotal role.However, the processes themselves are considered as research experiments on what types of interventions might help to guide transitions, adding to the theoretical understanding of transition management.
T-Labs require supporting conveners willing to invest resources into the process, and who have proposed a complex sustainability challenge to focus on (Westley and Laban 2015).They also have a small team responsible for designing and facilitating the process, as well as building the collaborative relationships.During the preparatory systems analysis, the team identifies individuals, organizations, and networks deeply committed to changing the system dynamics that they themselves may represent.This serves to include a diverse range of researchers (from different academic disciplines) and societal actors (e. g., designers, policy-makers, so cial entrepreneurs, but also practitioners like farmers, fishermen, etc.) in order to develop a shared sense of ownership of the process.In early workshop phases, methods support participants' collective work to "see the system", highlighting important variables and dynamics.This aims at identifying opportunities and key transition points, and to consider cross-scale dynamics.Later workshops focus on innovating at a systemic level, not just inventing a single product or idea (Olsson 2016).
Related RwL challenges include ownership for processes and results, knowledge integration, transparency, and conflict management (e.g., Wagner et al. 2016, Schäpke et al. 2017b).RwLs may learn from process experiences in, for example, UTLs on creating ownership and managing conflicts, and knowledge integration in T-Labs (e.g., systems mapping).Overall, target group selection in accordance with transformative aims appears important: UTLs, T-Labs and SLLs often engage participants who hold crucial roles for systemic change, participate voluntarily, and are open towards radical change.

Long-Term Orientation, Scalability and Transferability
SLLs are often set up with a long-term horizon as they involve specialized infrastructures, such as buildings, allowing for longitudinal research.They often aim to develop products and services for large-scale introduction to markets.Thus, scalability is an important goal of SLLs.Scaling up processes can be facilitated by including relevant decision makers and actors with access to resources, such as enterprises and well-established research institutions.Upscaling is then a process of diffusing newly configured social practices in the form of user practices related to developed products and services.
UTLs are set up temporarily, but may become more permanent if developed structures get transferred into societal ownership.They are designed to produce narratives that guide and accelerate actions well beyond the boundaries of the UTLs.Transition teams lab participants follow different strategies to help diffuse three types of outputs.First, they transfer narratives and visions (strategic) through direct communication and media.Second, new strategies, coalitions, and goals (tactical) are scaled through negotiation and institutionalization. Third, they accelerate experiments and actions (operational) through processes of deepening, broadening, and upscaling (Van den Bosch 2010).Deepening refers to learning as much as possible from a given experiment, for example, by comparing similar experiments.Broadening relates to transferring innovative results to other contexts, for example, via networks, repeating experiments in different contexts or developing overarching narratives.Scaling up relates to "embedding a transition experiment in dominant ways of thinking (…), doing (…) and organizing (…), at the level of a societal system" (Van den Bosch 2010, p. 68).Scaling and transferring in UTLs relate more strongly to societal than to scientific impacts.
T-Labs are set up temporarily and seek to generate social-ecological innovations aimed at challenging and changing existing roles and routines, power dynamics, relations among groups and networks, resource flows, as well as meaning and values (and culture) across different contexts and scales.Scaling, in this context, involves paying attention to social-ecological linkages across scales, since evidence indicates that without this, innovations can shift problems to other scales, sectors, and future generations (Olsson et al. 2017).The T-Lab perspective on scaling innovation stands in contrast to other styles of labs, that rely heavily on similar approaches for scaling "out" technological innovation (i.e., scaling is equal to a large number of people adopting a single product).Thus, T-Labs help move beyond replication, recognizing the need to scale "up" by altering institutional structures and processes, and scale "deep" to shift deeply held cultural beliefs, values, and ways of being (Moore et al. 2015).
Related RwL challenges include project architectures and funding structures that support long-term horizons, scaling, and transfer, as well as mechanisms of transferring and scaling insights from context-dependent lab research (Schäpke et al. 2017a).RwLs may learn regarding the role of the physical structure of labs (SLL) for long-term impacts, the respective role of lab ideas and narratives diffusing in society (UTL), and research on long-term developments (longitudinal research).Learning on scaling and transfer includes differentiations of objects of scaling/transfer (e. g., products versus processes and solution strategies), strategies for scaling and transfer (T-Lab: scaling deep, scaling up; UTL: deepening, broaden ing), and generation of generalizable insights (patterns, structures, and rules of transformation), for example, building on SLL mixed methods.An increased impact warrants a clear understanding of what is to be scaled and transferred and by what type of process, for example, building on a clearer, overarching typology of the various scaling concepts used.

Learning and Reflexivity
SLLs focus on explorational learning of individuals, addressing social practices and beliefs as well as the interpretative patterns and norms associated with them.Participants learn to re-construct their behavior by jointly creating and implementing products and services (Liedtke et al. 2015).SLLs address personal, social, cognitive-methodological, and subject-related competencies in a combination of formal, nonformal, and informal learning processes (Bliesner et al. 2014).They provide a setting and methodology for experiments.Integrating stakeholders in experiments through action-research-based methods facilitates social learning as well as insight into users' everyday needs and social practices.The idea is to create tacit knowledge about doings, resembling user practic es.Successful implementation is fostered by involving a large network of stakeholders.Transdisciplinary collaboration is crucial for co-creating sustainable products and services.Based on open didactic exploration (Bliesner et al. 2014), the (disciplinary) backgrounds and value propositions of the actors are made explicit during the development process.With regard to reflexivity, "the integration of dynamic feedback processes that support reflexive learning and goal adjustment is important in order for Living Labs for sustainable development to fully leverage their potential impact" (von Geibler et al. 2014, p. 587).
UTLs include a process of co-construction and learning through which the involved actors internalize the transitions perspective and translate this into a specific action perspective.As such, the pro cess aims to empower actors by creating shared networks, perspectives, and agendas, but puts less emphasis on the individual competencies needed to operationalize ideas and actions beyond the process.UTLs host a process that in itself is a social learning intervention to facilitate new discourse, networks, and actions.This is partly based upon problem-based learning: a participatory process of exchanging worldviews and perspectives on complex processes to create shared understandings, as well as reframing problems to arrive at new solutions.UTLs as a transdisciplinary research environment forces involved researchers to accept and understand insights from different academic disciplines, and reflect upon their own positions, but also to bring them into the societal dialogue.As the UTL setting is designed to explore desirable futures, researchers are also pushed to go beyond description and observation towards engagement and facilitation.
The T-Lab approach to learning draws heavily on numerous ownership, knowledge integration, transparency, conflict management RwLs may learn from process experiences in, e. g., UTLs on how to create ownership and manage conflicts, and knowledge-integration procedures in T-Labs (e. g., in developing systems understandings) and SLLs (e. g., in using mixed methods).Besides, they might select participants in accordance with pursued aims, as done by the three related approaches (e. g., actors holding capacities for innovation and system change).
project architectures and funding structures; transferability and scalability from situated and context-dependent lab research RwLs may learn regarding the role of the physical presence of labs (SLL), the diffusion of ideas in society (UTL); and longitudinal research.Learning on upscaling and transfer includes the question what is scaled/transferred, how this is done (T-Lab: scaling deep, scaling up, UTL: deepening, broadening), and how generalizable insights emerge, e. g., building on SLL mixed methods.
Respective challenges are manifold.Due to the broad relevance of learning processes, potentials are multiple, regarding, e. g., competency development for practitioners, underlying learning theories (e. g., experiential, explorational, problem-based learning), labs as learning environments, "unlearning" for radical change and reflexive co-learning blogs across labs (T-Lab), feedback for reflexive learning (SLL), as well as the relevance of learning and reflexivity to the other characteristics.

TRANSFORMATION LAB (T-LAB)
both, with a clear focus on (a), goal: socio-ecological transformation; contribution to change by developing and testing prototypes and overcoming system lock-ins prototypes actionable knowledge, potentially directly contributing to change participatory control to influence turning points in socio-ecological systems various forms and intensities of collab oration depending on approached challenge generally driven by facilitators with col laboration of broader group of participants, engaging key actors holding capacities needed for innovations knowledge from both sources is integrated to reflexive systems understanding temporary set-up by facilitation team via scaling up (altering institutions and structures) and scaling deep (changing underlying values) depending on process design, generally collaboration and system reflection competencies get augmented whole-system exercises, design thinking, and unlearning dominant perspectives to give way to fundamental innovations depending on specific lab design, e.g., co-learning blogs across labs strongly modified and extended.

>
like Lewin (1947).But it also goes beyond this to understand that learning alone does not result in systems change, and engages participants to generate and test actions and innovations in a creative, collaborative process.Therefore, participants not only co-produce knowledge about complex system dynamics, but they also develop competencies for moving to application and action while maintaining a sense of "systems reflexivity".Thus, they learn about the emergent impacts as they intervene in a system, and furthermore develop collaboration skills.T-Labs build on theories from whole systems approaches, including social learning practices (neutral facilitation, group exercises in problem (re)framing, etc.), but also relies on design thinking.This includes processes of unlearning, the need to transform perspectives and to disrupt existing system patterns to develop truly innovative action.T-Labs explicitly build on transdisciplinary collaboration, that can be addressed via exercises, for example, systems mapping to consider cross-sectoral, cross-discipline, and cross-scale collaboration.The methods help to make different values and meanings amongst actors explicit, and to determine how different perspectives are complementary for understanding the problem and identifying solutions.Co-learning blogs across different T-Labs are aimed to help researchers be reflexive about their own assumptions on transformation (Ely and Marin 2016).
Respective challenges for RwLs are manifold (e. g., Wagner et al. 2016, Schäpke et al. 2017b).Due to the broad relevance of learning processes, there are multiple potentials for advancement, for example, regarding competency development, underlying learning theories (e.g., experimental, explorative, problem-based learning), labs as learning environments, as well as the interrelation of learning and reflexivity with the realization of other characteristics.Frequently approaches relate learning towards problem solving, learning-by-doing, tacit knowledge, and action orientation.In addition, learning should open up for radical change (as "unlearning" in T-Labs).Further elaborations are needed.

Discussion and Conclusion
From the comparison of RwLs with other approaches, we highlighted similarities and differences that can help improve the RwL approach, and its contribution to experimental and transformative sustainability research (table 2, p. 94/95).
RwLs exhibit a broad and not clearly defined research format.The other approaches are more consistent in aligning the differ -ent characteristics (e. g., types of experiments correspond to the understanding of the role of science in transformation).SLLs, for instance, aim for marketable, standardized products and services as socio-technical innovations, and related generalizable insights.Therefore, they perform controlled experiments with limited participation.UTLs go beyond socio-technical innovations and regard alternatives more broadly within a comprehensive conceptualiza tion of socio-technical change, aiming to enable social learning and empowerment processes as key drivers of transitions.Thus, they develop a portfolio of participatory experiments in societal settings, with high engagement of participants embedded into a larger governance approach -namely transition management -linking experiments and envisioned futures.T-Labs focus on systemic, social-ecological innovations to fundamentally alter configurations of socio-ecological systems and overcome deep causes of related problems.Thus, they build on extensive pre-studies and collective system-analysis to develop prototypes of systemic innovations and awareness amongst participants that they are part of a system (reflexivity).
RwL approaches are subject to major challenges.These include high expectations (e.g., delivering evidence-based knowledge and governing societal change), blurring of boundaries and responsibilities due to the engagement of researchers in societal actions, and a lack of analytical distance in the research process between the researchers and their objects of investigation.These specific challenges are related to a broader challenge, namely the production of outstanding research results, also connected to the unclear definition of the role of learning and reflexivity in RwLs.Governing change, in contrast, requires researchers to engage with politics and administrative representatives, ensuring institutional diffusion of innovations.This necessitates particular skills and process designs, different to those needed to assure research quality.In order to refine the RwL approach, it is important to develop and evaluate new formats that systematically and realistically combine the expectation to inform societal change with the need to produce outstanding research results.
As a new and still developing research format, RwLs have a lot to learn from the other approaches.This might mean learning about scientific rigor in the production of evidence (SLL).It could also mean learning how to increase societal impact through embedding labs and experiments into governance (UTL), or taking into account the systemic embeddedness of labs, including ecological aspects (T-Lab).Along the same lines, the comparison with other approaches may highlight avenues for the selection and com-As labs create some isolated space for experimentation, their significance for societal transformation might remain limited by their borders.It is thus important to complement lab approaches with broader policy commitments if we want to harness the transformative potential of these approaches in the real world.
bination of approaches in RwLs, depending on specific goals and objectives.This is particularly true when approaches are understood less as monolithic blocs, and more as a flexible combination of components (e. g., experimental methods, scaling strategies as well as collaboration forms and intensities applied).Also, the different scales (households, buildings, neighborhoods, industry sectors), topical foci (consumption and production or beyond, sociotechnical or socio-ecological innovations), as well as processes (small-scale niche innovation or transition governance), could be creatively combined in RwLs.Their open approach may provide a suitable framework for combining different components of experimental transformative research.Thus, RwLs might contribute to building bridges between different styles of transformative research, the design and implementation of experiments, as well as the evaluation of processes of learning and reflexivity.
RwLs do not exist in a vacuum, nor are they unique or completely new in what they pursue and in the ways they proceed.They are part of a larger development: the emergence of a family of transdisciplinary and experimental approaches to transformative research.This is explicit in the fact that all approaches analyzed relate to the core characteristics proposed for RwLs.Current trends in funding programs and research collaborations provide space to further explore the poten tial of experimental approaches in transformative research.Long-term evaluation and comparisons will show which approaches or combinations are most promising, in terms of real-world sustain ability transformation and acceleration in given contexts.This re quires transparent and structured comparisons between goals, assumptions, processes, and methodologies of different approaches in general and their specific applications, as well as a comparable analysis of context conditions.If this is the case, the diverse emerging approaches may complement each other, rather than compete for being the "best" approach.This complementarity should, however, not be confused with an "everything goes" attitude towards transformative research, undermining quality and rigor.Yet it acknowledges the complexity of sustainability challenges and solu tions, and the need for adequate, adapted approaches and under lying quality criteria (Fazey et al. 2018).Transformative research thereby takes a particular stance, focusing challenges and solutions.It needs to be complemented by other forms of research oriented towards understanding phenomena, for example, basic research.And, it needs to be guided by an attitude of humbleness and awareness of its own limitations.
Despite the dynamic development of lab approaches in the last decades, their diffusion is still limited.The contribution of such approaches to societal transformation largely depends on them being embedded into a broader policy commitment to systemic change, as well as on the development of mechanisms to accelerate learning.On their own, lab approaches risk to have limited real-world impact.Creating some isolated space for experimentation, the significance of labs for societal transformation might remain limited by their own borders.It is thus important to comple ment lab approaches with broader policy commitments, if we want to harness the transformative potential of these approaches in the real world.

TABLE 2 :
processes for systems learning in groups informed by theorists >RESEARCH | REVIEWNikoSchäpke et al.Comparison of real-world laboratories (RwLs)  to similar approaches and derived learning possibilities for RwL research, based onSchäpke et al. (2017), a stronger focus on (a); contribution to change via socially robust and evidence-based solutions to sustainability challenges no particular form observed actionable knowledge, potentially directly contributing to change primarily participatory; partly fully controlled experiments reaching from information giving to consultation, collaboration to empowerment, (potentially) differentiated depending on process phase and lab/experiment no particular actor groups in focus both, different foci (either societal or societal and scientific) of knowledge generation stated temporary set-up by researchers; passing over labs and experiments to societal actors or institutionali zation often planned for aimed for, mechanisms currently unclear, e. g., generalization of results e. g., offering learning space, enabling experimental learning cycles offering protected space for negotiations and collective meaning-making to build trust reflection exercises relating to roles as well as epistemic and cultural differences Learning contributes to realizing the four other characteristics as a cross-cutting principle.SUSTAINABLE LIVING LAB (SLL)both, with a stronger focus on (a); contribution to change by testing socio-technical innovations in real-world settings and developing new production and consumption modes, generating evidence on sustainable product and service systems prototypes and field tests, comparative studies of standardized experiments, relying on mixed methods actionable knowledge mostly fully controlled combining different forms and intensities of user engage ment depending on phase, mainly low to medium intensities focus on integration of value-chain actors, mainly households and companies both long-term perspective covered by combination of permanent infrastructure and real-life testbeds (household/district panels) aimed for via market and social mechanisms, besides focus on generic and transferable insights via standardization and comparison of experiments various competencies addressed based on explorational learning developing tacit knowledge about sustainable user practices by integrating users in experiments open didactic exploration and feedback processes to foster transdisciplinary collaboration and reflexivity a clear focus on (a), goal: sustainability transitions; contribution to change via orchestration of transformative agency and experiments linked to a broader, reflexive governance approach (transition management providing an underlying theory of change) (portfolio of) transition experiments to test options for sustainability transitions embedded in transition management approach actionable knowledge, potentially (in)directly contributing to transformation participatory varying intensities, depending on process phase, in general strong orientation towards empowerment main focus: societal frontrunners, engaging with political actors both, with primarily societal outputs temporary set-up by researchers by request of societal actors, passing over labs and experiments to societal actors often planned for aimed for via empowerment, as well as deepening, broadening, and scaling-up mechanisms; stabilization via governance approach combining orchestration of experiments, strategic collaborations, and facilitation of social learning competency development no explicit aim; implicitly, collaboration and transition thinking competencies get augmented joint exploration to develop new, transition-oriented understandings of problems and solutions (problem-based learning) transdisciplinary learning environment, based on iteration between scientific and societal perspectives, asking for researchers' reflexivity CHALLENGES ENCOUNTERED IN RWL RESEARCH, LEARNING POSSIBILITIES FROM COMPARED APPROACHES balance dual aim of understanding and facilitating change, and related descriptive-analytical and prescriptive focus; underlying theory of change is lacking For shaping the transformative research practice, RwLs can provide room for plurality and systematic exploration -via explicitly relating RwLs to different theories of change, and practicing research components from the other lab approaches.epistemological questions about evidence, methods of participation in experiments, and ethics RwLs may learn about generation of generalizable insights from controlled experiments in quasi real-world settings (SLL), integrating experiments in larger governance activities towards transitions (UTL), and methodic embedding of experiments in socio-ecological systems (T-Lab).

TABLE 1 :
Core characteristics of real-world laboratories.Results of own research as well as input from exemplary literature is presented here (for an extended list of literature underpinning stated characteristics, see online supplement 3 ).

contribution to transformation experiments as core research method transdisciplinarity as core research mode long-term orientation, scalability and transferability of results learning and reflexivity STATING REFERENCES (EXEMPLARY)
). Re search ers in sustainability science use different forms of experimentation.A basic differentiation includes 1. the forms of control researchers can have on interventions (i.e., full, external control, participatory control, and no control), and 2. the subject that experiments seek to generate evidence about (i.e., sustainability problems, and related descriptive-analytical knowledge, or sustainability solutions, and related actionable knowledge).Labs usually provide a concrete temporal, geographic, communicative, and resource-based setting for experiments.In such settings, research -