Breaking Down Silos Working Meeting: An Approach to Fostering Cross-Disciplinary STEM–DBER Collaborations through Working Meetings

INTRODUCTION

Although there is much that is unique to the individual discipline-based education research (DBER) disciplines, there is also a considerable amount of overlap in research interests between DBER scholars. As an example, life sciences and geoscience education researchers both tend to be situated within disciplinary departments, they are governed by the rules of academic departments, they think deeply about student thinking, and practical experiences are critical to the learning of their students. For this reason, there is much that can be learned through collaborations across disciplines. At the same time, such collaborations are difficult to foster, because there are few formal venues designed to support them. Although professional conferences are effective for organizing individuals within the same field, they are not often well suited to cross-­disciplinary connections.

To address this challenge, we designed Breaking Down Silos, which was a small, collaborative working meeting focused on issues of institutional transformation in science, technology, engineering, and mathematics (STEM) higher education. The participants were DBER scholars who came from many different disciplines. Because STEM educational change is a relatively small field, and DBER scholars across all disciplines are actively involved in it, there is a great need for scholars to collaborate across the disciplines to generate knowledge in this area. In this meeting report, we describe our process for developing the meeting and share preliminary outcomes. We found that the structure of the meeting was productive for creating the types of collaborations we had hoped to foster, and we propose that it is one model that can be used to foster other cross-disciplinary STEM–DBER collaborations.

BACKGROUND

Change in STEM Higher Education

Change in higher education is a rapidly growing field; a recent meta-analysis reviewed 191 STEM change efforts from 1995 to 2008 alone (Henderson et al., 2011). Yet most change efforts are guided by overly simplistic, often implicit, theories of change (Borrego and Henderson, 2014). Indeed, the authors of the meta-analysis concluded that narrow approaches (e.g., dissemination, top-down mandates), which were nearly 60% of efforts reviewed, were “clearly not effective” (Henderson et al., 2011). A typical folk theory of change is “if you build it, they will come.” In the physical and biological sciences, when new techniques are developed, they are quickly adopted by labs across the world. Following this logic, one would similarly expect that active-­learning techniques and curricula that support active-learning approaches would be adopted in a widespread manner, but empirical evidence shows otherwise (Henderson et al., 2012; Apkarian and Kirin, 2016; Stains et al., 2018).

Given the inefficacy of simplistic “scale-up” approaches to change, there is a need for better theories to guide change efforts and to ground research about how change occurs (Fairweather, 2008; Austin, 2011; Kezar, 2011). Although there are many theories of change potentially relevant to the transformation of undergraduate STEM education, determining which theoretical framework(s) best inform a project is challenging. Relevant theories come from diverse areas, including organizational psychology, higher education, health sciences, and business management (e.g., Dutton and Duncan, 1987; Feldman, 2000; Gess-Newsome et al., 2003; Rogers, 2010; Kezar, 2014; Taylor et al., 2006). Moreover, scholars from different DBER disciplines draw upon different subsets of this theory, given their own backgrounds. The complexity and breadth of this literature makes identifying and understanding relevant theories challenging. Thus, we see a need for better organization and synthesis of theoretical frameworks potentially relevant to change in STEM higher education. Such resources could guide change agents and researchers as they plan, implement, and study change efforts.

Furthermore, most investigations of change in undergraduate STEM education focus on a single initiative. Consequently, many investigations must be compared if we hope to make generalizations about what promotes change. These comparisons would be facilitated if researchers investigated common factors and used compatible theoretical perspectives; this would be more likely if they were aware of one another’s work. Unfortunately, many leading journals and conferences for disseminating research on change in undergraduate STEM education are discipline specific, limiting the degree to which new work builds on prior work. Thus, we also saw an immediate need for better communication and collaboration among change researchers across STEM disciplines. Not meeting this need limits the impact of current and future change initiatives in advancing our knowledge of how to improve undergraduate STEM education. These two pressing challenges motivated the Breaking Down Silos working meeting.

Some important work has already been done to organize theories of change. For instance, Kezar (2014) presents six main schools of thoughts related to change in higher education. These schools of thought frame the types of issues that should be considered by change agents as they plan and carry out change initiatives. This is an invaluable resource to those planning change efforts, but more work is needed to develop syntheses of the change literature that can guide research across the STEM disciplines. This is also related to the social aspect of research; simply because there are common frameworks does not mean that they will be used across disciplines. Rather, it is the collaborative process of developing frameworks in a way that crosses disciplinary silos that is most likely to lead to comparisons across DBER fields.

The effort to organize and synthesize theories is timely. It is becoming more common for change efforts to aim at grounding their work in change theory (e.g., departmental action teams; Reinholz et al., 2017; Student Engagement in Mathematics through an Institutional Network for Active Learning; Association of Public and Land-Grant Universities, 2016). Additionally, funding agencies increasingly expect a strong theoretical basis in systemic change for STEM education reform projects (e.g., National Science Foundation [NSF], 2017). Projects of this sort have the potential to help contextualize the change literature to STEM departments in higher education. This is a promising new area of inquiry that would benefit greatly from a tightly connected network of scholars across disciplines who are aware of one another’s work.

STEM–DBER Alliance

Our effort to foster collaborations among STEM–DBER researchers builds on other recent national efforts. For example, the STEM–DBER Alliance (DBER-A; Henderson et al., 2017) is designed to bring together DBER scholars across the STEM disciplines to help create generalizable knowledge that exists at the intersections of DBER research across disciplines. In May 2017, a meeting was held to help launch the DBER-A, and at that meeting, systemic change was identified as one of seven priority areas for the work of this community. The other six priority areas were: active learning, assessment, broadening participation, how people learn, professional development, and scientific practices (Henderson et al., 2017). Discussions continued at the Transforming Research in Undergraduate STEM Education conference in July 2017, which is where the two conference organizers of Breaking Down Silos met and conceptualized this meeting. In this way, the present report represents one of a series of meetings that aim to establish stronger connections among STEM–DBER scholars.

Accelerating Systemic Change Network

Given our focus on institutional transformation in STEM higher education, our goals closely aligned with the recently launched Accelerating Systemic Change Network (ASCN; https://ascnhighered.org/index.html). The goal of ASCN is to help generate, curate, and disseminate research-based knowledge about how to scale and sustain changes in undergraduate STEM education. ASCN is intended to serve as a professional network and intellectual home for individuals and groups in a broad range of disciplines who are engaged in creating and/or studying change in undergraduate STEM education. ASCN consists of a variety of topical working groups and has a steering committee composed of many leaders in the field of educational change. In ASCN, Working Group 1 focuses on helping people engaged in change efforts understand change theory and aims to build a common framework and language related to change theories. ASCN provided considerable support to our meeting by providing connections to their network and also to expert consultants who contributed to the meeting.

THE BREAKING DOWN SILOS WORKING MEETING

Our goals for the meeting were to make progress on organizing and synthesizing theories relevant to change in STEM higher education, to support emergent outcomes of interest to meeting participants, and to foster cross-disciplinary collaborations. Given our goals of breaking down silos across STEM fields, we took a number of intentional steps to design a meeting that would foster connections. Here, we describe four: 1) the recruitment of participants, 2) premeeting work, 3) design of the meeting, and 4) building lasting collaborations.

Recruiting Participants

We engaged in an intentional recruitment process with the goal of diversifying the participants at our meeting across a variety of characteristics. Our goal was to include participants representing all of the STEM fields and to achieve diversity with regard to race, gender, and position. While we recognize that there are other relevant dimensions of diversity (e.g., dis/ability status), these dimensions were more difficult to address without knowing the participants personally. We focused on early-career scholars, because we wished to foster collaborations that could last many years to come and that united scholars who may not have otherwise had the opportunity to meet. Most participants were early-career faculty, but the meeting also included some recently tenured faculty, postdoctoral researchers, and graduate students.

Recruiting participants for a meeting of this sort is challenging, because by our very nature DBER researchers tend to be in disciplinary silos. To address this challenge, we used a variety of methods. First, we asked colleagues in our personal academic networks to identify scholars outside our disciplines (mathematics and biology) who would be appropriate for the meeting. Second, we searched all recent NSF awards focused on institutional transformation to identify principal investigators. Because this biased our search toward tenured faculty, we looked for early-­career scholars who were a part of these projects, using the NSF website and other online information. Third, we asked our advisory board—who spanned a variety of disciplines—who they would recommend for this meeting. These three procedures helped us develop an initial sample of participants.

We sent out preliminary invitations to participants from this list and asked them to complete a survey. The survey asked respondents to share the theories they were already drawing upon in work related to change in STEM higher education. The survey also asked them to recommend others who we should invite to the meeting. This process of snowball sampling allowed us to identify others who were less proximal to our own academic networks. It was also critical for identifying graduate students and postdoctoral researchers who could attend. We incorporated these suggestions as we sent out a second round of invitations. We were able to represent the following disciplines in our meeting: biology, chemistry, engineering, geology, mathematics, physics, and higher education. Although we also aimed to include computer science, we were ultimately unsuccessful. Of the 24 participants, five were graduate students and two were postdoctoral researchers. All other participants held long-term positions at institutions of higher education.

Premeeting Work

We modeled our approach to premeeting work after the initial Pedagogical Content Knowledge Summit. Preparatory work was key to meeting productivity and to creating a “level playing field” for all participants to rigorously contribute to discussion at the summit and at our Breaking Down Silos meeting (Carlson et al., 2015, p. 20). We aimed for premeeting work to allow us to dive immediately into the core work of the meeting and to foster community and inclusion during the meeting.

We asked participants to complete three tasks before attending the meeting. First, we asked each participant to read two papers that each summarized a change theory potentially relevant to STEM higher education. We selected a total of eight readings that represented a range of theoretical perspectives that we knew our participants were drawing upon based on the premeeting survey. We chose readings or sections of readings that presented the theories succinctly to minimize the premeeting time burden for participants. We included a theory of individual behavior change (theory of planned behavior; Madden et al., 1992), theories about organizational change (double-loop learning; Argyris, 1977; 4i framework of organizational learning; Crossan et al., 1999), a theory focused on the system in which teachers work (teacher-centered systemic reform; Woodbury and Gess-Newsome, 2002), and a theory about organizational culture (four frames; Reinholz and Apkarian, 2018). We also included three papers that present approaches to achieving change that are sometimes referred to as theories, including the Science Education Initiative (Chasteen et al., 2015), appreciative inquiry (Cooperrider and Whitney, 2001), and the Keck/PKAL Model for Systemic Institutional Change in STEM Education (Elrod and Kezar, 2016).

We divided the readings among participants strategically so we could begin the meeting with a jigsaw. We asked participants to be ready to explain the change theories they read to a small group of meeting participants who had not read the same papers. Second, we asked participants to prepare a 2-minute talk with a single slide to describe a change effort in which they had participated and the theories underlying this project. Our goal for these presentations was to help participants learn about one another’s work and to highlight for participants the wide range of theories on which they drew. Finally, we asked each participant to upload a picture and short biography so that participants could get to know one another without including lengthy introductions during the meeting. Our meeting was organized with Google Drive, which allowed us to share materials early and conveniently.

Meeting Design

We included intentional design elements with the goal of creating an inclusive meeting, much like undergraduate STEM instructors aim to create inclusive classrooms. We began by setting norms. We had a set of meeting guidelines that we shared with all participants and visibly placed on our working tables throughout the meeting (see the Supplemental Material). Our goal was to make explicit our goals for interactions and also to explicitly acknowledge that some participants tend to dominate a conversation, based on their power and privilege, and that this often aligns with historically dominant populations in terms of race, gender, and status (e.g., Sadker et al., 2009; Eddy et al., 2014; McAfee, 2014; Grunspan et al., 2016; Smith et al., 2019).

We aimed for all participants to be fully involved in the working meeting throughout and to meet and work with every other meeting participant. We also recognized that having too many talks or an expectation that participants will sit and listen for many hours straight is not inclusive for individuals who have dis/abilities. Therefore, we structured the meeting as mostly small-group work time broken up by report-out and generous breaks (Tanner, 2013). Recognizing that some participants would already know one another, but others would not, and that not everyone is equally likely to spontaneously strike up conversations, we used small-group structures to help participants mix and meet. We assigned participants to the first three groups with whom they interacted, and they completed substantive work with each group. The first group included all participants who had completed the same premeeting reading, and their charge was to work together to feel prepared to describe the theories in a jigsaw with participants who had read about other theories (see meeting agenda in the Supplemental Material). Next, we gathered participants in groups in which each group member had read a different set of two premeeting readings about theory. The goal of this breakout session was for each participant to explain two theories to the group and to begin to compare and contrast theories. This work was inclusive, because everyone had the opportunity to prepare and to share ideas with a small group (e.g., Seidel and Tanner, 2013). We deliberately created groups that mixed discipline and position type. In the third breakout session, small, assigned groups collaborated to generate ideas about what a synthesis and organization of theories could contribute to the field, and what they, as individuals, wanted to accomplish in the meeting. In total, Breaking Down Silos meeting participants worked in assigned groups for about the first half of the meeting and grouped by affinity into three groups thereafter. Following the opportunity to work with many of the other participants in small groups, we hoped that participants would have some sense of who they might collaborate with moving forward. We also hoped that, if we were able to create space to foster these collaborations during the meeting, they would be much more likely to blossom after the meeting was over.

We expected less-structured interactions to also be important to community building and to make the meeting valuable for scholars with diverse interests and needs. Therefore, we scheduled 30-minute breaks each morning and afternoon and stuck to that schedule, even if we ended up behind on other meeting activities. We also helped participants organize into small groups to eat dinner at local restaurants, providing additional opportunities for networking, and making sure that everyone had company for dinner if they wished.

As noted earlier, we designed the meeting to progress from more to less structured, with the ultimate goal of participants determining the next steps that would be most useful to them. Early breakout sessions gave participants explicit instructions about what to discuss. Much like designing a lesson in an undergraduate course, we sought to make expectations for what participants should accomplish in a given time period very explicit, both verbally and on slides that remained visible throughout the breakout session. We provided less explicit expectations in later breakout sessions. About halfway through the meeting participants collectively generated a wide array of possible next steps and then self-organized into “affinity groups” to pursue the next steps that most interested them.

We did not enforce a minimum or maximum group size for affinity groups, because we wanted participants to be able to pursue their own interests. However, one group divided into two smaller groups for some of their work time in order to give everyone a chance to contribute. This was important, because the voices of some of the graduate students were lost when the entire group worked together. In future meetings, we would offer more explicit facilitation for affinity groups larger than five or six participants.

Participants had extended periods of time to work in these groups (more than 3 hours) so that substantive progress could be made during the meeting. Notably, only one of the next steps taken aligned with the initial vision of the meeting organizers (organizing and synthesizing theories). We consider that a powerful example of the new and diverse ideas that can emerge from intentional cross-disciplinary events. We describe the efforts of the three affinity groups in more detail later.

We were also intentional about the role of facilitators and participants in the meeting. The meeting organizers, who are also the authors of this report, facilitated the meeting by managing the report-out sessions after small-group breakout sessions and adapting the schedule as needed to respond to what was occurring. We also joined small groups during breakout sessions to listen and contribute. In addition to 17 early-­career STEM–DBER scholars, the meeting participants included four advisory board members who are well-established scholars of change in STEM higher education. We asked these advisory board members to participate in the same ways as other meeting participants. We were aware that including established scholars could lead to early-career scholars being less likely to contribute their ideas. Therefore, we strategically divided the advisory board members among assigned groups and aimed to include diverse voices in all report-out sessions. We encourage other meeting planners who aim to focus on early-career scholars to choose established scholars who do not dominate conversations and to spread them throughout groups. A rough rule of thumb might be to include no more than one established scholar for each small group that will be created during breakout sessions.

Building Lasting Collaborations

Although we are reporting on the meeting shortly after its completion in February 2019, we also have intentions to foster ongoing community after the meeting. One concrete way this will take place is the authors are taking over leadership of ASCN Working Group 1, and they have invited all participants from the meeting to join that effort. A second method is to keep in touch virtually with the various affinity groups (described later) that formed from the meeting. One of the goals of the affinity groups was to engage participants in work that would support them in the early stages of their careers, which we thought would increase their likelihood of continued engagement. We also recognized that not all meeting participants would be interested or able to continue to collaborate after the working meeting. We acknowledged this explicitly in the meeting and invited participants to be as involved going forward as made sense for their careers.

OUTCOMES

Participants organized themselves into three affinity groups, all of which are working on concrete outcomes. In addition, an important outcome from the meeting was building community among participants. We have strong preliminary evidence that this was achieved based on the evaluation survey, which we discuss later.

The Inclusion Assurance Group

Four participants at the meeting formed into the self-titled “inclusion assurance group.” The goal of this group, which was completely emergent (i.e., not determined before the meeting), was to address potential biases that could be implicitly embedded in an organizing framework to synthesize theories of change and in any process aiming to review and organize scientific literature. For example, our sampling process for finding scholars meant that the literature we were drawing from at the meeting was somewhat proximal to our own work. In any review of the existing knowledge base, how do we make sure we include an appropriate diversity of research? And how do we account for the somewhat homogeneous gender and racial demographics of scholars we may have easy access to in this area? These were the types of questions this group aimed to tackle.

The group has produced a guide to help researchers ensure that inclusion and diversity are seriously considered in the processes of organizing, synthesizing, and reviewing literature in STEM higher education and beyond. The guide, titled Guideto Inclusion Awareness in the Organization of Knowledge, is freely available on the ASCN website (https://ascnhighered.org/ASCN/posts/inclusion_guide.html; Apkarian et al., 2019). In alignment with the goal of inclusivity, the other two affinity groups stated their commitment to using these guiding questions to move their work forward.

The Elephant in the Sandbox

The second affinity group, consisting of nine participants, named itself “the Elephant in the Sandbox.” The goal of this group, which was completely emergent, was to use a variety of theoretical frameworks to analyze existing change efforts in STEM educational transformation. The group decided to contrast multiple theories that its team members had close knowledge of—the “sandbox”—to learn what different frameworks could offer to understanding an effort. The “elephant” in their group name reimagines a popular proverb about blind men examining an elephant and arriving at different conclusions about what it is. The man who touches the elephant’s trunk may conclude the organism is a snake, while the man touching the elephant’s ear may conclude that it is a fan. Similarly, this affinity group expects that applying different theories to understand a single change effort will provide different perspectives about what is occurring and why. In essence, this group is creating an early test case to demonstrate the value of change theory in understanding change efforts in STEM higher education. This affinity group has had ongoing meetings following the conference and envisions writing a journal article describing their application of theory.

The work of this group highlights the potential of a meeting of this sort to create new knowledge. Both the theoretical frameworks and the specifics of particular change projects are distributed among the participants in the group, so it would have been very difficult for any of them to work individually on this goal. These participants also noted that few of them knew one another before the conference and likely would not have encountered one another through other professional venues. Thus, Breaking Down Silos created the opportunity for them to begin this collaboration.

Creating an Organizing Framework for Theories

The third affinity group took steps to move forward with creating an organizing framework of theories relevant to change in STEM higher education. This goal aligned with the original intentions of the meeting organizers to synthesize and organize theories to make them more accessible and useful to change researchers. One area of early consensus among this affinity group included the recognition that the term “theory” is broadly and inconsistently used to describe a wide array of potentially relevant intellectual resources. Specifically, the affinity group concluded that some theories provide step-by-step, actionable, and prescriptive stages or steps to achieve change and thus may be especially useful for change agents. These may be similar to logic models developed for projects that articulate how change is expected to occur. Other theories aim to explain how or why change occurs and can provide important theoretical grounding for change efforts and change research, but generally do not stipulate specific actionable steps. These theories tend to be more conceptual and abstract and aim to articulate underlying principles related to change. This affinity group considered a number of theories that were curated and annotated before the meeting, and began to classify them using an emergent scheme. This work generated rich discussion that will be continued as the work progresses.

A subset of these participants will continue to do this work. This is a long-term enterprise, with a number of possible outcomes, including journal articles or other resources for STEM–DBER researchers studying change in higher education.

EVALUATION

The evaluation survey completed at the end of the Breaking Down Silos meeting provided strong evidence that the meeting achieved the desired goal of building connections and collaborations across disciplinary and theoretical silos (see full survey in the Supplemental Material). This evaluation was determined by San Diego State University to not be human subjects research (#Temp-1702). When asked how many new people they met at the conference, 19 participants indicated that they met 11 to 25 people, and two participants said that they met six to 10 participants. The participants also noted that they had some prior collaborations with other participants, but these generally were with the conference organizers or a member of the advisory board. Almost all participants who completed the evaluation (19 out of 21) agreed that the conference gave them the opportunity to engage in conversations about systemic change with researchers from other disciplines “to a great extent.” Unfortunately, meeting evaluations do not include a common set of questions, and the data often go unreported, so we cannot compare our outcomes with those of similar meetings.

Figure 1 provides a summary of participants’ responses to questions we asked about whether they had learned of new theoretical frameworks relevant to their work and built new collaborations.

Participants also provided written responses to a variety of questions, such as what features of the meeting worked well, what could be improved, and how the meeting would support their ongoing work. We used qualitative content analysis to identify the ideas shared by participants and the number of participants expressing each idea.

In describing what worked well, the participants drew attention to many of the intentional design features: prereadings followed by a jigsaw (n = 5), long and frequent breaks (n = 6), assigned groups that mixed up participants (n = 4), and extended work time in affinity groups (n = 4). For example, one participant mentioned,

Other participants appreciated the “intentionality in diversity and inclusion (participants from all career stages, disciplines, and ethnic/racial diversity)” and that the small size of the meeting and frequent small-group work helped them get to know everyone. This participant valued that the meeting was small: “That in and of itself helps to break down the silos because it communicates that your individual perspective is valued.”

We also asked participants to highlight features of the working meeting that differed from their prior experiences. This feedback echoed the features they reported as working well, including long breaks and the small size. One participant emphasized that the narrow focus of the meeting (e.g., change theories for STEM higher education) and the deliberate choice of participants who actively work in that area helped the group feel cohesive: “The strong shared interest of those present—it was more cohesive than I had experienced previously.”

Participants also provided useful feedback about areas where they felt the meeting could be improved. The two most common areas of feedback related to perceived ambiguity about the intended outcomes of the meeting and a need for greater enforcement of the guidelines for meeting interactions. Related to meeting outcomes, some participants felt unclear of the end goal (n = 4). For instance, one participant stated,

This response highlights the tension of planning the meeting, which indeed tried to support both a predetermined outcome and emergent outcomes.

Three participants highlighted the challenges of creating an inclusive space and moderating participation. One respondent noted,

This response highlights the challenge of creating equitable participation and notes that the setting of norms is insufficient. Norms must also be intentionally enforced. In the future, we would improve this by engaging in more active facilitation as the meeting organizers. This might include specifically inviting less vocal participants to contribute and asking more vocal participants to share the stage. An additional approach would be to empower each group to select an “equity checker” who would be responsible for making sure all voices were heard. This role could rotate regularly to distribute both the responsibility and burden. Another option that can be successful is hiring a trained facilitator. That approach might be particularly important if meeting organizers worry about actively facilitating some meeting participants due to power dynamics associated with position, gender, and so on. Despite this challenge, we note that participants were generally positive around this issue. For instance, one of the graduate students responded,

Additionally, one advisory board member provided unsolicited feedback about the value added to the meeting by including mostly early-career faculty, and these comments echoed our own experiences as meeting organizers and participants:

DISCUSSION

The Breaking Down Silos meeting convened with the goal of bringing together early-career scholars in STEM educational transformation to synthesize and organize theories of change and build community. Our perceptions during the meeting, anecdotal feedback during the meeting, and evaluation feedback all indicated that this goal was largely achieved. Participants are working on a variety of concrete projects. Their work has already produced and made available a new guide for other researchers (Apkarian et al., 2019). We also anticipate that other affinity groups will continue to collaborate to produce published scholarly work.

The meeting was intentionally organized with a variety of features to create an inclusive space that embraced a diversity of perspectives. Perhaps most telling is the fact that an affinity group formed with the explicit intention of promoting inclusion for the work that emerged from the meeting. This suggests that the meeting created a safe space for having such conversations. Participants noted that power dynamics were at play, with some participants taking up more space during the conversations, and this warrants more attention in the future.

Our purpose in writing this meeting report was to externalize a variety of principles and the format for designing a meeting of this sort. At this time we cannot compare our meeting outcomes with those of other meetings that gather researchers across DBER, because those data are not published. We describe our approach in order to foster conversations about how meetings are designed and what they can achieve, not to claim that we have excelled where other meetings have not. Given the emergence of the DBER-A and the push for greater cross-disciplinary connections, we hope to offer a productive starting point for others who wish to organize a similar meeting.

ACKNOWLEDGMENTS

REFERENCES