Developing an Organizational Framework for Informal Physics Programs

Characterizing the landscape of informal physics learning is a necessary and important endeavor that requires an in-depth study of the different types of existing programs. In this project we focus on informal programs sponsored by physics departments in academic institutes and physics national labs. We seek to understand the structural elements and cultural practices within the diverse array of informal physics spaces. Thus, we are developing a framework based on organizational theory that is contextualized for informal physics programs. We have collected data using survey and interview protocols for different informal physics activities. Here, we present an in-depth case study in which we have analyzed a “science cafe”-style event facilitated by volunteers from a physics and astronomy department. Applying organizational theory to the data set allows us to determine the ﬁne-grained details of the program and the interconnections between the key elements of programming, personnel, resources, audience and institution. We further characterize and discuss the challenges the program faces from the facilitator’s perspective. From these ﬁndings, we will be able to look at larger data sets in the national study.


I. INTRODUCTION
Informal physics educational environments comprise a wide-ranging landscape of diverse programs, activities, formats, types of physics content, and public engagement.These environments are complex, and understanding their activities, how they are connected to their institutions, and how they perform and function is a major challenge.Unfortunately for both learners and educators, informal physics programs are understudied compared to formal physics classrooms, and often exist on the fringe of academic physics departments and national labs.However, since these programs and activities are the face of physics to the public and the next generation of scientists, it is critical that they be studied, characterized, and evaluated.Not only do such outreach activities impact the public, but also, they are important venues and groups for physics students and physicists to feel a sense of belonging, find an outlet from daily research challenges, and demonstrate their expertise to the public [1].Without a clear picture of what resources, time, and experiences are being committed by the physics community as a whole, we are not able to collectively leverage the powerful affordances of these environments for learners.We are also limited in our ability to critique these efforts in terms of inclusivity and equity so as to improve public engagement with physics.
This work is part of a large-scale project to characterize the landscape of informal physics efforts, with an intended outcome of understanding what factors influence the effectiveness of such programs [2].Our broader goals are to understand what factors are important for the 'success' of an informal physics program and how these factors affect each other.We also want to know what types of challenges these programs facing.
The purpose of this article is to test ideas from organizational theory (OT) on an informal physics program to see what emergent factors play a role in the functioning of the program.Our specific research question is thus: Which ideas from organizational theory reveal factors that play a role in the functioning of informal physics programs?We build off earlier work that looked at program aspects at a coursegrained level [3].Here, we describe our development of a coding scheme through a case study on a science cafe/pub style program.By conducting highly detailed coding of an interview with the lead program facilitator we are able to see themes and connection emerge between key areas of program personnel, content, resources, audience and institutional connections.We are also able to identify challenges the program and facilitator face related to these key areas and explore the connections between them.The framework presented here is not a final product, but rather it will continue to grow and be refined with the addition of new programs to the analysis.Thus, the framework will evolve to give a basis for comparison across different informal programs and development processes [4].Connecting to the larger project, this work is the beginning of creating a model for understanding and evaluating the indicators of 'success' in informal physics efforts.

II. INFORMAL PHYSICS CONTEXTS
Non-school science learning environments have the powerful capacity of attracting the public to science.Significant effort has been put into building informal science programs and activities over the years in different capacities and with different formats.These groups have many differences, but they have common goals: They all interact with the public to enhance scientific capacity by generating excitement and interest in science and motivating the younger generation to pursue science.Examining and studying how those programs work has been the focus of several studies over the last decade in informal science education.These studies tried to investigate the related efforts, identify their challenges, share ideas and resources, and discuss future opportunities [5][6][7][8][9][10][11][12][13][14].
To tackle the complexity of environments, we intentionally limit our focus in this study in several ways.We focus on academic informal physics programs and activities, often called physics outreach, that are run by physics departments, physics centers and national labs.We also currently look at only in-person interactions between the public and physicists.We take public audiences to be those who are not enrolled in college level physics programs or who hold degrees in physics.Examples of programs fitting these criteria include after-school programs, summer camps, public talks, demonstration presentations, open houses, science festivals, and planetariums.Many physicists also engage the public through social media, websites, publishing books popularizing physics, and developing related television, movies and games; these areas will be explored in future work.

III. ORGANIZATION THEORY (OT)
Since we seek to understand the way these programs function, we turn to OT. OT is concerned with the effect of social relationships between individuals within an organization along with their actions on the organization as a whole.Every organization consists of interdependent, socially constructed groups with different goals, different ways of working, different formal and informal training, and personality types.Furthermore, OT describes how organizations are affected by internal and external components such as social, cultural, legal, and political aspects of the environment [4].
Our primary research focus in this paper is an in-depth understanding of how informal physics programs function as interdependent, socially constructed groups, and how they connect or can connect to their institutions.We propose that some informal physics programs may share similar features to nonprofit organizations, thus making organizational theory a reasonable way to approach our research questions.Typical indicators include tangible and intangible resources (such as personnel and equipment), finances, management, and environment [15].However, contextualizing organizational theory to informal physics programs is necessary.There are already variations in the literature on the specific manifestations of the constituent parts of an organization.For instance, a common business category is "board members", while "stakeholders" or "collaborators" would be more appropriate for an informal physics program.We integrate different existing organizational theory frameworks from the literature.This 'multiple-constituency model' [15] brings together different indicators and takes into account different stakeholder's expectations, goals and objectives.With this approach, we seek a multidimensional, comparative and objective analysis to evaluate informal physics efforts.

IV. METHODS
The overarching landscape project is designed to collect the basic information and the detailed stories of existing informal physics programs.Six themes were identified from OT [4] and relabeled as the main topics: 'personnel', 'program', 'institution', 'resources', 'audience', and 'assessment'.Our data collection is a package of a 20-minute survey followed by a one-hour semi-structured interview as well as conducting some site visits with lead practitioners.Survey and interview questions covered factual information as well as information about the history, changes and connections of the program.Protocols, described in [2], were developed through multiple rounds of testing and feedback with varied subjects.We are in the process of determining potential participants, obtaining as high as possible response rate, and collecting diverse data to be a good representation of the national landscape.
In this paper, we focus on an in-depth case study of a 'science cafe'-style event series, which we will refer to as "Pub Physics".This program model is common in the United States -in this specific case, Pub Physics consists of monthly meetings at a local bar where public talks on astronomy, physics, and related topics are given by members and friends of the physics and astronomy department at a large predominately white R1 university in the Midwest.Graduate students from the department are recruited to attend as well and answer audience questions after the talk.This event is advertised to the general public and has consistent attendance of 80-100 people.Pub Physics is run by 'Tom', a white, maleidentified physicist, who is a non-tenure track instructor and whose main job in the university is not associated with the outreach activity.He runs almost all logistical aspects of the program on his own, relying on graduate students and other faculty to speak at and sometimes attend these events.The program does not receive any financial support from the institution, despite its advertising vaguely alluding to the department being a sponsor.This program was selected because: 1) it has been running for over four years with consistently high attendance, despite the challenges it has been facing, 2) it represents a traditional public lecture which no previous skills are needed for running it, and 3) it is run by physicists.We collected survey and interview data from Tom, the program's lead facilitator, as well as write-ups and field-notes from attending the event several times.
Our analysis process started with an initial coding of interview and survey data in MAXQDA by a single coder, using the six themes, exploring emergent ideas and information.After discussion in the research team, these emergent ideas were then coarsely organized under the broader themes, guided by the main areas in the OT framework [4].Those themes are commonly used in OT literature, but can have slightly different labels [4,15].The broad OT themes were then taken up by a second coder who used them as guideposts while coding in the same emergent fashion as the first coder.Then the emergent codes for both coders were compared and discussed in detail by the research team.Through long discussions, the emergent codes and the OT themes were iteratively modified until the research team came to agreement on the organization.A single code constitutes the sentences in a response that meets the definition of a code or sub-code, which can be from one sentence to a few paragraphs, but not partial sentences.Codes can overlap and each response can contain a number of codes.The more specific categories of emergent aspects of the programs were made into sub-codes of these themes.This step was followed by additional discussion, reorganization and re-coding processes that resulted in the final version of our code system.
Our final step was to explicitly look for mentions of any type of challenges in the data set that the program is facing and categorize them based on the broader theme they fit into.For example, if some challenge is related to the recruitment of the personnel, it was coded as 'personnel challenge'.By looking at the overlaps between different categories of challenges, we were able to find what challenges are the most identified issues of the program under study.

A. Complexity
For this program, we find at least 33 different sub-codes describing key features that are mentioned in the interview or in the survey response, as shown in Figure 1.This finding demonstrates the complexity that informal physics programs have to grapple with.However, this finding is likely not surprising to practitioners, as public engagement programs have at least as many different moving pieces (so to speak) as most classroom settings, and in some cases maybe considerably more.The dimensions shown here are organized by highest number of counts to lowest in each thematic area, thus giving a holistic picture of the specific key aspects that are important to the functionality of the program.
From Figure 1, 'personnel role in the program', 'advertisement and visibility', 'program events and activities', 'personnel culture', 'audience satisfaction', and 'personnel main job' have the highest number of codes for this program.These sub-codes identify the program features that are most important to the facilitator interviewed.For Pub Physics, these aspects are interconnected.Tom speaks at length about plan- ning the events nearly single-handed, although he works fulltime as an instructor at the university.His role in the program includes recruiting speakers and volunteers, coordinating with the venue, making sure the program is advertised, and hosting the actual events.He also describes the importance of creating a certain welcoming atmosphere at the events for audience members and graduate students from the department alike.He mentions that the program is wellattended in comparison to similar programs in the area and nationally, but that the demographics of the audience are not necessarily representative of the diverse population of the surrounding city, which he would like to improve on.Financially, the program is in good shape, with donations from the audience and contributions from the venue covering costs.
Figure 1 also shows that 'audience monetary cost', 'training provided', 'impact on the audience', 'audience community needs', and 'collaborations' were least mentioned by Tom.Some of those aspects may be less important to the people who run the program, potentially because they are well achieved thus not a cause for concern.For example, this program is free and located in a bar and it would naturally attract its audience.Tom also mentions how he cares about creating a comfortable and relaxing environment for the audience, which positively affects how they feel about attending the events.

B. Challenges
From our analysis, we find that challenges that the Pub Physics program faces are prevalent, important to its functionality, diverse, and complex.Challenges were categorized by thematic categories, which helped us define and operationalize this aspect of the framework.For example, 'personnel challenges' were directly related to the sub-codes within the personnel category, with the same for 'program challenges', 'institution challenges', 'assessment challenges', 'resource challenges', and 'audience challenges'.In Tom's interview, we coded challenges in every thematic category.
The highest category of challenges is personnel (39% of the total counts of all six challenge types).Recruitment, role in the program, and personnel main job issues are the three main contributors to this category of challenges.One example reported by Tom is having difficulty finding speakers in his current institution's smaller department compared to his previous university.Tom also says it can be difficult to find help from other members of the department for the actual logistical planning of the events; this was identified as personnel recruitment challenges.While some graduate students offered to help with live-streaming and some other details, these students often graduated and then left the program, or they did not volunteer to administer larger tasks.Tom explains: "...Awhile back I tried to recruit sort of, a team of people to help sort of do the social media management, live tweeting, recordings, speakers, and all of that stuff.I got a couple of grad students to tentatively administer it...And then when I tried to dole out tasks, they didn't happen, so I just started doing it all myself." The next three categories of challenges are similar in weight: institution (20%), program (17%) and resource (15%) challenges.The main institutional challenge is institution culture, while the main program challenges are advertisement and visibility, and the main resource challenges are finding new resource opportunities and physical resources.For example, Tom states: "...So when [the original] venue went out of business, that threatened the program because obviously I didn't have a place to do anything, right?I spent about four months shopping around.I tried a couple of different places.One building worked, very supportive but just not the space."Pub Physics comparatively had few audience challenges; this was understandable by looking at the audience satisfaction sub-category, which has the highest count among audience category.By 'audience satisfaction' we mean capturing any situation that describes how frequently audience members attended the events, how they engaged in repeated participation, and any evidence of loyalty to the program, such as asking facilitators for more programming, donating money to the program, or telling friends about positive experiences.
Not only did we capture all the challenge categories, but we also investigated the overlaps of challenges with each other, and with other sub-codes.Figure 2 is a schematic illustration of how different challenge categories are interconnected.Each circle represents the challenges related to one of our main categories (i.e.personnel challenges).The size of each circle is directly proportional to the number of code counts in that category.Since 'personnel challenges' has the highest number of codes among all categories, it is the biggest circle and 'assessment challenges' which has the lowest number of challenges is the smallest size.The lines between the circles shows the overlap between different challenge categories, and the line weight is directly proportional to the number of counts for the overlaps between each two challenge categories.In other words, the lines show how interconnected any two challenges are.Personnel challenges and program challenges have the highest amount of overlap resulting in the link with the most weight as shown.And 'program challenges' has overlaps with three other categories of challenges: 'institution', 'resource', and 'audience'.The quote below from the interviewee talks about the kind of support he receives or would expect to receive from the institution: "...I never really got much in the way of actual support from the department either advertisement or financially supported, or anything like that.So I got a pat on the back in some sense at one point, but other than that, yeah.I don't even know how many people in the physics department knew I was doing it even though oc-casionally we would send out department-wide emails."We coded this section as an 'institution challenge' because Tom talks about some difficult situations in which the institution does not choose to help with the outreach program.One aspect of this situation is that Pub Physics does not receive financial support from the institution, a 'resource challenge'.We also identify a 'program challenge' because this statement shows that the lack of support from the institution was affecting the program advertisement and visibility.These issues are again connected to the related personnel challenges arising from the fact that Tom does all these tasks on his own since he has limited support from other members of his department.Our findings do not suggest that informal physics programs are least concerned about assessment and audience, as Tom talks about how he would like to do assessment, but he does not have the knowledge nor the resources to do that.

VI. IMPLICATIONS
The case study in this paper shows that each program consists of various crucial dimensions that can affect the program's functionality and sustainability.Our comprehensive multi-dimensional analysis identified over 30 different individual sub-categories, from the facilitator's perspective highlighting the key elements of the program.This large number of categories demonstrates the complexity of the studied informal physics program and other informal efforts in general.
We also identified the challenges that this particular program is facing as the first step towards diagnosing the factors that affect the functionality of the program.Providing this coding scheme as an 'external' evaluation for study participants could be useful for programs' self assessment.In future work, we will explore the opportunities for tool development and dissemination.With the help of such an in-depth analysis, programs could find and focus on the areas they need help with, make use of communication channels to reach existing and potential resources, and form possible collaborations.
Going forward, we will continue to use our collect and analyze data for more and different informal physics programs.We will look at how success can be defined for programs and how programs might define success for themselves.To this end, we will apply our model to more data to be able to find and measure some of the social concepts associated with such programs, including 'accessibility', 'effectiveness', or 'inclusivity'.

FIG. 1 .
FIG. 1. Counts of sub-codes, color coded by dimension and organized by highest number of counts to lowest in each thematic area.