Diaries as a Methodological Innovation for Studying Grand Challenges

Introduction

The burgeoning literature on grand societal challenges spans a variety of spheres (e.g., Ferraro, Etzion, & Gehman, 2015; Gatzweiler, Frey-Heger & Ronzani, 2022), addressing, among others, poverty alleviation (Pradilla, Bento da Silva, & Reinecke, 2022) to affordable health care (Rauch & Ansari, 2021). However, there is still much to learn about how individuals at the frontline of these challenging contexts, experience and cope with extreme situations involving human suffering. While extreme contexts and grand challenges are two distinct strands of literature, many grand challenges play out in extreme contexts (see Hällgen, Rouleau, & de Rond, 2017, for an overview). Relatively little research has attended to the emotional trauma of individuals working to alleviate human suffering in the midst of extreme contexts, such as warzones (De Rond & Lok, 2016; Fraher, Branicki, & Grint, 2017) and refugee camps (De La Chaux, Haugh, & Greenwood, 2018).

Social science has documented numerous sources of data to study grand challenges such as surveys, case studies, interviews, ethnographies, and natural experiments, to capture the complex and manifold dimensions of grand challenges. Such data, however, may not be able to apprehend the real-time feelings, thoughts, and experiences of affected individuals confronting extreme situations. For example, in their study of the social stability in a Kenyan refugee camp, De La Chaux et al. (2018, p. 160) acknowledge the difficulty of studying grand challenges:

[…] sensitive topics such as incidents of rape and the role of mafia gangs remained difficult to discuss as we sensed a clear reluctance to delve into these topics. […] In the follow-up phone interviews, silences, pauses, and hesitation took on a similarly central role and again occurred primarily when we inquired about incidents of gender-based violence and gang structures […].

In the past decades, several methodological innovations have emerged in management scholarship to address various ontological and epistemological considerations. This includes video data, team ethnographies, and multimodality – addressing material, sensory and visual aspects of data (Jancsary, Höllerer, & Meyer, 2016; Kress, 2010) in order to strengthen the existing methodological arsenal, such as field observations and interviews. However, concepts such as emotions and cognitions are still difficult to decipher and access (Zietsma, Toubiana, Voronov, & Roberts, 2018). Oftentimes, “emotions are not worn on sleeves” but instead actors engage in emotional labor, emotional self-censorship, or a strategic display of their emotions (Jarvis, Goodrick, & Hudson, 2019). The well-known example of the air hostess smiling on the job (Hochschild, 1979) points to the emotional labor in the service industry, in which actors perform certain roles (e.g., being friendly and welcoming) that do not betray the feelings that they actually experience.

We propose the use of personal diaries (Burgess, 1984) as a fruitful and rich source of data in the endeavor to study grand challenges by giving voice to participants in the daily praxis of their work. Diaries capture people’s innermost thoughts, reflections and memories that they document without the gaze of social judgment. Writing personal diaries described as a “technology of the self” (Foucault, 1982) is a time-honored and culturally sanctioned way of animating personal values and innermost feelings (Klein & Boals, 2001), and embodying experiences through self-talk. Indeed, diary writing is an established tradition and practice across different cultural backgrounds dating back to famous examples such as the so called “Pillow book” by the court lady Sei Shanoagon to Empress Consort Tieshi during the 990s and early 1000s in Japan. Other notable examples include James Cook’s board journals from the eighteenth century, the diaries kept by Samuels Pepys from London in the 1660s, and the diaries written by Anne Frank and Thomas Mann – all of which represent important cultural and historical artifacts of modern society.

Drawing on our own experience of working with diary-based data in extreme contexts (e.g., see Rauch & Ansari, 2022 for a diary-based project studying military personnel), we illustrate their potential of diaries to study and advance scholarship on grand challenges and organization studies. While diary writing is known to bring psychological benefits to people coping with traumatic events (Amabile & Kramer, 2011), we showcase the value of diary writing for management studies (Balogun, Huff, & Johnson, 2003; Ohly, Sonnentag, Niessen, & Zapf, 2010). Diary studies seem particularly suitable for investigating personal accounts of people deeply affected by grand challenges, and capturing their innermost thoughts and feelings, as well as their coping strategies in the midst of extreme contexts (Farny, Kibler, & Down, 2019). We show the value of unsolicited diaries in revealing the inner world of people that cannot be captured by other modes of communication. We outline different ways of working with diaries and the advantages and disadvantages of conducting research with diaries, and provide a set of practical guiding principles by drawing on our own experiences of working with diaries-based data.

Using Diaries for Studying Grand Challenges

Despite its frequent use in private life such as in the famous examples of Anne Frank, Max Frisch, Andy Warhol, and a myriad of diary writers across the globe, the use of data from diaries in (social) science is rather rare. This is despite the few attempts to introduce diaries into social science (Alaszweski, 2006; Balogun & Johnson, 2004; Rauch & Ansari, 2022). In many disciplines, such as chemistry, pharma, and medicine, using and writing diaries are integral parts of research, as well as a means of transferring and communicating knowledge between individuals to keep track of developments (Bartlett & Milligan, 2015; Hislop, Arber, Meadows, & Venn, 2005). In comparison, social science research rarely relies on diaries as a key source of knowledge (see for exceptions Balogun & Johnson, 2004; Hannaway, 1989; Rauch, & Ansari, 2022). The use of self-reporting methods like diaries has been described as an “unconventional method” (Wickert & Schaefer, 2015, p. 124), and can serve “as a supplement to the active role of critical researchers” (Wickert & Schaefer, 2015, p. 124) in shedding new light on managerial work and the performative aspects of language. indeed, “self-reporting methods have been applied successfully [...] and appear to be a promising tool” (Wickert & Schaefer, 2015, p. 124).

Diaries vary in terms of the degree of detail but also in the way they are written, as well as regarding the motivation of people to write diaries. Diarists adopt different styles in writing about different aspects of their lives. While some diarists engage in daily writing, for example, in the morning or the evening, others engage in less frequent journaling. The diversity of diaries is also illustrated by the different forms that diaries take; some write on Microsoft Word, others use weblogs or diary-apps, a classical blank page or a book with or without a lock.

Here, we follow the definition of diaries by Alaszweski (2006, p. 2) “as a document created by an individual who has maintained a regular, personal and contemporaneous record.” In the extant literature, journaling, keeping a diary, and diarist, are treated as synonyms (Alaszweski, 2006). Diaries are emic reflections and not etic observations. Emic reflections refer to viewpoints obtained from within the social group from the subject’s insider perspective. Etic observations refer to the observer’s outsider perspective (Headland, Pike, & Harris, 1990).

We provide granularity and specificities regarding how diary data can be used to study organizational phenomena and provide the distinctions between two forms of diaries, and their advantages and disadvantages (see Table 1); that is, (1) unsolicited diaries, which are diaries that are conducted voluntarily without the observation of others; or (2) solicited diaries, which are written for trials, research projects or to cover a particular event (Milligan & Bartlett, 2019). For example, Rauch and Ansari (2022) use unsolicited diaries of military personnel working in the U.S. Air Force to study their perception of drone technology and its influence on nature and morality of their work, and the subsequent consequences it had for them and their work. Balogun et al. (2003) solicited managers to write their reflections in a diary based on their responses to five questions to track the progress of change implementations in their organizations. Others, such as Buchanan and Boddy (1992) studied the change managers’ experience of managing change by drawing on audio diaries over a two-week period. Schilit (1987) drew on diaries by middle managers to record the frequency, nature, and outcome of interactions among supervisors and themselves, in relation to strategic decisions over two months. These examples highlight the importance of diaries as a tool for research.

We zoom in on the use of unsolicited diaries, as they offer unique insights into the “thought world” of the diarist without being subject to social desirability and other biases that potentially affect solicited diaries. One type of such unsolicited diaries is also referred to as “natural diaries.” In other words, “the test is whether the interaction would have taken place in the form that it did had the researcher not been born or if the researcher had got run over on the way to the university that morning” (Potter, 1996, p. 135). Diaries allow for immediate reflections on events, experiences, and thoughts. In particular, diaries provide a more nuanced understanding of situations that evoke strong emotions. As such, unsolicited diaries can mitigate retrospective bias and socially desirable responses by providing a thick description of events in “real-time,” and how those events unfold over time (e.g., in change processes).

Studies on extreme contexts have illustrated how people “emotionally control” (de Rond & Lok, 2017; Fraher et al., 2017) their behavior in difficult situations, and are reluctant to speak about difficult experiences. One example is the study by De La Chaux et al. (2018) on maintaining social stability in Kenyan refugee camps. Fraher et al. (2017, p. 252) allude to the role of emotions in their study of U.S. Navy Seals working in Afghanistan and Iraq, arguing that

the Seal candidate is forced to compartmentalize his emotions – and not fixate on them – to provide his best effort in the moment and to not obsess over the “what-ifs” […].

The social norms of working in extreme contexts such as in the military encourage a “culture of silence” (De Rond & Lok, 2016; Fraher et al., 2017), where participants are unable to not just express or display their emotional distress but also not share their experiences with colleagues. By observing or interviewing actors in such extreme contexts, one might conclude that actors are not emotionally affected by the experienced atrocities as they appear calm and controlled in getting on with their job, preferring not to display their emotions in public (Farny et al., 2019; Jarvis et al., 2019). As such, diaries can capture the experience of individuals in a way that is not possible by using traditional designs. They permit the examination of events and experiences in their natural and spontaneous context (Reis, Erber, & Gilmore, 1994), offsetting some of the problems associated with retrospective accounts (Bower, 1981).

Empirical Examples from Diaries

To show their methodological value, we provide illustrations of how diary studies can be used to enrich qualitative studies. Potentially, diaries can be used in the same manner as sole diary studies and as a support for quantitative studies (e.g., Amabile, Barsade, Mueller, & Staw, 2005; Schilit, 1987). However, we will focus on diaries as part of qualitative analysis. We draw on one example conducted for a larger project on “organizing in extreme contexts” in which we studied how actors cope with brutal reality in such contexts. In doing so, we draw on examples from our research drawing from unsolicited diaries obtained from medical personnel working for Doctors Without Borders (MSF).

In the case of MSF, we analyzed a set of over 70 diaries of medical personnel ranging from nurses, doctors, and surgeons from different parts of the world with diverse backgrounds, gender and sexual orientations. The organization did not prescribe its personnel to write diaries. Instead MSF personnel engaged voluntarily in journaling. We interviewed all diarists (besides one) after we read through the diaries several times. In addition, one author joined several missions to extreme contexts such as Afghanistan, South Sudan, and Yemen to observe work in extreme contexts and get a feel of the context. This was done in light of our research interest in the realm of grand challenges, in contexts that scholars have described as people being “silent” or numb when it comes to the expression of their emotional experiences. In these situations, actors have deployed coping strategies when they are unable to openly voice, express, and display their emotional distress.

Our observations regarding emotional experiences from field trips align with the findings of studies of contexts where actors prefer to remain silent rather than display their emotions and articulate their experience. Actors cope in different ways when faced with trauma, such as black humor, cynicism, planting vegetables, or engaging in substance abuse. By analyzing the diaries, we were able to observe that actors are actually not emotionally numb or in control of their emotions as it may appear from observing their behaviors in the field. Instead, they are highly affected emotionally by the experience on the ground but simply don’t wear them on their sleeve. As such, a surgeon illustrates in his diary his experience working in an MSF clinic in Afghanistan:

I started two days ago and I remember he [Youssuf] was one of my first ones. I can’t describe what I have done. I know the medical procedure but in reality, what I did. I cannot. I should be able to but I cannot. I had no other option. Had the inner urge to puke. […] Youssuf was one of my longest patients. He had a severe infection. […] No surprise he had a severe infection. Every double here without infection is a medical miracle. Prosthetics in plastic bags. No wheelchair. No doubt no wheelchair. What do I even say? I am happy if we have enough pain pills to hand them […]. We have set him up with new prosthetics today. It was Christmas and birthday all in one. It is the best of all feelings here. It is the only feeling worth describing here. (Diary 56)

In a follow-up interview with this surgeon, we probed further into this diary entry in relation to the patient which he called Youssuf:

Of course I have never talked to anybody in the field about my sentiments towards Youssuf. Talking isn’t a way of solving anybody's problem in the field. We all do the work and we all need to find ways to come to terms with it. But to your question. No, I have never articulated or share this with anybody. We all performed procedures together but we have never talked about it from a human empathic point of view. We kept it always to the medical point of view. For me writing a diary which I still do to his very point was a way to find a space for myself but also a space to share my thoughts, both negative and positive ones. And I can tell you in the field there are more negative ones. (Interview 56)

Looking at diaries in relationship to the interviews allowed us to access the inner thoughts, reflections, and feelings of the actors in the field. By looking at these diaries, we reveal how medical professionals are highly affected emotionally by what is happening in the field in contrast to what one may conclude from observing their behaviors in the field where they may appear as “cold rational agents.” We could also observe how actors described the need to change their way of “seeing” and “perceiving reality.” They described such a change as necessary “to survive” and cope with the traumatic reality at hand. A doctor in a field trip to Yemen reflected on the situation of civilians and the lack of access to basic food and medical supplies amidst the ongoing war:

Once starvation was on the table and starvation was used as a weapon and tool of war, things radically changed for me. One needs to change to stay sane. No way to rationalize and find a justification why a human being would use and try to eradicate other humans in such a manner and cruel way […]. At this point I learned the hard way to look the other way. A different way! Which way this is I am still figuring out. For now, I just know it can't be the old way. (Diary 70)

Despite the need to cope with the difficult situations on the ground, and “look the other way,” we identified a “culture of silence” (De Rond & Lok, 2016), where people do not share nor openly talk about their difficult experiences and emotional distress. Rather, they express themselves through writing personal diaries that reveal the internal conflicts they experience and the mechanisms they use to cope with the conflict. They engage in this form of “self-censorship” (Edmondson, 1999), not out of fear that speaking up will lead to being penalized by their superiors or concerns about personal consequences (e.g., Morrison, 2011). Rather they do so, as a coping mechanism to deal with the emotional distress they experience in extreme situations and for being able to “get the job done.” Speaking about emotional distress and sharing feelings even with colleagues only made matters worse, as in many cases, it led to emotional breakdowns.

Overall, looking at the diaries while not being a researcher in situ, allows us to gain an insider’s understanding of their daily experiences and a peek into their innermost thoughts and feelings unobservable elsewhere.

Methodological Guidance

In the excerpts we showed, we have used the example of emotions illustrating the power of diaries. The use of diaries is not limited to the aforementioned concepts and ideas. Studying diaries can be also used for concepts such as cognition, sensemaking, and framing but also for studying identity, and interactions with technology. We see strong potential in using diaries revealing other aspects of organizational life and changes in the nature of work. For example, in following organizational norms, procedures, and rules, there may be a stark difference between the emotions experienced and the behavior displayed in which people may not express their innermost doubts and cynicism.

We provide practical guidance informed by our own experience, and some important questions to be raised when engaging with diaries (see Table 2). We have characterized them alongside steps in empirical research which may vary depending on the scope of projects.

Conclusion and Implications

In this article, we illustrate the importance of diaries in studying people’s inner thought worlds. Diaries are an important example of the use of self-reporting methods, and part of “unconventional method” in conducting research (Wickert & Schaefer, 2015). Their innermost thoughts and feelings that they write about in solitude cannot be gleaned from other sources of data such as interview data or field observations. For this purpose, we drew on an empirical example building on a qualitative study that we conducted in extreme contexts based primarily on data from diaries. This was paired with the support of interview and observational data, in which we illustrate the power of diaries in studying the phenomenon of how people cope with emotional distress in extreme contexts, and other key concepts relevant to organizational phenomena. In addition, we offer a set of practical guidelines for scholars intending to work with diaries.

We address how diaries help to understand that emotions may be strategically displayed or purposefully withheld. By focusing on diaries, a researcher can capture some of these innermost thoughts and feelings, immediate reflections, and the sensemaking process in these situations. Drawing on diaries allows us to better understand the “thoughts, feelings, considerations and reactions”; and be able to “capture these events as they happen to avoid the problems associated with retrospect” (Radcliffe, 2017, p. 190). Gaining insights into these experiences and infer the underlying meanings not readily discernable from observed behaviors and interactions requires the use of “creative methodologies” (Gray, Purdy, & Ansari, 2015). While previous methodological advancement has been built on new tools such as analyzing video data and multimodal communication more broadly, diary studies help us to look on the “backstage” of the “backstage” (Goffman, 1959). This also echoes with calls for innovations in qualitative methods in business ethics, corporate responsibility, and sustainability research, as “business ethicists cannot afford to ignore under-researched topics of great ethical import because reliable data is hard to obtain” (Reinecke, Arnold, & Palazzo, 2016, p. 14).

We show the importance of analyzing diary writing as a “technology of the self,” where one writes to and for oneself to work upon oneself (Foucault, 1982) that enables a “deep analysis of individuals internal processes and practices” (Radcliffe, 2018) – which are otherwise unobservable. While diaries have long been neglected in organization studies (Alaszweski, 2006; Bartlett & Milligan, 2015), the use of writing about the self was even appropriated by the church in the practice of confession. Many famous examples such as the diaries by Anne Frank have shown how expressive writing allows people to share their innermost feelings and thoughts with proven psychological benefits (Amabile & Kramer, 2011). Drawing on diaries allows us to infer the underlying meanings not readily discernible from observed behaviors and interactions, or interviews. By drawing on diaries, we show the value of a novel methodology in understanding how people feel when practicing emotional control in difficult circumstances such as in extreme contexts.

A second contribution is to provide a systematic overview of the potential of using diaries to study people engaged in the fight against grand challenges and organization studies more generally. We illustrate the benefits of drawing on diaries as a data source as well as some of the challenges and drawbacks. Furthermore, we offer an alternative to the existing portfolio of data sources in studying grand challenges (Schoeneborn, Vásquez, & Cornelissen, 2021: Stjerne, Wenzel, & Svejenova, 2021). The surge and emphasis to study grand challenges and unconventional events oftentimes goes hand in hand with travel to extreme settings and conduct unsettling fieldwork including strong ethical implications (Wood, 2006), such as refugee camps and the aftermath of earthquakes. As such, work entails both physical and psychological consequences and poses risks to scholars, working with diaries written in situ by people directly affected reduces the level of risk and danger. Furthermore, diaries allow for an investigation which in particular is suited for longitudinal understandings of phenomena and hence ideal for process studies (Langley, 1999) due to the temporal nature of diary writing.

Our article has two limitations. First, we focused on unsolicited diaries given our experience of working with these kinds of diaries. Future research should extend our methodology by developing procedures for capturing and analyzing data from solicited diaries. We specifically call for further methodological development grounded in empirical analysis that may find additional and more nuanced ways to gain insights from diaries. Second, we did not aim to provide a methodological guide to coding data from diaries but instead simply offer some guiding principles when engaging with diaries. In other words, we provide a platform for conversation about how diaries are a novel way to gain insights into people’s thoughts and feelings that are hard to access through other methodological tools.