1 Introduction

Following a sudden-onset disaster, crisis-affected populations now often remain connected. People will begin calling, texting, emailing or using social media to connect with their family, friends, governments and aid organizations. Among these communications are the most important pieces of information for recovery: updates about the changing situation; the locations of make-shift aid stations; people requiring immediate medical attention; a child alone. Finding this information is vital to relief and recovery efforts—it is difficult to imagine any information processing task that is more important or more time-critical.

Humanitarian information processing gained a new direction in the wake of the January 12, 2010 earthquake in Haiti. With most cell-towers intact, the nation remained connected and millions of people were communicating with the world from across the country. Across the world, millions in the Haitian diaspora were online, looking for information about loved ones and for ways to help in any way that they could. Mission 4636 was an initiative that allowed anyone within Haiti to send a text-message for free to a phone number, ‘4636’. The messages, in plain language, were translated, categorized and mapped by Kreyòl and French-speaking volunteers worldwide via online crowdsourcing platforms. 80,000 messages were processed (about 10 novels in length) with 45,000 relevant unique structured reports streamed back to responders on the ground. The median turnaround time from a message being received to it being translated, categorized, geo-located and streamed back to the responders within Haiti was less than 5 minutes.

Mission 4636 is still the largest humanitarian information processing initiative of its kind to-date and in just under 2 years it is already widely cited as the beginning of a new information processing strategy for humanitarian information processing.

1.1 Communications in Haiti following the earthquake

While crowdsourcing efforts like Mission 4636 are a new strategy for humanitarian information processing, it is a mistake to think that humanitarian crowdsourcing, implemented properly, builds on new methods of communication or new technologies. The media focus on technology following the earthquake was largely around new internet-enabled technologies and social media, but the most important communication technology for the crisis-affected population was actually radio. This is still apparent in a handful of media articles that reported the importance of radio, although often published as if this fact was a surprising footnote alongside the much greater focus on social media.

One of the most important radio stations following the earthquake was Signal FM, with the owner/manager, Mario Viau, taking over the airways in the immediate aftermath of the earthquake, with a large number of announcers sharing information around the clock, collecting it by hand-delivery, SMS, and (when the phone networks stabilized) direct phone calls (Lush 2010, Watson 2010):

“[We told the listeners] this is where you can find water … that is where you can find food … these are the hospitals that are open.” Mario Viau (Lush 2010)

Although the article above reports that Signal FM was the only radio station running in the days following the earthquake (Watson 2010), this was not the case. Another example is the Radyo Pa Nou station, whose general manager, Jeffrey Joseph, spoke about disseminating missing person information:

“They ride to the neighborhood or, if they know someone that’s closer to that neighborhood, they’ll contact someone from there … If they give us [back] a long list of 20 to 30 people, we’ll say ‘The Joseph family calling about your two sons, they’re alive.’ And then we’ll go from there, on and on.” Jeffrey Joseph (Kaufman 2010)

This account might seem like simple common sense, but by looking at it closely it is clear that it exceeds the information processing capacity of any of the large relief agencies. Based on their local knowledge, the people working at Radyo Pa Nou immediately knew whether the best form of communication was direct or remote; they were able to immediately identify the individuals at a location that they should contact directly for more information; and they were able to iterate on this process, filtering the information shared based on the volume and content. This builds on a lot of implicit knowledge about the conditions on the ground: the people who are available to be contacted for information; the people who will be reliable sources of information; the roads that are passable/uncongested; the potential security implications of identifying people at given locations; and, most importantly but most easily overlooked, the linguistic knowledge to take in and share this information in Haitian Kreyòl. This is the kind of information that is vital for large-scale information sharing initiatives, and yet it would be completely opaque to the overwhelming majority of international relief workers entering the country. Of all the radio stations, Radyo Pa Nou was singled out as an example here for one reason: it is based in Brooklyn, New York, more than 2,000 kilometers from Haiti. This wealth of local knowledge was therefore not necessarily, itself, local to the crisis.

In their recent report about communicating with the crisis-affected community in Haiti, Wall and Che´ry found that information was among the most important requirements for the crisis-affected population and closely linked to operational success (Wall and Chéry 2011). They emphasized the role of local media, radio in particular, and highlighted the emergence of the diaspora as a key ‘information player’ in the response.

1.2 Engaging the diaspora

Following the earthquake, thousands of people from among the Haitian diaspora were exchanging information, with each other and with those within the crisis-affected regions of Haiti. They were looking for the best ways to bootstrap their own recovery and to support the international relief organizations arriving in their country. Mission 4636 was established with the goal of leveraging this wealth of local knowledge, enabling an interface between the crisis-affected population in Haiti and the international relief efforts, engaging those with vital local knowledge wherever they happened to be in the world.

This is the first report about Mission 4636 from those who ran it. There have been reviews and reports about parts of the system, especially those most closely associated with international organizations (Chan and Crowley 2011; Heinzelman and Waters 2010; Hester et al. 2010; Morrow et al. 2011; Nelson et al. 2010; Roberts and Payne 2011; Tapscott and Williams 2010). Overwhelmingly, the reports so far have credited much of the success of Mission 4636 to international organizations. This is not the conclusion reached here. At the time, it was never clear that international organizations were playing a significant role in Mission 4636 (or the crowdsourcing aspects of the response efforts more broadly) and subsequent reports to the contrary have been surprising. Close analysis of the information processed (volume, timing, accuracy) and the processing strategies and workforces (collaboration, Haitian/international) shows that the majority of work should be credited to the Haitian population, both within and outside of Haiti, and to those international workers who collaborated most closely with them. In fact, one member of the Haitian diaspora, Fred Michel, a graphic designer in Montreal, processed more information than any single (non-Haitian) organization, and among the organizations the one that processed the most was FATEM, based in Mirebalais, Haiti. The broader conclusion is that while humanitarian organizations should increasingly adopt microtasking as an information processing strategy, and that there are still roles for international workers, relief organizations need to look within the crisis-affected populations for the core workforces.

One motivation for this paper is to demonstrate that workers from within a crisis-affected community are the ideal workforce for processing information about that crisis. Of all the reviews of crowdsourcing for humanitarian purposes to date, none have actually conducted an empirical analysis of data quality. While crowdsourcing itself is new to information processing more generally (Alonso et al. 2008; Carvalho et al. 2010; Grady and Lease 2010; Kelly 2009) the methods for analyzing the data here are well-known: accuracy, speed and volume of throughput (Baeza-Yates and Ribeiro-Neto 1999; Jones et al. 2002). It is also hoped that the exposure to this kind of problem will motivate more information retrieval professionals to take part in humanitarian-focused work, as the amount of unstructured information coming out of crisis-affected regions will only grow. In turn, it is hoped these knew capabilities will lead those in the humanitarian world to adopt more robust methods for evaluating information processing strategies.

In the particular context of crowdsourcing, managing and motivating workers in humanitarian contexts is more complex and less well-understood than for commercial and/or non-time-critical tasks. Management, motivation and ensuring accuracy all become more complicated, and particular attention is paid to how collaboration between workers improves all of these.

1.3 Outline

Section 2

outlines the different crowdsourcing initiatives that were deployed following the earthquake in Haiti.

Section 3

discusses the benefits of remote translation, the biggest component of Mission 4636 in terms of workload, but not the focus of this article.

Section 4

looks at the volume of information processing over time by the different crowdsourced workforces.

Section 5

looks at the categorization tasks, estimating accuracy through inter-annotator agreement.

Section 6

looks at the geo-location of messages, comparing the accuracy and speed of those with local knowledge to those from purely international workforces.

Section 7

focusses on the collaborative nature of Mission 4636, with volunteers interacting online to share information and collectively arrive at the correct translations, locations and categories for the messages they were processing.

Section 8

provides a critical analysis of the role of social media and the potential role of international actors in humanitarian crowdsourcing initiatives. It debunks the myth that Mission 4636 ran on ‘new technologies’ and discusses the trade-off between running an information service and raising expectations. It further discusses information as a form of aid, the motivation and management of volunteers, and critically discusses the findings of the article in relation to previous reports about Mission 4636 and crowdsourcing in Haiti more broadly.

Section 9

concludes the article and offers recommendations for people considering the use of crowdsourcing for humanitarian information processing.

2 Crowdsourcing for humanitarian information processing in Haiti

Large-scale crowdsourcing Footnote 1 is a relatively new strategy in the humanitarian world but is being deployed with increasing frequency. Figure 1 shows that many reports have been processed by crowdsourced workers in the past 18 months, across much of the world. Mission 4636, the focus of this report, was established by Haitian and international aid workers/volunteers following the earthquake before being transferred to control by organizations within Haiti. Pakreport was established by Pakistani citizens following the floods in 2010 that left millions homeless. Sinsai was established by Japanese citizens after the earthquake and subsequent tsunami and nuclear reactor crises. Christchurch Recovery Map was established by New Zealand citizens following the earthquake in Christchurch. Alabama Recovery Map and Oil Spill Crisis Map were established by citizens from the United States following the tornadoes and BP oil-spill. Libya Crisis Map was established by the United Nations Office for the Coordination of Humanitarian Affairs (UN OCHA), following the civil unrest in the country.

Fig. 1
figure 1

Deployments of crisis data collection and processing, between January 2010 and June 2011, showing reports processed by crowdsourced workers. The deployments total almost 100,000 reports globally. While there are a large number and geographic variety of deployments, and the author involved in most of these to some extent, Mission 4636 is by far the largest and therefore the focus here. The numbers here represent structured reports. For example, Mission 4636 created 45,467 reports from the 80,000 messages that were received, the remainder being filtered as duplicates, messages spanning multiple text messages, or considered irrelevant (a subset of messages were about the weather, a previous use for the ‘4636’ short code)

As Fig. 1 makes clear, Mission 4636 was by far the largest of these to date. For this reason it is the primary focus of this report. While all these initiatives, Mission 4636 included, were small parts of the relief operations it is reasonable to predict that crowdsourcing will become an increasingly well-used tool for information processing among humanitarian organizations.

2.1 Mission 4636

In the wake of the January 12 earthquake local emergency response services were inoperable but 70–80 % of cell-towers were quickly restored. With 83/67 % of men/women possessing a cellphone, the nation remained largely connected. People within Haiti were texting, calling and interacting with social media, primarily in Haitian Kreyòl (Munro 2011). A number of people came together quickly to establish an SMS-based reporting service within Haiti. ‘Mission 4636’ was the blanket name given to 1000s of individuals and more than a dozen organizations that contributed to the ‘4636’ response effort. The number of international actors was very small. Footnote 2 Only four people in the international aid community worked full-time from conception to the pass-off to local partners within Haiti, and on the core platform only the author of this paper.

The Mission 4636 system was first conceived of as a way of gathering and disseminating missing person data, reaching people via text messaging, the most dominant form of remote communication, which, unlike direct calls, was more robust on overloaded phone networks (Fig. 2).

Fig. 2
figure 2

Mission 4636 workflow. The five processes in or partially in Haiti represent the main workflow. A person in Haiti sends a message to 4636; that message is streamed to the main 4636 platform where a volunteer (green) or a paid worker within Haiti (orange) translates, categorizes and geolocates the messages, and extracts missing person information; the structured information is then streamed to the information managers within the United States Southern Command who were based in various locations within Haiti or off the coast or at a command center in Miami, who in turn communicated with those who were able to respond directly to actionable reports within Haiti or to those monitoring for more general situational awareness. The size of the boxes indicates the volume of information processed. Dashed lines on boxes indicate that the volume is unknown (Color figure online)

Understanding that Mission 4636 could not completely control the type of information that people would send, and that the organization had a responsibility for any information that passed through our system, the administrators contacted the ‘114’ emergency reporting service in Haiti to find they that were essentially inoperable in the wake of the earthquake. We instead reached out to contacts within the response efforts, arriving at the US Military as the potential responders to the messages, via contacts in the US State Department who were a Mission 4636 partner and would later come to fund the paid workers in Haiti.

The SMS-protocol does not encode the location of the sender and most people among the international relief efforts did not speak Kreyòl or have time to sort through potentially thousands of messages. Therefore, it was quickly decided that online Kreyòl-speaking volunteers could translate and structure the messages in real-time on a microtasking platform. A makeshift platform was adapted from the HaitianQuake missing person’s platform (see below) that allowed volunteers to read one message at a time and translate, categorize, map and enter missing person information (Table 1).

Table 1 Example messages, with translation, actionable flag (a specialized category), latitude and longitude, location name and categories

From conception to launch was just 48 hours. The number was publicized within Haiti via radio and word of mouth. The task, for a volunteer or worker, was very simple. A person would see a web form with an unstructured text message and fields for translation, categories, and any additional notes. For messages with an identifiable location, the person could also click on an embedded map to generate the longitude and latitude. In the first weeks there were also fields to extract the names of people for missing person records, and further categories to indicate whether they were missing, found or deceased. From this single form, unstructured messages in Haitian Kreyòl were turned into structured, geolocated reports in English. The messages were streamed back to various response organizations via an api-key-protected geo-rss data feed. Because the messages contained personally identifying information, we did not publish them online, and tried to recruit volunteers as directly as possible, avoiding public call-outs for help.

The core management and information sharing interface was a simple online chat-room embedded in the same site as the task, allowing the exchange of information between people working on the tasks. Some examples of interactions are given in Tables 2 and 3, and the nature of the collaboration is the focus of Section 7.

Table 2 Example interactions from the online chat-room
Table 3 Additional examples of chat-room interactions with people collaborating to exchange information and create structured reports

After being recruited into the initiative, primarily through social media, around 2,000 Kreyòl and French-speaking volunteers collaborated to structure and translate messages. In the first 2 weeks there were volunteers from 49 countries (see Table 4). The majority of volunteers worked (physically) alone or in small groups.

Table 4 The 49 countries/regions from which the online volunteers collaborated

After the first week, Mission 4636 was receiving more than 1,000 messages per day. Within a week the US Military were asking for an increase in emergency messages as they had a greater capacity to respond to specific ground reports. They defined the types of reports that were to be considered ‘actionable’, that is, that they had a mandate to respond to, which included requests for water, aid, time-critical medical emergencies, logistical support for hospitals and clinics, and reports of unaccompanied minors. Although we worked with a number of relief organizations, the US Military became the main responders for the emergencies sent to 4636 and credit it with saving 100s of lives and helping deliver the first aid to 1,000s of people among the Haitian population.

By the third week, the microtasking service was transitioned to CrowdFlower, a commercial microtasking platform. Their partner, Samasource, a non-profit that brings microtasking work to women, youth, and refugees, had been training over a hundred workers in Haiti. Samasource had signed an agreement with workers via the 1,000 Jobs/Haiti and FATEM organizations in Mirebalais, Haiti, just 20 minutes prior to the earthquake. They had anticipated delaying the establishment of the center but by partnering with Mission 4636 they were able to expedite its construction. The workers in Mirebalais gradually took over from the volunteers in the sixth week following the earthquake.

It is unlikely and hopefully unique that an emergency response service will need to be rebuilt from scratch in exactly this manner. However, the greatest impact of Mission 4636 to the relief agencies was not in targeted medical response, but in directing aid to large populations. The media has focussed on the search and rescue operations, but Mission 4636 did not play a big role in these, with most medical emergencies reported to Mission 4636 not being directly related to the earthquake itself (a car crash, a falling wall, etc), and even these emergencies were low in frequency. The most lives saved were in less dramatic but arguably larger ways: directing aid to populations with particularly at-risk people like infants, pregnant women, and the infirm, and providing supplies to hospitals and clinics. The most common response type from messages to Mission 4636 (from what we observed) was to deliver aid and supplies to people and organizations outside of the capital, Port-au-Prince, complementing the work of other big response organizations like the United Nations and Red Cross which were centered in the capital at the airport where the main logistics base was established to receive and distribute supplies.

One example of internally displaced people requiring aid is the final example message Table 1, referring to Morija Diquini 63, where the person sending the message reports that their district had received no aid (it was one of three around the same time from the area from different numbers). The area is poorly accessible by road—just hours before the earthquake on January 12th, an organization called CHF International had posted a video on YouTube outlining a not-yet-complete project to improve the road (CHF International: Haiti−Route 2010). Near Carrefour, it was one of the worst affected places, on the far side of Port-au-Prince from the airport. Combined with the already poor condition of much of the road, this made it one of many places that were difficult to reach by relief workers on the ground early on, for evaluation let alone the delivery of aid. It was a small part of the response to simply allow them to say ‘we are here’ and get this to the responders in a structured report (the World Food Program made the crucial aid delivery and reported there were about 2500 people) but it was the best we could do for the very large populations who were otherwise cut-off.

At a higher level, all the reports contributed to the situational awareness of the responders, helping track the changing conditions as populations moved internally. Tracking the locations, vulnerabilities and needs of refugees and internally displaced persons (IDPs) is crucial to planning aid operations in many contexts, as is getting targeted aid and information to those populations. So while the specific circumstances were unique, the potential application is much more broad (Fig. 3).

Fig. 3
figure 3

A comparison of the processing volume over time by different workforces and initiatives. It shows that the second and third weeks were the busiest, and that while non-SMS sources of processing tapered to near-zero very quickly, the 4636 shortcode received a steady stream over an extended period of time. The approximate breakdown of work between the four workforces was: the online volunteers for 4636 contributed around 60 % of all processing, the paid 4636 workers in Haiti around 25 %, and the Haitian Quake and Ushahidi Haiti volunteers a little under 10 % each. The y-axis corresponds to the work-load ratios, as outlined in the body of the paper. The ratios vary according to type of work, but each interval in the graph approximates a little over 1,000 reports

The most important aspect of Mission 4636, in terms of what can be learned and applied to future disaster response initiatives is not the specific nature of the response, but the ability to engage distributed workforces with crucial local knowledge, supporting information sharing within the crisis-affected population as much as the international relief efforts. As this paper argues, the most important people to engage are from within the crisis-affected community – a knowledgeable workforce that will always be looking for ways to help.

2.1.1 HaitianQuake

A software developer, Tim Schwartz, quickly developed and launched a missing person’s platform called ‘haitianquake.com’ shortly following the earthquake, which was later folded into Google’s Person Finder missing person’s database. Schwartz continued working on missing person’s information, building a microtasking platform that enable people to create/update Person Finder records from unstructured reports.

A person working on this task would see a split screen with an unstructured report (blog comment, text message, etc.) on the left and the Person Finder plugin on the right. They would then search Person Finder for existing information about any person mentioned in the report and add/update Person Finder accordingly.

I also managed the workforce for this effort, seeking out English speakers who were not able to help with the core translation tasks. We had observed that people were posting information about missing people in places that were difficult to parse information from and had potential privacy implications, so it was motivated by the need to centralize this information in one location where the issues were more easily managed. Table 5 shows some of the interactions around the missing person efforts.

Table 5 Interactions from the chat-log showing the people were being encouraged to use the centralized Person Finder database of missing people rather than social media and blogs, and the launch of the microtasking service to copy existing information over

It was after seeing that so many records were being recorded as unstructured data in different places that we decided to launch the microtasking initiative to add these to Person Finder. As much as we could automate the import of blog posts, etc, it still required a human to accurately pull out people’s names and select categories like ‘missing’ or ‘found’. From this point we were then coordinating people in efforts to add/update records from sources like CNN’s iReport and social media groups.

Although most of the missing person records were not from 4636 messages, many that were were from people asking to forward a message to people outside of Haiti (perhaps because of network connectivity problems, or maybe simply a lack of phone credit), most often to tell people that they were ok. In many cases, it was from people that already had missing person records. In these cases, where numbers were given and people were asking the message to be forwarded, volunteers were doing this:

“i have spoken to many families on the phone….”

                  ronny, 19 Jan 2010, 04:46:54 (chat-log:1642)

“Every family sans one have found their loved ones”

                  ronny, 19 Jan 2010, 04:47:18 (chat-log:1643)

This shows one potential use for crowdsourced workers that goes beyond information processing: bridging communication divides. As a work practice, it is similar to volunteers in aid organizations who directly interact with people in order to elicit donations, but in this case it is an exchange of information.

For the missing persons platform, developed with more time than was available for the 4636 platform, there was time to more carefully capture details about the people joining. Figure 4 gives the self-reported affiliations of those that registered to help. More than three-quarters gave no affiliation or explicitly stated that they were unaffiliated—it was truly an initiative of individuals.

Fig. 4
figure 4

The strength of the unaffiliated. This figure shows the self-reported affiliations of the volunteers who registered to help with the missing person platform; at the time a core part of Mission 4636. It shows that most of the volunteers were individuals that had no previous affiliation to aid agencies or NGOs. Most people chose to leave the ‘affiliation’ field blank (‘None given’), and among those that added a value to this field, the majority explicitly stated that they were individuals with no formal affiliation. The most common affiliations were Universities, with most of these from MIT (thanks to a close involvement in rallying volunteers for the missing person’s efforts by a student there, Christina Xu, and faculty, Chris Csikszentmihályi) and then tech, mostly bigger Silicon Valley-based organizations like Facebook and Google. While Crisis Commons were the most well-publicized volunteer coordination initiative at the time, and the single-most cited organization as an affiliation here, only 3 % of volunteer listed themselves as affiliated with Crisis Commons or having learned about the initiative through the ‘Crisis Camps’. This was clearly an initiative of individuals

Of all the crowdsourcing efforts in Haiti, the missing persons initiative has been studied the least but is probably the most transferable to other crises. It deserves to be studied in greater detail.

2.1.2 Ushahidi Haiti

The third and smallest crowdsourcing initiative within the broader Mission 4636 collaboration was run by students in Boston, primarily from Tufts University. They used software from an NGO called ‘Ushahidi’, a blogging organization that produces a content management system with the same name. It differentiates itself from other content management systems by enforcing the geolocation of reports. That is, it is not possible to add structured data about reports unless a latitude and longitude are also present. The primary interface to data is an interactive map with the reports on it. This initially independent effort was started by students in Boston, who came together to start aggregating and mapping information from all available sources: social media, traditional media, radio reports, email, and web-based reports.

Like the other two initiatives, Ushahidi Haiti Footnote 3 was also first launched as a missing person platform, but in this case for international people in Haiti. Two students from Tufts University were in Haiti at the time of the earthquake as interns for an NGO called Digital Democracy, and so their colleagues at Tufts started collecting data that might be relevant to their whereabouts.

At the start of the second week, they became connected to the rest of the collaboration, taking in the structured reports from the microtasking platform. Using the translations of the messages, they confirmed/refined the coordinates and categories and they also worked directly with the responders to identify actionable items. They would process and publish about 3,000 of the 80,000 messages by the time their part of the initiative wound down a few weeks later.

In their case, most of the data structuring actually took place on Google Spreadsheets (p.c.), with the reports manually copied to the Ushahidi system. The workflow let people flag the reports they were processing on the forms, so with a generous definition it could be thought of as a form of microtasking, but with manual workflows and quality controls.

Reports made on the website did go directly into the system. When Mission 4636 volunteers had information sources other than the text messages, they were instructed to use the Ushahidi website:

“Thats what we do we monitor the haiti radio, facebook feeds and twitter frome all our contacts. filter it and redistribut it.”

              UNIONHAITI, 18 Jan 2010, 02:46:34 (chat-log:182)

“alot of us got back from haiti after the summer break, and grew up in pap”

              UNIONHAITI, 18 Jan 2010, 02:47:02 (chat-log:183)

“it sounds like you are doing a lot of great work”

              Robert_Munro, 18 Jan 2010, 02:49:14 (chat-log:184)

“where are you redistributing the information you find?”

              Robert_Munro, 18 Jan 2010, 02:49:26 (chat-log:185)

“our twitter and facebook group. we also have a web site up unionhaiti.org but it is not fully functional yet.”

              UNIONHAITI, 18 Jan 2010, 02:50:24 (chat-log:186)

“in case of emergency we also have a few contacts on the ground in haiti”

              UNIONHAITI, 18 Jan 2010, 02:50:52 (chat-log:187)

“all the inforamtion we post has been confirmed to the best of our ability”

              UNIONHAITI, 18 Jan 2010, 02:51:49 (chat-log:188)

“UnionHaiti - if you are able to get locations about any of the information you gather you can also post it on the main Ushahidi site:”

              Robert_Munro, 18 Jan 2010, 02:53:43 (chat-log:197)

http://haiti.ushahidi.com/reports/submit

              Robert_Munro, 18 Jan 2010, 02:53:45 (chat-log:198)

As a result, the Ushahidi platform became one way for Mission 4636 workers to share information with various parties:

“I have a good news update for you guys”

              Jennifer, 20 Jan 2010, 02:26:35 (chat-log:2272)

“This is from the clinic that my friend is operating in on the ground: just received an email to put the diesel need on a map at http://haiti.ushahidi.com/main Then I got a call from [NAME] to see if the need was legit.He is an emergency manager supporting the Haiti response. He just sent the deisel request to [NAME], the senior FEMA advisor, who will contact the State Department. They will contact the military and the diesel should be on its way.”

            Jennifer, 20 Jan 2010, 02:27:24 (chat-log:2274)

“They were desperate for fuel yesterday and very grateful for the fast response they got back once entering their clinic location and need on the map”

            Jennifer, 20 Jan 2010, 02:28:16 (chat-log:2277)

“they are seeing some horrific injuries”

            Jennifer, 20 Jan 2010, 02:31:16 (chat-log:2280)

“the response really helped to lift them up yesterday”

            Jennifer, 20 Jan 2010, 02:31:48 (chat-log:2281)

This shows the importance of the two-way communications between those within the crisis-affected region and those outside. Beyond the actual action taken it was a morale boost for people working on Mission 4636 to know that they were helping, and also a morale boost for people in Haiti to know that they had support from outside the country:

“Just got an update from the clinic - “We got 10 of our more critical patients transfered to the Comfort Ship for major work that we are not able to do. Hope we can continue that relationship.” ”

            Jennifer, 21 Jan 2010, 01:55:52 (chat-log:3603)

“The help they’ve been getting on the ground has really meant the world to them, so thank you”

            Jennifer, 21 Jan 2010, 01:56:38 (chat-log:3605)

The above example might also make it understandable why the Ushahidi Haiti initiative were erroneously attributing credit for response to their map (CNN: Coordinating Haiti 2010). From their point of view, someone called Jennifer posted the need for fuel on their map and Ushahidi later learned that diesel was delivered to meet that need. In reality, it was just a convenient centralized location to quickly store information. Footnote 4 From what we observed at the time and have been able to discover since, this seems to be the most typical use case for the public map: people were passing information through conventional means and used the platform as a convenient place to store the information. Rather than being part of the critical path for response, it might be better to think of Ushahidi Haiti as a window on other people’s information going by. The students in Boston certainly deserve recognition for establishing the platform that was used as a convenient place to store non-4636 data, although it is argued here that this kind of information would be better stored on a highly redundant and secure platform, rather than an open database.

One of the reasons that Ushahidi Haiti was the most independent of the different initiatives was because of one of the few major sources of disagreement between parties: the decision to publish the 4636 messages online. As stated above, Mission 4636 decided not to publish the messages on a public platform. When Ushahidi Haiti started to add 4636 messages in the second week we asked them to do the same and post only the categories and locations online. As we argued at the time, there was no need to post the messages online as they were already being streamed directly to the responders. Ushahidi declined this request, withholding the return phone numbers but publishing the messages themselves, and personal identifiers within, including names and typed phone numbers.

Perhaps because of the disagreement about whether to publish the messages, Tufts were the only group of students that remained separate to Mission 4636, with Tufts/Ushahidi’s Patrick Meier, the spokesperson of the group of students in Boston, declining an invitation for Ushahidi Haiti to become part of the main crowdsourcing effort. Although this study concludes that this was probably detrimental to their part of the initiative, the separation allows the comparison of the two systems for speed, volume and accuracy in processing. As a result, it is possible to investigate the difference between information processing within the crisis-affected population and by a group consisting only of international individuals.

Despite the false-start at the organizational level regarding the publication of text messages, we were able to form a strong relationship with the students running the activities at Tufts. A member of Mission 4636 visited them early on:

“Hey, I visited the Boston HQ, it was cool”

                Giscard, 24 Jan 2010, 20:31:21 (chat-log:37339)

“I heard! I was talking to Denise online when you were visitng. it sounds like they’ve got a great group of people there”

                Robert_Munro, 24 Jan 2010, 20:31:32-48 (chat-log:37341,3,5)

“That was 45 golden minutes. I don’t regret paying them the visit.”

                Giscard, 24 Jan 2010, 20:32:31 (chat-log:37347)

Therefore, despite the strained relationship with the Ushahidi corporation, we were able to establish good relationships with the students at Tufts doing the work. The students, unable to speak Kreyol, were reliant on the translation component of Mission 4636 to further structure the data. In return, the members of the Haitian diaspora were extremely grateful that the students were helping.

As the semester was beginning at Tufts around the same time, and the students in Haiti had safely returned, this initiative quickly wound down. Footnote 5

3 Translation

No professional translators of Haitian Kreyòl were sitting idle following the earthquake and translation resources among all aid organizations were stretched, so the ability to tap into an online workforce was vital (Munro 2010). As Fig. 5 shows, translation was the main task by overall workload, accounting for about half of all work.

Fig. 5
figure 5

A breakdown of the type of work completed by time: reporting to Mission 4636, categorizing the messages, translating the messages, creating/updating missing persons records, geo-locating the messages. The initiative is often referred to as ‘crisis-mapping’ but given the relatively low amount of time mapping took compared to the other types of work, the ‘mapping’ part refers more to the visual appeal of one interface into the data than to an account of the actual work that has gone into processing and structuring the data

There has been much debate about the potential utility of crowdsourcing in humanitarian response, but one thing that no one has argued against is the ability to translate between the language of the crisis-affected population and the relief workers. Among the response organizations, Admiral Stavridis of the US Navy highlighted the use of cloud-based (human) translators for 4636 as a positive new step in information processing (Anderson 2010).

The translators worked on tasks other than the 4636 messages themselves, as we fielded ad-hoc translation requests from people on the ground during the entire period, and directed volunteers to other tasks with response organizations:

“The US Coast Guard needs someone to jump on skype with them. Can anyone here do that?”

      Robert_Munro, 22 Jan 2010, 15:56:27-35 (chat-log:5914-5)

“yes I can”

              Ronald, 22 Jan 2010, 15:57:01 (chat-log:5917)

In the exchange above, Ronald had already proved himself to be one of the most trusted and reliable workers. In just under a minute, then, a vetted translator could be found for the US Coast Guard, at the time our main contact to the responders, for a language that was not on the radar of most the relief organizations just 10 days earlier. From this alone, it is easy to conclude that Mission 4636 was an operational success in allowing communications between the crisis-population and the relief workers.

The US Coast Guard also seemed to view translation as the most important aspect of data structuring. When the volume of messages was too great to translate all but the important ones, they immediately requested that we please try to translate them all, regardless of whether they seemed relevant. This is encouraging for people sending the messages—if you were in Haiti and sent a message to 4636, someone wanted to hear you and noticed when your message wasn’t translated. Categorization and mapping were less important. For locations, in particular, the responders would try to call back the senders to confirm the details regardless, so it was not as important as simply understanding the content in the first place.

Translation was the most novel part of the initiative. Even in research circles, crowdsourced translation is still new (Ambati et al. 2010; Zaidan and Callison-Burch 2011), with no one previously reporting the use real-time (or near-time) crowdsourced translation, in academic circles or industry, let alone for crisis response. A full account of the translation aspects of Mission 4636 is not intended to be part of this paper as it has been addressed in Munro (2010), and for the most logical extension to machine translation in Callison-Burch et al. (2011), Lewis (2010), Lewis et al. (2011).

Despite Stavridis’ comments in Anderson (2010), the translation component was not perfect (although no fault lies in the people who actually made the translations). The arguments made later about the importance of engaging a paid workforce apply equally to translation as to the other tasks, especially regarding motivation and appropriate tasks for volunteer workers. It was not difficult to motivate people to translate emergency messages from within the country or to directly help the response organizations, but a different kind of task came up when third-party NGOs wanted help translating a manual for (non-emergency) work. A volunteer who was asked to help pushed back and claimed that he would find someone in Haiti who should get paid. Their perspective was that the NGO should expect to pay for information processing services, just as they pay for transport, accommodation, and the materials they use. Regardless of whether the NGO would make a net positive impact on the country, it was wrong to assume that someone from among the crisis-affected population should work for them for free, especially as the NGO workers themselves would be drawing a salary during their work.

Similar situations arose when several people describing themselves as ‘citizen journalists’ or ‘independent documentary makers’ wanted volunteers to help transcribe and translate their self-filmed expeditions to Haiti. Their objectives were loosely described as ‘awareness raising’. In every case, the individuals expected free work and protested against paying. As Van Hoving el al. highlight, even trained medical practitioners can be ‘disaster tourists’ when their efforts are not correctly aligned with relief efforts (van Hoving et al. 2010). This is an important demographic to consider for future deployments, because it might be possible that immersive online environments that engage in response will attract the same kinds of people—‘digital disaster tourists’—who might at best drain resources or at worst be a (well-meaning) detriment to response efforts.

4 Volume of work over time

About 95 % of the workforce was from within the Haitian population. The first workforce was predominantly the Haitian diaspora who collaborated online from at least 49 countries and processed most of the reports on the core Mission 4636 system. The second group, wholly Haitian and working together from within Haiti, processed about a quarter of all reports. The third group, including many non-Haitian volunteers working globally, processed a few thousand reports about missing people on the HaitianQuake platform that allowed them to create/update the official missing person’s records (from both 4636 and non-4636 sources). The final group, predominantly students worked on the separate Ushahidi platform in Boston (and to a lesser extent New York and Washington, DC) to reprocess a few thousand reports to 4636 post-translation and also process reports from non-4636 sources. The workers on all platforms were supported by coordinators and technical staff, both Haitian and non-Haitian. The work, broken down by week, is given in Fig. 3.

In order to calculate the relative amount of work completed, the four different tasks (translation, categorization, mapping, and structuring missing person information) were divided into the respective work ratios 25:5:20:10. This was based on Samasource’s paid division of 30 cents for translation and categorization and 20 cents for each location, which they arrived at after their internal analysis of the relative work times. The 25:5 division for translation/categorization and ‘10’ for creating a missing person record are simply rounded approximations. With so much variation between tasks, this is only an approximation of the actual time spent, but adjusting the ratios (or having none) does not make a significant relative difference between the work attributed to each workforce, so none of the arguments made here rely on small differences.

The graph in Fig. 3 shows the result when the calculation for relative work is applied to all the messages and reports processed per-week. One of the leaders of the search and rescue teams in Haiti reported that the average cost per successful rescue was about $1,000,000 (p.c.). If we had needed to pay for every 4636 message to be translated, categorized, mapped and flagged as actionable/non-actionable, it would have cost $200,000–$300,000. In other words, the entire Mission 4636 eco-system (if it was paid, not largely voluntary) would probably have cost about one quarter as much as one single search and rescue success. Put another way, the responders in Haiti could have processed a report from every family in the Haiti for the additional cost equivalent of 5 search and rescue missions. Unlike the resources going to most rebuilding and reporting efforts, the cost would have stayed in the country as wages for the workers, helping the economy at a crucial time. Allocating resources to crisis-affected populations in this manner should be obvious.

Even by the most generous calculations, management and technical support comes to less than 1 % of the time committed to Mission 4636. As about 90 % of the online workforce was from the Haitian diaspora, it is safe to conclude that Mission 4636 was overwhelmingly a Haitian-run initiative.

5 Consistency in categorization

Analysis of the microtasking data shows that there were 9,170 repeated judgments for 1,971 messages with more than 100,000 pair-wise judgments to compare. CrowdFlower’s internal evaluation of the classification accuracy reported that it was not possible to calculate inter-worker agreement on the classification task as each message was only given to one worker (Hester et al. 2010). While it is true that each message was only given to one worker, there were identical tasks given to multiple workers because of three reasons. First, some messages were sent repeatedly by an individual and were processed by different workers. Second, identical messages were sometimes sent by two or more individuals. Third, one of CrowdFlower’s internal systems automatically gave some old messages to workers at times when no new messages were waiting to be processed. This third reason is a deliberate automated strategy within the CrowdFlower platform to gauge worker reliability by generating statistics for inter-annotator agreement. However, the importance of an ‘empty queue’ was judged to be more important than automated calculations for inter-annotator agreement, as an empty queue allowed volunteers a psychological and physical break. This feature was therefore only active for a short period.

Table 6 shows the results from calculating the average amount of agreement for the eight major category types. The categories were UN-defined categories, with some additions like ‘Asking to forward a message’. Most of the categories have extensive sub-types. For example, within 1. Emergency there is 1a. Collapsed structure and 1d. Contaminated water supply among others. Mistaking 1a. for 1d. would be an error, but labeling either 1a. or 1d. as 1. is a loss of specificity more than an actual error. The inter-annotator agreement was therefore also calculated where sub-type/super-type mismatches by different workers were not counted as disagreements. (The distributions between most subtypes were too sparse to get meaningful figures for inter-annotator agreement).

Table 6 Inter-annotator agreement for categories, showing very different agreement levels depending on the treatment of outliers and subtypes

There were two outlier categories: 4. Response and 5. Other. The latter became a catch-all for worker uncertainty, so it doesn’t necessarily represent inter-annotator disagreement. The former was a consistent error that simply wasn’t made clear enough in the instructions. It is meant to define that a response had been taken. For example, 4d. Food distribution would be to categorize a report of where food was being distributed by aid agencies. However, it was more commonly used to report requests for food, which should have been 3d. Food Shortage. The relatively high volume of both 4. and 5. skewed the data. As systematic errors they are easy to identify and ignore. The inter-annotator agreement is therefore also calculated with these outlier categories removed. This makes up the final column in Table 6.

The agreement remains low for 2. Threats at 55.6 %, but this is not statistically significant—there were less than 10 repeated reports in total and despite frequent rumors of rioting and escaped criminals there was no large-scale violence in the first few weeks following the earthquake. The same is true for Child Alone: there are reports about (and sometimes from) unaccompanied children, but thankfully not many.

For 3. Vital Lines, the majority of errors are disagreements between sub-types: 3a. Water shortage and 3d. Food Shortage. For most messages this is not an error – both are correct interpretations. Similarly, for 6. Missing Persons there is some overlap between the other categories, resulting in low agreement. 6. Missing Persons also changed the most over time. Initially, any report containing a person’s name was considered to be about missing persons (someone might be looking for them), but after several weeks almost all communications channels had been restored and we were no longer collected missing persons information.

The results are therefore encouraging. While there was uncertainty and limitations in the instructions, the workforce was largely in agreement on a per-category basis, suggesting a high degree of accuracy. It is probably not chance that when outliers are removed, 1. Emergency becomes the most agreed on at 94.0 %. This was clearly the category in which the greatest care was taken.

6 Accuracy and speed for identifying locations

Geolocation was one of the least time intensive information processing tasks. As described above, there were two steps in identifying locations. First, the predominantly Kreyòl-speaking workforces online and/or in Haiti plotted the locations on maps as part of the microtasking platforms they used to translate, structure and map incoming messages. For the first few weeks, the non-Kreyòl-speaking workforce at Tufts refined coordinates. The two are compared here for accuracy and speed.

A recent simulation was run by a humanitarian organization to test a new information processing platform. The simulation used about 1,000 of the Mission 4636 messages for this, attempting to remap them. Unlike the real-time structuring during the crisis, the workers in the simulation had access to much more detailed maps and were not working in as time-critical a situation. While the locations will not be perfect, they will certainly be better. For comparison, they can therefore be used to determine the relative accuracy of the mapping workforces.

For the time taken, we can simply look at the timestamps when the messages arrived compared to when they were processed. It is difficult to calculate the average time as the strategy for treating irrelevant messages changed. At some points in the initiative irrelevant messages languished in the system while at other points they were not included at all. In addition, for the initial microtasking platform, the last-modified date was recorded but not the first-modified date. Both of these skew the data towards longer times. However, they are a minority so taking the median time is fairly robust.

Table 7 reports the analysis for identifying locations. Using the average distance from the actual location, the students in Boston were slightly more accurate but using median distance from the actual location shows the Kreyòl-speaking volunteers to be slightly more accurate (although only the latter number is significant). When the students were refining the locations first mapped by the Kreyòl-speaking volunteers there was a significant increase in accuracy. This indicates that both types of location detection workforces are desirable. From looking at the data, it is clear that there were at least a few errors by Kreyòl-speakers that were corrected by the students at Tufts. In both cases, there were locations that were identified by one group but not the other.

Table 7 A comparison of work-forces for identifying locations from the messages

Looking at the timestamps, the difference is stark. Even with the additional translation task, the median time for the Kreyòl-speakers to locate messages was under 5 min. For the students working independently of the Haitian population, it was over 4 h. Where someone familiar with the landscape can easily click on an unlabeled map (see the chat-log extracts here for examples) somebody not familiar with the region might take a very long time to find a location that was not labeled on a map, expressed in slang, or a language barrier preventing the person from immediately querying the sender, then processing the message takes much longer. It is concluded that while quality control is important for mapping, employing local knowledge is vital.

For the technology component of mapping, Open Street Map was the favored mapping software (Seldon 2010), especially because of its collaborative component that meant that people could easily update it with new locations (Liu and Palen 2010). The introduction of this technology came from among the volunteers themselves:

“@Robert_Munro… looks like the openstreetmap.org will do better for us for a few reasonsso… data is more complete, accessible on the net… and there is ways we can update the map so it is customizable…”

          Sebastien, 20 Jan 2010, 23:55:42 (chat-log:3471)

“@rebecca http://www.openstreetmap.org/ the best street map so far”

             marc, 23 Jan 2010, 02:19:13 (chat-log:6783)

“Kenny, you can use the search function on http://www.openstreetmap.org . That sometimes helps”

             Danya, 23 Jan 2010, 15:44:09 (chat-log:8355)

“I’ll put [Open Street Map] as my facebook status, people”

             Fiona, 24 Jan 2010, 02:13:09 (chat-log:9890)

While the Mission 4636 volunteers contributed much to Open Street Map, it has actually become one of the few areas where the role of the diaspora was inflated. About 90 % of the edits to the map for Haiti were by existing Open Street Map users, and more than half of all edits by a small handful of people (p.c.). It was certainly a large, vital, crowdsourcing effort, but mostly as a digitization process with details added to the map from existing sources, rather than straight from the knowledge of individuals. However, the diaspora were very effective at adding labels, especially slang terms, like ‘Okap’ for ‘Cap Haitïen’, which were particularly valuable within Mission 4636 itself, as the slang terms for locations were among the most common in the messages. Footnote 6

7 Collaboration

The volunteer component of Mission 4636 centered on the collaborative chat embedded in the website, from which many excerpts have been extracted here. With around 40,000 lines, the excerpts in this article are less than half of 1 % of all interactions. There were around 1,000 people who contributed to the chat at different times, with about 75 % of interactions made up by a core group of around 100 people, and 25 % by just two people: the most prolific volunteer, Fred Michel, and myself.

While it is the hardest to quantify, the collaborative nature of the task sets it aside from many microtasking platforms where workers are in isolation and monetary incentives act against the desire to help other people with their tasks. Several conversations show non-Haitian volunteers asking questions about locations, so it is clearly a gain in location accuracy, and about ambiguous water-related terms, so it is also a gain in categorization. It isn’t possible that any one person is the expert on geography/slang in all parts of a country, so the collaboration also extends the knowledge beyond that of any one individual—so long as someone knows the correct translation/location in the online collaborating crowd, it can be identified. As a means of tapping into the wisdom of the crowd, Mission 4636 is the largest application of this kind of ‘collective intelligence’ for humanitarian purposes to date.

Just as importantly, the collaboration was frequently cited as an important motivation for contributing to Mission 4636. For the majority of volunteers, Mission 4636 gave them a way to plug themselves directly into the response effort despite their physical distance. For those working alone, it also gave them a support network while they did this (all taken from the Mission 4636 website or Facebook page):

“Since we can’t be on the ground to help, we are doing everything we can from miles away.”

“In the memories of the loved ones we have lost … we can say we are saving lives and helping people.”

“Working on Mission 4636 also means learning, joining forces and sharing knowledge with an incredible group of volunteers who maintain such an inspiring work atmosphere anytime of the day or night.”

“Even my aunts and mother pitched in to translate. My Kreyòl was rusty but we helped each other well! and it was a privilege to have helped. There was never any doubt in my mind about the importance of what we were doing.”

It is not just the immediately affected population that can benefit from crowdsourcing, it is also those closest to them who are looking for ways to help. In an increasingly connected world, crisis-affected populations will increasingly remain connected to close friends and families outside of the crisis-affected region who will gladly take on much of the information processing duties, while possessing vital local knowledge and personal connections. This can be a gain for all three groups: the crisis-affected population, their friends/family, and the humanitarian organizations serving them.

While most chat-room extracts in this paper are from information exchanges about the core objectives of the initiative, there are just as many conversational interactions that were not directly related to the efforts but were crucial for forming strong ties and trust among the physically remote volunteers.

Socialization was an import motivator, both at the professional and personal level. Microtasking may have been an effective strategy for information processing, but as with any aid work it can have a profound psychological impact on the people who work on the systems. Often for days or weeks on end, people were processing graphic accounts from the affected region—a concentration of information about the most traumatic aspects of the crisis. As volunteers, they often lacked the training of experienced aid workers and were more vulnerable as a result. As remote workers, personal trauma might not as easily identified by their peers and they are not able to take full advantage of the support found in face-to-face socialization. The ability to interact online was one way to mitigate this.

The chat-logs contain a number of exchanges between people describing the loss or injuries of loved ones, and it was clear that having a place to exchange this information was very important. To respect the privacy and dignity of the people mentioned within, these personal interactions are omitted entirely from this report.

Just one similar interaction is included as an example here, showing socialization around an iconic brand of Haiti, while not being directly related to the Mission 4636 tasks themselves:

“My friend i have some terrible news to annouce”

                M, 25 Jan 2010, 02:11:56 (chat-log:13715)

“this will affect us all”

                M, 25 Jan 2010, 02:12:09 (chat-log:13718)

“Prestige brewery hase been badly damaged”

                M, 25 Jan 2010, 02:12:53 (chat-log:13723)

“oh noooooooo pas prestigee can we take a plane and fix it timoun”

                S, 25 Jan 2010, 02:13:18 (chat-log:13728)

“you guys are ridiculous”

                F, 25 Jan 2010, 02:13:48 (chat-log:13730)

“there is still hope”

                R, 25 Jan 2010, 02:14:17 (chat-log:13736)

“timoun gen on bagay ki rele patrimoine”

                S, 25 Jan 2010, 02:14:34 (chat-log:13740)

“this matters lol”

                S, 25 Jan 2010, 02:14:39 (chat-log:13742)

“its like the national palais”

                R, 25 Jan 2010, 02:15:06 (chat-log:13745)

“no our national identity”

                M, 25 Jan 2010, 02:15:14 (chat-log:13747)

“its a symbolic pride”

                R, 25 Jan 2010, 02:15:24 (chat-log:13748)

The names above are deliberately reduced to initials—while they were conversations on an open chat, these personal interactions are clearly not public conversations.

It is important to briefly note that humanitarian work, even when remote, can also result in negative physcological effects. The increase in remote work within the military, such as with drone operators, was predicted by some to reduce the psychological trauma of military personnel. This has turned out not to be the case, with early evidence that remote military personnel suffer post traumatic stress disorder (PTSD) at higher rates than their on-the-ground counter-parts (Chelala 2010). It is suggested that one reason for this is the sudden mental shift that the remote worker has to make between sometimes traumatic work and their home lives, with the disconnect between the two meaning that they lose the benefits of socialization. The psychological effects of remote work are already well-known among emergency dispatch operators, where it has been observed that the rate of psychological trauma among remote dispatchers is equivalent to those responding to the emergency on the ground (Jenkins 1997). The same could as easily be true for some types of remote, crowdsourced aid work. One member of Mission 4636 who did not lose friends or families in the earthquake reported that they suffered from poor sleep and depression for weeks afterwards. It is clear that they took on an emotional burden on top of the information processing burden. As remote workers, it will be harder to identify possible trauma among people contributing to crowdsourcing efforts.

This was not lost on the people within Haiti. In reference to the international workers, one of the volunteers in Haiti offered a plain compliment and observation:

“ils ont donne leurs coeurs et leurs temps”

(‘they give their hearts and their time’)

                Pierre, 17 Feb 2010, 18:49:04 (chat-log:35964)

It is impossible to fully imagine what it must have been like for the Haitian diaspora during this period, and those of us helping with the relief efforts are eternally in their debt for their selflessness in putting aside their grief in order to help. One of the most active volunteers offered this account:

“Like many other Haitian immigrants in the US, I watched with horror, despair, and helplessness the media coverage of the earthquake that devastated my home country and destroyed so many lives in Haiti. Having lived in the US for nearly two decades, I have never felt so disconnected from my home country and people than at that moment. I was not there to help or even be there, to pay witness to their suffering, in their greatest moment of need. Grief ridden, overwhelmed with guilt and sadness, I sought ways to help by providing assistance to loved ones in Haiti and making financial contributions to various aid agencies in Haiti. But, it still did not feel enough.

Translating as a volunteer in Mission 4636 helped me to do something more active and emotionally engaging. It helped me to connect emotionally, though far away, with the people of Haiti. It was satisfying to know that I was making tangible contributions and that I was helping to save lives. By helping others, I also helped myself. I felt useful once again and able to participate in and sit with the pain of other fellow Haitians. It brought me closer to home and closer to my people. Indeed, it was painful at times to read traumatic stories over and over again and to read about peoples desperate cries for help and not being able to help. But, this was pain that I welcomed, a pain that made me feel Haitian.

The other volunteers also provided much needed psychological support at the time. I was able to connect with others all over the world and share stories, experiences, and reactions to the earthquake. I quickly became part of a community where I was able to speak my native tongue, Haitian Creole and relate to other Haitians and friends of Haiti. Feeling isolated in a predominantly white community in Northern California, those connections were significant sources of support”

Johanne Eliacin, Mission 4636 Volunteer

8 Discussion and critical analysis

8.1 Social media

Much has been made of the role of social media in emergency response. For the most part, its role has been inflated, with 75 % of recent research focussed on Twitter relative to SMS and Email, when Twitter only makes up less than 0.16 % of the relative global volume in communication (Munro and Manning 2012). In other words, Twitter has received almost 500 times as much attention as it deserves, relative to SMS and Email.

There are no known cases of emergencies reported through social media in Haiti that were responded to. As some of the exchanges in the article show, though, there were a large number of emergencies/requests reported through SMS, email and direct phones calls. Some of these events were later reported on social media, and this may have lead to belief that social media itself played an important role.

However, social media was the primary means through which we found volunteers. At the time, none of the major social networks allowed people to list languages spoken as part of their online profiles, so Kreyòl speakers were found through direct connections or in groups that seemed to have a large number of speakers. This can be seen in Table 8, where people were cross-posting for volunteers so quickly that they risked being classified as spammers by the Facebook platform.

Table 8 Chat exchanges showing Mission 4636 workers cross-posting for more volunteers, and the resulting actions by Facebook to whitelist our actions, allowing us to connect and dynamically manage a large number of potential volunteers

Facebook engineers were responsive to the need to cross-post quickly, especially when we needed to quickly address a backlog of messages (they often arrived in bursts). While I did not have close contacts at Facebook, the VP of Product, Chris Cox, had been a member of the same Natural Language Processing research group at Stanford. I reached out through mutual connections and was speaking with Facebook engineers within a few hours. After diagnosing the problem, they whitelisted our activities which allowed us to quickly post for short-term help across a number of groups in quick succession, allowing us to dynamically adjust our workforce according the need at a given point in time.

The turnaround for Facebook modifying their platform for us was just 5 h. Compared to the many days it took for NGOs to make small changes to their systems (even just to reach out for volunteers), it is one of the reasons that this report recommends that aid agencies looking to engage in humanitarian crowdsourcing engage more directly with technology companies, rather than technology-branded NGOs.

For Mission 4636, and for information sharing among the diaspora more generally, Facebook were by far the most important social media platform that we used.

As stated above, there has been a great deal of literature about the possible role of Twitter in crisis-response, much of it mentioning Haiti. However, researchers have struggled to identify examples of it actually be used, typically talking about the potential use-cases that might use specific vocabularies (Starbird and Stamberger 2010). There were, in fact, some cases, but they were rare:

“this is surreal i just tweeted the exact street address of the sodec hospital to a faifax county rescue team”

                marc, 21 Jan 2010, 21:15:13 (chat-log:4453)

“i didn’t know these guys where on twitter”

                marc, 21 Jan 2010, 21:16:17 (chat-log:4454)

Here, an international search and rescue team was able to geolocate a hospital by reaching out to Haitians via Twitter. Those of us working closely with the crisis-affected population did witness cases like this, but not a large volume and all are similarly anecdotal and rare – the incredulity of Marc, above, highlights this. Most use cases were like the one above, where it is an international organization that is requesting help via social media, a use case seen more frequently in response to the 2010 Pakistan floods (Munro and Manning 2012). Given the uncertainty of response in reaching out via Twitter, and the difficulty in evaluating the reliability of a given response, it may have been more desirable for there to be a more systematic way of finding people with the right expertise.

While examples like the one above will no doubt be well-received by researchers who are thinking about a role for Twitter in emergency management, it should be kept in mind those close enough to witness these events are also arguing that Twitter is currently over-researched in this area, compared to other communication platforms like SMS and Email, and even the larger social networks like Facebook and LinkedIn (Munro and Manning 2012).

As the earlier chat extracts showed, most of the interactions with people on social media were simply to encourage them to verify information (to the best of their ability) and report through the most appropriate channels possible. Mission 4636 was not always the most appropriate channel, as with the examples encouraging people to use Person Finder, and in many other case where a local connection was available:

“we referred the poster to a help number on the ground, in Haiti”

                marc, 21 Jan 2010, 20:59:43–51 (chat-log:4415-6)

In this case, Marc was referring to a Facebook post that asked for information about a hospital in Port-au-Prince that happened to be run by his cousin. These direct correspondences and connections were no doubt of a much greater volume than the text messages to 4636, employing the crucial local knowledge in the same way, but very few of them are documented like the one above, which happened to be discussed on the Mission 4636 chat room.

As a social network that explicitly tracks skills, LinkedIn would seem like the best natural fit for targeting volunteers with required skills. This occurred in Mission 4636, especially via professional connections (see the message from Jenn, a professional French-English translator, in Table 8), but there isn’t enough evidence to track the importance of LinkedIn to Mission 4636 one way or the other. However, the ‘LinkedIn for Good’ initiative was reportedly inspired by Mission 4636 discussions (p.c.), which is a positive move in the right direction for social networks. Similarly, Facebook now explicitly lets people declare which languages they speak.

After having found a strong collection of a few hundred volunteers in the first 2 weeks, the use of social media dropped off as a recruitment platform. The core few hundred volunteers joined one of a few Facebook Groups, named “Mission 4636 Volunteer Translators & Location Specialists” and “Together We Can Find 100,000 Speakers of Haitian Kreyol”. When we needed to rally volunteers, we simply pinged these groups, writing on the wall of the groups in low urgency situations and messaging all members directly whenever a more substantial backlog of messages appeared.

8.2 The role of international actors

The Haitian population processed at least 90 % of the crowdsourced information and possessed the crucial linguistic and geographic knowledge. However, it would be wrong to credit no success to the international actors. Among the online workforce were many non-Haitian volunteers, some of which were people who had picked up Kreyòl while working in Haiti or with the diaspora:

“Jimi how u know Kreyol?”

                S_Michel, 09 02 2010, 02:05:24 (chat-log:32298)

“u teach youself?”

                S_Michel, 09 02 2010, 02:05:45 (chat-log:32299)

“My degree is in French. I worked for a company here that was contracted with MCI back in the 90s to field calls from Haiti to the US and Canada.”

                Jimi, 09 02 2010, 02:06:14 (chat-log:32300)

Other people were leveraging their French where possible, and others still were simply natural organizers, monitoring the volume of messages and seeking more workers when needed (as in the examples in Section 8.1).

While the analysis here champions the role of local workers, it also shows that there are many important roles for international actors in crowdsourced response efforts:

8.2.1 Ensuring employment for workers

This is the most important role for international workers, and arguably the most important action by international actors in Mission 4636, establishing a paid workforce in Haiti for the latter half of the initiative. In terms of local knowledge and the need for work, the best workforce will be the people on the ground from among the affected community. However, access to funds and resources might be limited, so international actors could encourage and expedite employment possibilities, as in Mission 4636 with the FATEM workers.

Paid workers do not necessarily have to be in the disaster-affected country or region. In the case of Haiti, as with any large scale crisis, there were some logistical constraints in getting computer equipment and supporting infrastructure to the workers in the country. For example, in the case of Mission 4636, cargo-space designated for aid efforts needed to be negotiated for computers, the first internet solution didn’t work, and people needed to fly there to train the workers. This presents a fair amount of overhead cost that will be present in any response effort. Why not, then, look to employ people with local knowledge outside the country? In the case of Mission 4636, the volunteers were disproportionately from among the wealthier Haitians: “we are the country’s middle and upper class and Haitians living abroad” (chat-log, 438). This was for a number of reasons, such as a higher likelihood of being overseas and speaking English in the first place. The most common demographic were professionals in their 30s and 40s, and there were many people who helped for only an hour or two each day because that was all they could spare between work and family commitments. People like this outside the disaster-affected region would be an ideal paid workforce. For example, a Haitian lady in New York, who is working as a contract cleaner and has the vital local linguistic knowledge, geographic knowledge, and social connections within Haiti, could undertake this kind of work for supplemental income. Against the backdrop of hype around crowdsourced information processing, major relief organizations like the UN and International Red Cross were refreshingly candid about how they had fallen short in translating information back into Haitian Kreyol (Chan and Crowley 2011; Wall and Chéry 2011). At the same time, their donations greatly exceeded their ability to get aid into the country because of transportation bottlenecks. Perhaps setting up a translation center in somewhere like New York, with a large concentration of the diaspora, could address this? It would avoid the transportation bottlenecks and costs, payment would ensure the workforce were able to remain committed to the project, and it would allow the scalable information processing that was so difficult for already-overworked aid workers to establish within the country. Footnote 7

8.2.2 Quality control

Another role for international workers is quality control. Having an extra step for quality control takes the burden-of-error off workers within the crisis-affected community, addressing some of the concerns about the possible psychological trauma that might result from remote aid work.

8.2.3 Rallying and training workers

As with quality control, it might be desirable to have the responsibility for work volumes fall on people outside the crisis-affected community. It takes specific skills and time to rally workers, train them in information processing, answer specific questions about the workflow, and to establish methods by which they can reach out through their own networks for more volunteers. These tasks will often not require the local knowledge of the crisis-affected region, leaving those with the vital local knowledge to apply it to the tasks where they are needed most.

8.2.4 Curatorial control

International workers can maintain curatorial control in potentially dangerous situations. For example, the Libya Crisis Map had pro-insurgent reports from parts of Libya that were briefly recaptured by pro-Gadaffi forces. An objective international observer with reliable connectivity could update security policies according to the changing conditions, keeping people within the crisis-affected region updated about the possible security implications of their past reports, and pro-actively remove/anonymize potentially dangerous personal information when needed.

8.3 Why the ‘new’ technologies were really old ones

Table 9 gives a summary of the technology used. The communication and collaborations technologies used in Mission 4636 (SMS and IRC) pre-date the web. The forms used to process the messages are basic HTML from 1995 while the format used to share content, RSS, dates from 1999. Dynamically-generated maps also pre-date the web, but the most-used map, Open Street Map, launched in 2004, meaning that the youngest technology was still 6 years old at the time. The most cutting-edge technology in the whole process was probably the phones in the hands of people in Haiti.

Table 9 The technologies used in Mission 4636. The online editable maps we used were 6 years old at the time, although (non-editable) dynamically generated maps were 17 years old. Every other technology was more than a decade old. This goes against the conclusions by organizations including the BBC, FEMA, Google, the New York Times and the United Nations (see (Nelson et al. 2010, Roberts and Payne 2011, Zook et al. 2010) and references within), that the initiative was built on or the result of ‘new technology’. It was not new technology that allowed Mission 4636 to operate, but it may have been the social change resulting from 10+ years that people had been using established technologies for distributed communication. The phones kept working, so people maintained their communication practices

It could be argued that distributed microtasking was a new technology, in which case it was 5 years old, with Amazon Mechanical Turk launching in 2005. However, distributed information processing is much older. Grier puts the computational start in the early 1700s, where three French astronomers discovering that they could divide the mathematical labor for calculating the orbit of Halley’s Comet (Grier 2005). The first more widely distributed calculations of this sort were in the late 1700s with England’s annual Nautical Almanac, where calculations were performed by dozens of people in parallel, with their parts of the equations delivered by hand or horseback (Croarken 2003). In terms of creating translated, structured reports from distributed information sources in plain language, the strategy is as old as any military or political organization. For distributed microtasking, it is probably best to think of Mission 4636 as using a well-established information processing strategy that happened to leverage more recent technologies.

Our impression at the time, and the conclusion of this report, is that any success was not the result of new technology, but the result of social change from the 10+ years that people have been using these technologies—it is an expectation that we can stay connected and contribute from anywhere in the world. The crisis-affected population helping itself is always the main responder and we simply helped to expand this a little to those outside the country.

Despite this Nelson et al. (2010), Roberts and Payne (2011), Zook et al. (2010) (and references therein including FEMA and the United Nations), have called the initiative ‘new technology’. None have attributed it to social change, although Zook at al. (2010) were close by beginning their paper by extolling the virtue of individuals, but then fell back into a familiar reporting pattern by framing each section of their paper around a particular international crowdsourcing organization or technology (the role of Haitians in their own recovery is largely ignored). Like most other organizations talking Mission 4636 in Haiti, they talk about past uses of the some of the technologies, but omit the fact that the Haitian population were the single biggest workforce.

In terms of the source of the technologies, almost all of them were built in California, primarily from Silicon Valley and around. Perhaps around 90 % of the international technology and management can be attributed to: Josh Nesbit who brought together the original parties; Robert Munro who ran the Mission 4636 platform for volunteers; Lukas Biewald who built the instance at CrowdFlower which was largely developed by Chris Van Pelt; Leila Janah who headed Samasource; Alex Onsager and Eric Nguyen from Samasource who worked most closely with the workers in Haiti; and Tim Schwartz, who built the HaitianQuake platform which was also the base of the first Mission 4636 platform. Six of the eight lived in San Francisco and six of the eight were Stanford alumni. Only Schwartz, in nearby Los Angeles, had not lived in the area.

If the crisis-response world is changing on the back of new technologies developed from unlikely parts of the world, like so many commentators have claimed (Nelson et al. 2010; Roberts and Payne 2011; Zook et al. 2010), then this wasn’t it.

It was by deliberate choice that the core of Mission 4636 used the most established and well-tested technologies—it would have been irresponsible to use potentially faulty technology for information critical to saving people’s lives. By wrongly advocating new technologies, external commentators of our work run the risk of making innovation sound appropriate in high-risk situations. The only innovation here was strategic—the use of real-time microtasked translation—and even then it was because we didn’t have a choice. While it built on new approaches in technology, it also built on the most established form of information sharing in a disaster: a crisis-affected community helping itself.

8.4 Eliciting information and raising expectations

One of the hardest aspects to control was the volume and content of messages coming into Mission 4636. While we were able to dynamically adjust the number of people working by calling out for volunteers at times of high volumes, the system remained potentially fragile in this respect.

At peak, we were able to process around 10,000 messages per day, or perhaps a little more in the first week. This is still only around 1 message for every 1,000 people in Haiti. Although emergencies were responded to after messages were sent to 4636, we did not advertise the number within Haiti as an ‘emergency response service’ as it was never clear to us running Mission 4636 what the actual capacity to respond to specific emergencies was, but it was safe to presume it was always very low. If just one bad rumor was announced by radio in the country and only 1 % of the population sent a message to ‘4636’, then it could have inundated the system well beyond capacity. This is true, of course, for any reporting service. In our case, we also did not want someone submitting a message to think that a response was guaranteed, and therefore not take other action to help themselves or those around them. Haiti has never had a fully functional ‘911’ service, especially not for medical emergencies, so the population was less predisposed to assume this should exist than people in other countries might, but it was a constant concern. In the case of Mission 4636, many volunteers were listening to the main radio stations over the internet and could monitor for misinformation. We had a few direct contacts at the radio stations, meaning that we could contact the stations and correct misinformation when appropriate. This is also one area where local social and linguistic knowledge is vital.

In order to send the most neutral reply possible, the messages sent to 4636 received and automatic reply that said “message received”, in Kreyol, French and English. It was to let people know that they were heard and that their message was not lost in transit, but also made no commitment to response. The exact wording was because “message received” is a cognate in all three languages, and therefore a Kreyol-only speaker would not be confused as to the content of the following French and English translations (and true for other language combinations).

From shortly after launch we were given the opportunity to put a message on every phone on the main network, Digicel. We decided not to (a few reports to the contrary (Clémenzo 2011; Thomson Reuters Foundation 2010) are incorrect). It is one thing to be asking for information via radio broadcasts—many radio stations were doing the same for their own information sharing initiatives— but quite another to directly engage the population with a message on each of their phones. We would not have been able to process the volume of replies anywhere near quick enough, and it may have raised expectations so that the population changed their behavior in ways that did more harm than good.

For similar reasons, Mission 4636 was almost concluded at the beginning of the second week, after a largely unsuccessful attempt to engage other relief organizations. For 3 days after launch, the messages were streamed to two people from an NGO called InSTEDD. They had arrived in the ‘eye of the storm’, shortly after we launched Mission 4636, but left shortly before large volumes of messages started coming through. It was an invaluable local contact of theirs (whose name they unfortunately can’t recall) who took them to radio stations in Port-au-Prince to first publicize the number. At this time, they were working directly with aid organizations in the base at the airport (manually) injecting reports into the search and rescue efforts. They only know of one case of someone being rescued during this time, a foreign national who they met at the airport when he was leaving (Morrow et al. 2011). It is likely that there were more successful responses for Haitian citizens during these few days, but there is no way to track this.

The ongoing support of the volunteers was a constant source of motivation during this time, as we considered the best methods for interfacing with international relief efforts:

“Even my aunts and mother pitched in to translate. My Kreyol was rusty but we helped each other well! and it was a privilege to have helped. There was never any doubt in my mind about the importance of what we were doing.”

Steve Michel (http://www.mission4636.org)

A little over 2 days later we received confirmation that the US Coast Guard were still using the messages, Footnote 8 with the request for a higher volume, if possible, and established more direct interactions with them. This was night-and-day compared to the troubles interfacing with the aid agencies in the base. It is not clear how long they had been taking the messages, but from this point on the translated, structured messages were monitored closely and in real-time. It was because of the resilience of the diaspora looking to help that we reach this point, not the actions of international actors.

8.5 Information as aid

For a short time, the Thomson Reuters Foundation were sending information messages out on the 4636 number. These were general instructions, like sanitation advice and steps to ensure clean drinking water. The use of the same number was based on some initial confusion – they started on a different number and we expected the Thomson Reuters Foundation to be answering specific questions to 4636, not using the number as a broadcast (known as a ‘text blast’). The dual purpose created some confusion among the population who were replying to the 4636 text-broadcasts thinking that it was a reply-based information service, which produced immediate backlogs in processing while trying to sort them from the genuine reports of information from the ground. Footnote 9 There were only 1 or 2 days when it took hours to clear the backlog of erroneous messages, but it was a reminder about a potential breaking point.

On three separate occasions, organizations had indicated that they would start replying to individual requests for information sent to Mission 4636. All three fell through. This is one area where we fell short of our initial goals. Immediately following launch, we allowed volunteers to read the phone numbers of the messages and therefore reply. Our first change to the live system was to remove the numbers from the front end, and only allow a select set of people access to the complete set of messages. While we wanted to facilitate as much information sharing as possible, there was a fear that volunteers might reply ‘help is on its way’ when it was not. This may not have been a well-founded fear, as the volunteers had a very clear idea of the true scale of disaster as they were immersed in a constant stream of messages. However, it was agreed that it would be better to find a systemic way to do this, preferably with a community or relief group at one location where everyone replying could be monitored.

After the third offer from a community group to respond to individual messages did not materialize, we decided to allow volunteers to once again reply. However, we waited until we had moved to the CrowdFlower platform so that we could track exactly who had seen a given message, and did not allow people to see phone numbers by browsing ones other than those presented to them for translation. It was still a difficult decision to make and one that we would probably not have made in the context of a conflict. Footnote 10 In the absence of a pre-existing community group, they were probably the best possible people to be replying to messages for the same reasons they were not going to wrongly promise that help was on its way: they had been looking at reports for most of their waking hours for weeks, and as a result probably had among the richest and broadest knowledge of anyone in the extended relief efforts.

The most important information sharing associated with Mission 4636 were not the text-blasts or even the personal replies, but the constant communications between the volunteers and their friends and families within Haiti:

“was anyone able to reach people in haiti today. for some reason i couldnt get thru at all”

              smile, 25 Jan 2010, 02:07:48 (chat-log:13681)

“@smile, yes, chatted with 2 cousins”

              FionaUH, 25 Jan 2010, 02:08:06 (chat-log:13683)

“okay. thanks”

              smile, 25 Jan 2010, 02:08:19 (chat-log:13685)

“@smile yeah with my stepmom just keep trying”

              Rebecca, 25 Jan 2010, 02:08:27 (chat-log:13687)

“Yeah I just lost $20 in a phone card today for one phone call that didn’t connect.”

              Kat, 25 Jan 2010, 02:08:36 (chat-log:13688)

“okay. thanks. i will”

              smile, 25 Jan 2010, 02:08:42 (chat-log:13689)

“smile ive only gotten through to haiti twice since the earthquake its totally normal”

              Sarah.U.H, 25 Jan 2010, 02:08:59 (chat-log:13691)

Communication into the country was difficult. As Sarah.U.H. says above, she had only been able to get through via calling twice in the 2 weeks since January 12.

For 4636 messages, this allowed replies to messages to give people more information. With access to the web, international volunteers were able to take the time to more carefully research the locations and procedures for getting aid, and communicate this back to people within Haiti. For people not fortunate enough to have their own connections in the diaspora, Mission 4636 was able to plug the information gap by interacting with the Open Street Map relief layers, and communicating the information back to people within Haiti who only had access to radio broadcasts and non-internet-enabled phones. In some cases, the volunteers also plugged the aid gap, as can be seen in Table 10.

Table 10 Chat exchanges that show the exchange of information between people in the crisis-affected region and those outside of it, including information about the locations of aid stations, the process for receiving aid, and remote volunteers directly donating money to people within Haiti

In these cases, the volunteers were able to plug at least some of the gaps in the aid net. In both cases, Steve and Ronny are referring to groups of about 200 people. Aid organizations within Port-au-Prince were operating at a larger scale, and even among the US Military, who are better equipped for targeted response, we don’t know of any responses to requests for (only) food for less than 1,000 people. (The volunteers' frustration is understandable and does not necessarily reflect badly on the actions of the aid organizations within the country in either case.)

The most common case when exchanging information was that people were more successfully able to receive the information that directed them to the nearest aid station or informed them about the coupon system for food. The exchanges in Table 10 show that these were also attempted before the volunteers transferred money to the recipients. It is difficult to argue with Ronny’s point that his charity was best given directly (Western Union had lifted fees for transfers to Haiti during this time) as he had been interacting with the people long enough to trust his donation was well-placed. However, for the same reason as the emergency response component of 4636, we did not make public at the time that money was being donated to people as a result of their sending a message to 4636. It could have raised expectations with ultimately negative consequences.

8.6 Motivation and management

Even though no social networks recorded the languages spoken by their users at the time, it was not difficult to find potential volunteers among the Kreyòl-speaking diaspora. As soon as a few people were found, the exponential effect of reaching out across their existing social networks brought in many people.

Maintaining the volunteer initiative was the hardest part, and retaining volunteers correlated strongly with how engaged they were in the online chat room. The most prolific volunteers (by messages processed) were also the most active people in the chat room. The most prolific on both was Fred Michel, who processed more than 4,000 messages individually while also responding to hundreds of online requests for help with translations and geolocation.

When Samasource began planning their operations for paid workers they also began a public campaign to find more volunteers. Unfortunately, there are no recorded volunteers joining at this time for more than a few hours. Of all the volunteers that were still working in large volumes from the fourth week on, every single one had been there in the first week. It became increasingly difficult to find new volunteers, probably for a number of reasons, but primarily because of general burnout among people who had worked on other efforts. It may also have been in part the nature of how the volunteers were recruited, as many of people who became the most dedicated were recruited directly by someone in their social circle. It may have been through these stronger social ties that it is best to engage someone for a longer period of volunteer work. There were certainly some truly exceptional people who started volunteering without a direct connection to anyone else in Mission 4636, but even then they quickly formed strong ties with many others. For about half a dozen people, I had one-on-one phone conversations early on for various reasons. Every one of them was still volunteering a week later.

The crowdsourced missing persons initiative completed its mission in too short a time to gauge volunteer management strategies one way or the other, but the students in Boston also seemed to observe this. Several people there noted that many of the volunteers, especially as things wore down there, only came in for short periods and may have been motivated, at least in part, by wanting to be associated with what was then a high-profile media effort at Tufts University. While they no doubt wanted to help, the dual motivation is less likely to attract someone who will want to work for an extended period because part of the motivation, the chance to be affiliated, is immediately fulfilled. Despite the very high media attention, and repeated visits by camera crews, the efforts in Boston were still not able to process one tenth of the reports of Mission 4636 (even when excluding the paid FATEM/Samasource workers). By tying the initiative too closely to the media arc of attention, the worker-pool therefore lost workers and motivation as the attention moved elsewhere. Even among the most dedicated workers, observing the quick turnover of volunteers could be a demotivator.

Another danger with an initiative following the media arc is that the media attention disappears well before the needs on the ground do. In preparing this article, I found that no one from the Ushahidi Haiti at Tufts initiative realized that the third week was actually the largest volume of messages sent to 4636. A consensus seemed to have been reached that the second week was the busiest and then it rapidly dropped off. This seems to be based on the amount of work committed by this part of the initiative, which saw a substantial 80 % drop off between the second and third week at Tufts (see Fig. 3). The core Mission 4636 platform also saw a drop in work, but only by 8 % (this was one period when we had to regularly not translate messages that were clearly of no use to responders, even for situational awareness). It could be dangerous to map efforts so closely to the media in this way, as a reliance on the output could be built that then disappears at the most crucial time.

In the case of the core Mission 4636 platform, we were aware of the media-affect at the time, deliberately keeping the website out of public locations to avoid well-meaning but potentially detrimental ‘digital disaster tourists’:

“robert… is there a web site where they have the link to the SMS and the other to the people finder….”

          Sebastien, 21 Jan 2010, 01:08:53-10:50 (chat-log:3557-8)

“@Sebastien we are deliberately not making the SMS page public right now, but just relying on word of mouth”

            Robert_Munro, 21 Jan 2010, 01:10:59 (chat-log:3559)

“ok… sounds good… i can find it here if needed…”

              Sebastien, 21 Jan 2010, 01:11:16 (chat-log:3560)

“because there has been a lot of media, we don’t want people coming to just read messages”

              Sebastien, 21 Jan 2010, 01:11:40 (chat-log:3562)

“good… make sense…”

              Sebastien, 21 Jan 2010, 01:11:52 (chat-log:3563)

It can be concluded that it might be best not to pursue media attention during future efforts of a similar nature. This may well be one of the critical factors that has made Mission 4636 so much larger than all the other initiatives in Fig. 1: these other deployments all deployed public maps of events and mostly only structured existing media reports, so the volunteer involvement and frequency of information faded with the media attention.

For the most part, we were successful in avoiding popular media attention and keeping the volunteer URL off public websites. However, shortly following the launch of Mission 4636, the Thomson Reuters Foundation published an article about the number, Footnote 11 which was shortly followed by an article from Ushahidi. Footnote 12 Neither organization’s article mentions the other organization and both portrayed the respective organizations as running the Mission 4636 initiative. This led to some of the tension between international organizations, although it is not the intent of this paper to explore them. For the Ushahidi article, at least, the actual workflow is more or less correct, describing the Mission 4636 system as supplying information to response organizations rather than to a public map.

The eventual use of a public map may have also driven away the volunteers with the most crucial local knowledge, as they are likely to be the most sensitive to the possible security concerns (Morrow et al. 2011). This was also observed with the later ‘Libya Crisis Map’ in 2011. It was initially a private map, and people from Libya or with close connections to the country joined the volunteers on the condition of privacy, for fear of association with what might be perceived as an unwanted external influence (Bott et al. 2011). When the United Nations Office for the Coordination of Humanitarian Affairs (UN OCHA) decided to make the map public, every Libyan volunteer left. UN OCHA had come under criticism for not adopting crowdsourcing earlier, and this may have been a contributing factor. Perhaps it was because UN OCHA wanted to show that they were using the crowdsourcing-related Ushahidi software, that they actually drove away the most important members of the crowd.

One common conclusion in many reviews on humanitarian crowdsourcing is the need to institutionalize the crowd (Chan and Crowley 2011; Morrow et al. 2011). The argument for this is dependability. Referring to Haiti, in the Disaster 2.0 report UN OCHA expressed concern about the reliability of the workforce.

“Maybe nobody shows up [next time] … for me it is not always just who is authoritative, but also who is consistently there and predictable.” Andrej Verity, UN OCHA (Chan and Crowley 2011)

This is in reference to the crowdsourced response efforts in Haiti and is clearly based on the misperception that most of the crowd were a selection of international individuals and organizations willing to help. As this article makes clear this simply wasn’t the case. The majority of people came from the crisis-affected population and they have no choice but to show up. They will be there in large numbers, often without work and always looking for ways to help.

I agreed with the position of Chan and Crowley (2011), Morrow et al. (2011) and others at the start of Mission 4636, which was why there were several attempts to find an existing organization to manage the initiative. After several failed attempts, and from witnessing the internal tensions of the international organizations, it is not clear that this was, in fact, the best possible option. What community groups will have organizational structures that are the best suited to crisis-response? Organizations are varied, but established diaspora and volunteer groups are often lead by people with the most available time and/or publicity skills. While vital for maintaining a successful organization over time, these people may not necessarily be the most suitable leaders for an organization during the high-stress environment of an actual crisis. By engaging people at the individual level, it may therefore be easier for effective leaders to emerge in ways that don’t create tension within existing organizational structures. Footnote 13 While organizations will all have their own agendas and objectives for response, unaffiliated individuals are more likely to be looking for initiatives to join, and may be more flexible in terms of how they are able to contribute.

In terms of management, Mission 4636 had a very flat organizational structure, with only about ten people given additional access privileges where they were permitted to browse already-processed messages to check for consistency, accidental errors, messages requiring an immediate reply (like a request for more information) and time-critical messages to immediately escalate to the responders through direct communication channels. This was also important for verification—there are messages with added notes like ‘I spoke to him and his leg is ok, but he is worried about his son’, which helped inform the appropriate response. The people with access were a small number of individuals from the international NGO community and people that I hand-selected after observing them as natural leaders among the volunteers. Some of the selections took into account the person’s professional experience, such as Sebastien (see chatroom exchanges) who worked as a emergency medical technician (EMT) in his home in Michigan, and a handful of people with military experience. Others, like Ronald, were natural leaders with a background in commercial or community management. However, I deliberately omitted from selection some of the most dedicated volunteers that I knew had lost family members in the earthquake. I trusted them completely, but by being given access to all the messages they would be taking on the additional burden of ensuring that no errors slipped through. While they would no doubt have been willing to take on this role, it seemed like too much of a responsibility to put on people who were already carrying such a strong emotional burden—they were already contributing so much—or to risk any errors that might have occurred as a result. Even in hindsight, it is difficult to evaluate whether these were the correct management decision to make (which would be better evaluated independently, in any case). They are reported here primarily to share the kinds of issues that might arise when managing volunteers from within the crisis-affected population, to help be better prepared for similar, future scenarios.

8.7 Privacy and security

As the messages contained personal information about at-risk individuals, security and privacy were always a concern. This is one area where distributed crowdsourcing by volunteers is weakest, and the volunteer component would not be appropriate for a more high-risk context such as the response to a conflict, or reporting under an oppressive regime.

The potential security issues are very broad and also include compromised communication channels and deliberately false data (see Chamales and Baker 2011 for a more complete account of the types of security problems that might occur in crowdsourcing for crisis-response.) All security works on a principle of degrees of difficulty: no security is impossible to break, but it can be made more difficult than someone is willing to commit to breaking, based on their perception of the value of what is being protected.

The biggest concern for Mission 4636 was that people’s identities would be exposed, which is why the messages were not published online. For their part, the students at Boston asked two of their professors whether it would be acceptable to put the messages online. The advice was that it would be acceptable if the people knew they were sending messages to a public map (p.c.).

Clémenzo’s recent ethnographic study of fifteen people who have messages on the Ushahidi Haiti map found that all remembered 4636, but only two had heard of Ushahidi Haiti or knew of the online map (even more than a year later) (Clémenzo 2011). We can conclude, then, that it is likely that the use of a public ‘crisis-map’ in this context was not permitted for many of the messages sent to ‘4636’. By Ushahidi Haiti’s own criteria, they did not have permission to publish the messages online, and our initial decision not to do this was the correct one.

Unfortunately, a leaked document from the US’s Department of Homeland Security shows that Ushahidi Haiti was among 50 websites that they were actively following for information about potentially dangerous individuals (Triner 2011). This makes for a disturbing situation: people in Haiti sent a message, on a Haitian phone number to connect with responders within their country. An overseas organization then published those messages online (despite conflicting advice) where they were exposed to an intelligence agency. We have to conclude, then, that some people who did not receive any aid, nonetheless had their identities logged by a foreign security agency as a potential threat as a result of sending a message to 4636.

Fortunately, Mission 4636 was not listed among the 50 websites in Triner (2011), even though some other password protected sites like Facebook were. The security for Mission 4636 was fairly low—at the time an API key allowed access to the data—but this seems to have been enough for this particular case, meaning that only the 5 % of messages that ended up on the Ushahidi Haiti platform were exposed.

This lesson can be applied more broadly. In all humanitarian information management systems, the system should be as far along the security continuum as possible. With finite resources, third-parties looking to covertly obtain personal information about at-risk individuals will balance the cost of circumventing security with the perceived value of the data itself.

8.8 Who’s afraid of the crowd?

Humanitarian and media organizations have deliberately omitted the contribution of non-affiliated individuals, Haitians in particular.

Within humanitarian and social development circles, it is much more common to talk about actors in terms of the organizations they represent rather than individual identities (this is true of organizations outside of humanitarian and social development circles, of course, but it is more pronounced here). This is understandable for continuity, as just 18 months after the events described here almost half the people in this report have changed their affiliations. But this leaves out the unaffiliated—the crowd. When at least 90 % of the people making and/or processing reports are not directly affiliated with a humanitarian organization, this omits an unacceptably large piece.

At least part of the error can be attributed to guess-work as the result of incomplete information (which this article hopes to address). However, the first report about Mission 4636 (Munro 2010) is the only report linked from the Mission 4636 website. It states the technology was mostly a decade old and that it was the people, not the technology, that made the difference. The ‘history’ section of the website Footnote 14 also clearly states the contribution of individuals working together, and the importance of the community connections between those within Haiti and those outside.

Is it the case that this site/report was simply missed by those reporting on Mission 4636? No, there are plenty of places where both are quoted verbatim, but with the focus changed to international organizations. For example, Internews is a media organization that reports on humanitarian developments. They ran an information service within Haiti following the earthquake. Their report about the use of media in Haiti contains a reasonable account of Mission 4636 (Nelson et al. 2010). However, it also contains a timeline of media events that is lifted almost verbatim from http://www.mission4636.org without citation. It is mostly word-for-word identical, even containing the same heading names. The small amounts that changed? Internews wrote themselves into the timeline and simply removed the names of all non-affiliated people, which happened to include the timeline’s actual final author and every Haitian individual. The two accounts are given in the Appendix, showing edits. Without the citation no readers of the report will be aware of the editorial choice to remove individuals, primarily Haitian ones.

To a similar extent, the same is true of the United Nations and Harvard Humanitarian Initaitive’s Disaster 2.0 Report. In the section on Mission 4636, they present a workflow diagram that only shows the most complicated and lowest volume route, with messages being passed through InSTEDD’s software, through the core Mission 4636 process, on to Ushahidi and then finally to the responders. This counts for, at most, 3–4 % of the messages sent to 4636 that were streamed to the responders. The majority of messages came directly into Mission 4636 from the telecommunication companies and all the messages were streamed streamed directly to the responders, with only about 5 % additionally rerouted through Ushahidi Haiti. Unfortunately, these two major workflows are omitted entirely from the diagram in the Disaster 2.0 Report, listing only those that went through NGO-branded platforms. The third major workflow in Mission 4636 was the communication between the people working on Mission 4636 and the crisis-affected population (see Fig. 2). It is also absent from the diagram in the Disaster 2.0 report, with the core Mission 4636 system in the furthest corner from the area marked as ‘in Haiti’ (which is especially confusing as it contains the Samasource workers who were physically in Haiti). It is almost as if the NGOs inserted themselves into a workflow where the crisis-affected population were already helping themselves, only to play a very minor role and then take credit for all the success. Their work deserves to be highlighted, but perhaps not to this extreme.

It is not clear how much of this is the analysis of the report itself or the authors simply relaying information from the people they interviewed, who were predominantly from NGOs and therefore understandably predisposed to talk about their own work. Footnote 15 Whatever the cause, it may be that this bias—inflating the 4 % NGO-run to sound like the entire project—that has caused relief organizations to believe that it is important to engage with the ‘volunteer and technical community’ by parterning with NGOs, not the actual people from within the crisis-affected community.

The Independent Evaluation of the Ushahidi Haiti Project (Morrow et al. 2011) falls into the same bias in reporting. Commissioned to review the collaboration, Footnote 16 budget and timing constraints meant that they had to skip a planned trip to Haiti and were thus unable to contact any of the people who sent the actual reports, received a response, or the Samasource/FATEM workers within Haiti (Morrow et al. 2011). In the planning stages of the evaluation, the basic workflow of the collaboration was misunderstood and the reviewers planned only to ask about the Ushahidi platform and to only interview the non-Haitian volunteers. Because the planned review had to pass a lengthy IRB process, they were unable to change their plans for this part of the review (Morrow, p.c.). The result was that the survey they had to give to the Mission 4636 volunteers could not be changed. The linguistic constraints of the reviewers meant that the survey was only offered in English. Therefore, the only information collected from 95 % of the actual workforce was through a ‘survey monkey’ online questionaire asking the workers about software they didn’t use, written in (for most) their third language. The response rate was poor. In the final version of the report, the authors concede that they did not attempt to give a full review of Mission 4636. Aside from this organizational constraint, the report reaches many of the same conclusions as this one, especially regarding the security implications of posting text messages online and the lack of evidence that the public map contributed to the responses resulting from 4636 messages.

For their part, the Thomson Reuters Foundation conducted a phone survey of 400 people who received text-blasts via 4636 and found that 94 % found the service very useful (Thomson Reuters Foundation 2010). This is positive, but unfortunately the numbers don’t hold up to scrutiny. Having obtained the phone numbers used in the survey, a straightforward analysis of the messages sent to ‘4636’ from these 400 numbers shows that some words, like enfòmasyon (‘information’) occur far more likely than chance (ρ < 0.05). This means that it was not a random selection of the 4636 users and biased towards people who were more likely to be predisposed towards an information service. Second, they did not distinguish people who received the text blasts from those who received individual responses, so it is not clear they were getting feedback only about the blasts. Finally, more than 10 % of the numbers they called were from weather reports, a legacy of a previous use of ‘4636’. It is unlikely that more than half the weather reporters were requesting information about the earthquake, so presumably the ‘94 % positive’ comes from only those who responded to the phone calls. Therefore, like the Ushahidi Haiti review, it looks like the evaluation design and interpretation of results was not completely objective, and the actual figures are lower. There were no doubt people who were very happy to receive the information, but via the 4636 number itself we received many complaints, too, (there was no way to unsubscribe) so it is reasonable to conclude that there were positive but ultimately mixed outcomes.

In the 18 months following there has been a very large amount of attention paid to these efforts, both in the popular press and among the humanitarian community. It has overwhelmingly (and sometime exclusively) focused on the smallest and most peripheral of these groups: the students in Boston. For example, Crowdsourcing crisis information in disaster-affected Haiti (Heinzelman and Waters 2010) includes the history of the content management system used by the students in Boston, Ushahidi. However, the report doesn’t mention Samsource/FATEM, the organizations that completed the most work, or CrowdFlower, the platform where most of the work happened. The single largest group contributing to crowdsourced efforts to Ushahidi platform was from people within Haiti itself but they were omitted from a report that otherwise contained a detailed account of the software’s origins in Kenya and use in many other countries since. The authors, Heinzelman and Waters, both worked on Ushahidi Haiti and were commissioned by the United States Institute of Peace to report on Ushahidi, so it is natural that the report is more centered on this part of the work (p.c.). By extension, though, it means that it is focusing on the platform with the largest concentration of non-Haitians. In part, it is clear that report also mirrors some exaggerations of Ushahidi Haiti by external parties. 2010s bestseller about crowdsourcing is one example:

On Sunday, January 17, a full 5 days after a devastating earthquake struck Haiti, a text message sent from a cell phone in Port-au-Prince was translated from Creole into English and posted on an interactive crisis mapping site that was being closely monitored by emergency responders.

Opening sentence from ‘Macrowikinomics’ (Tapscott and Williams 2010).

This never happened. In fact, it couldn’t have, except to report on events elsewhere. Ushahidi Haiti (the ‘interactive crisis mapping site’) did not start importing 4636 messages until January 19, 2 days after this event. In another case, a graphic designer in Montreal and a writer in France processed reports on CrowdFlower about people needing emergency medical attention in Grand Goave. The translated, geolocated reports were streamed directly to the responders from the CrowdFlower platform, allowing the Marine Corps to save two people’s lives. Confusingly, the Marine Corps then thanked (only) the students in Boston, while the people who actually processed the reports – both members of the Haitian diaspora – did not find out about the successful response until the students in Boston (who had no way of realizing the misattribution) later blogged about it. Footnote 17

Although Tapscott and Williams don’t attribute the source, it may have come from the ‘Remarks on Internet Freedom’ speech by the US Secretary of State, Hillary Clinton:

“on Monday, a 7-year-old girl and two women were pulled from the rubble of a collapsed supermarket by an American search-and-rescue team after they sent a text message calling for help … Americans eager to help in the aftermath of a disaster and the girl trapped in the supermarket are connected in ways that were not even imagined a year ago, even a generation ago. ” (Clinton 2010)

While this was not actually a case where the Internet played a role, the comments about being connected in new ways is true. On the day of Clinton’s speech (21st Jan, 2010) there were more than 1,000 interactions on the Mission 4636 chat, but not one of them from any of the international organizations that would later receive credit for the initiative. So while the attribution was incorrect, the evaluation of the importance of connectivity was spot on—perhaps more so than has been realized to date.

(In the case of this example of people at the Supermarket, they texted their families, not international aid workers. It was announced via radio within Haiti, but it is not clear whether this lead to the search and rescue efforts there. The later appearance of this story on other media or communication channels was not on the critical path of information to rescue.)

It is more than simple misattribution for the actual emergency responders to make mistakes like this. As Tapscott and Williams note, the aircraft carrier USS Carl Vinson, off the coast of Haiti, would later ask Ushahidi Haiti to help identify coordinates. The USS Carl Vinson had millions of Haitians nearby, and hundreds of Haitians online, and yet they made this request of a dozen online non-Haitians. The median turnaround time for processing a message by Ushahidi Haiti was about 5 h, compared to just 5 min for the predominantly Haitian volunteers in Mission 4636 (see Fig. 5). That an aircraft carrier could (and would) reach out to unknown parties to augment its own information processing is itself positive and indicative of social change, and that Ushahidi Haiti was there to help was also positive, but obviously it would have been far more optimal to ask those with local knowledge to help in mapping. The systematic exclusion of Haitians from the story of how they helped their own recovery, from wherever they were in the world, is therefore corrected here. Tapscott and Williams (who also leave out the contributions of the Samasource/FATEM workers within Haiti) continue:

“The story of how Ushahidi got started, and where it’s gone since, reveals a great deal about a powerful new form of economic and social innovation that is sweeping across all sectors.”(Tapscott and Williams 2010)

This mostly reveals a great deal about Tapscott and Williams’ failure to understand social innovation. Following the violence in Kenya, a group of Kenyans came together through social media to share information and affect a better recovery for their family and friends. Following the earthquake in Haiti, a group of Haitians came together through social media to share information and affect a better recovery for their family and friends. There is nothing innovative about a crisis-affected population helping itself—they have always been the true ‘first-responders’—and the fact that around 5 % of the messages went through software with the same name is irrelevant. Footnote 18

Another potential reason for the inaccuracy in reporting so far is because military relief efforts (when not part of the United Nations’ own military) are sometimes considered outsiders by other relief organizations, with tensions existing between military relief efforts and those from large organizations like the UN and the Red Cross, with reports of arguments between the two in the early days of the Haitian response (Chan and Crowley 2011). In this case there was a physical separation too, with much of the information management for the military efforts centered in Miami or on ships off the coast of Haiti. As the military were the main responders to messages sent to Mission 4636, the initiative was therefore more predisposed to being out of the spotlight of the other international relief organizations. An example of this can be seen in the largest infrastructure deployment for communications by international relief organizations, which was the US Military’s distribution of 50,000 hand-crank radios among displaced populations within the country (de 2010; Watson 2010). The radio-delivery operation has generally been left out of reports about communications within Haiti, even those primarily reporting about connecting people via radio, such as Nelson et al. (2010) and Wall and Chéry (2011). This is more understandable than the omission of the Haitian’s role in their own response, as it is at least partially due to the US Military’s own information policies, which, by default, kept information about operations private or announced operations only within their own media publications, which are not generally treated as sources of information by other relief organizations (de Leon 2010).

9 Conclusions and recommendations

Humanitarian organizations often talk about the importance of engaging local populations in development and rebuilding. Here is the proof. As part of Mission 4636, the Haitian population and those working most closely with them processed the greatest amount of information with the highest speed and an accuracy equivalent to international professionals.

Unfortunately, much of the humanitarian world almost missed this. To this day, the FATEM workers in Mirebalais, Haiti, have processed more crisis-information than any other crowdsourcing organization at a single location, in any crisis in any place in the world. Hopefully, the analysis presented here will cause some humanitarian organizations to rethink some current directions in information processing strategies and look at more ways in which they can help crisis-affected communities help themselves.

There is one overarching recommendation for the humanitarian community looking to leverage digital communications: engage the crisis-affected community directly. They are the quickest workers with the most in-depth local knowledge. They will be looking for work and even nation-scale projects will rarely put a dent in big aid budgets.

Crisis-affected populations will continue to use all digital technologies available to them to bootstrap their own recovery. The failure to engage crisis-affected populations in their own recovery will likely mean that response organizations will be sidelined in the future information management of recovery efforts. If they continue to focus only on other international actors, then they might not even notice.

9.1 Summary of recommendations

Here, a summary of recommendations from the lessons learned are shared, with the hope of providing a positive influence on people running or considering the use of crowdsourcing for humanitarian information processing.

9.1.1 Find and manage volunteers via strong social ties

Most people in Mission 4636 were found by directly engaging individuals who were already in the professional or social circles of those already involved. The same was true for the Open Street Map contribution to mapping which was primarily the result of existing members.

Calls for volunteers via popular media outreach initiatives did not contribute many dedicated volunteers, and there is evidence that the media attention on students in Boston was actually detrimental, attracting volunteers who, while well-meaning, may have contributed less resources than they used, undertaking training but quickly moving on. Either way, this more public initiative processed only a tenth of the information of the more private core Mission 4636 system, so publicity was certainly no gain.

Among the few volunteers with no prior connection to people within the initiative, those who remained the most dedicated had formed strong, new relationships within the initiative. These new relationships also provided a critical socializing function in a high-stress environment, so encouraging these ties aids the well-being of the volunteers as much as it helps the work itself.

9.1.2 Maintain a ten-to-one local-to-international workforce

This is a ballpark ratio based on the observed demographics. It is reasonable to conclude that people with local knowledge will always be much faster and more accurate in processing information from within a region, but there is still a role for international (or otherwise non-local) people, especially if they can interact with people who have the local knowledge.

The ten-to-one ratio comes from the observation by some of the interview responses in Morrow et al. (2011), where the Haitians working on Mission 4636 noted that there were occasions when there were too many international people asking questions, which ultimately held-up information processing. At less than a ten-to-one ratio, it might be a better use of resources for international workers to try to rally more local volunteers, or rally funds for local paid workers, rather than attempting to process information themselves.

If it is not possible to engage a substantial local workforce, this might be evidence that those with the vital local experience do not actually view the initiative as a worthwhile effort, which should prompt a critical analysis of whether crowdsourced information processing is an appropriate task in the given context.

9.1.3 Default to private data practices

There is no evidence that publishing information online led to any responses – all people and organizations said to have used the messages would have been able to access them through private channels. As the analysis here shows, Mission 4636’s partners did not have permission to publish the 4636 messages on a public-facing map (by their own conditions for publication), and this action resulted in privacy breaches. In a more high-risk conflict context this would have serious consequences for those people whose identities were exposed.

9.1.4 Publish in the language of the crisis-affected community

Where information about a crisis is published, publish it in the language(s) of the crisis-affected community. While Mission 4636 chose to keep specific details about the crisis-affected population off public websites, several partner organizations who did published information used English-language only platforms and interfaces. The majority of the Haitian diaspora do not speak English, so they were of limited value in this regard.

9.1.5 Do not elicit information for which there is not the capacity to respond

While this seems like common sense, it is understandable that someone closely connected to a crisis will want to immediately share as much of it with the world as possible. However, this impulse could easily lead to publishing information about at-risk populations, or storing that information on platforms with too low security. The natural impulse might not be the correct one here, and so careful data management strategies or direct intervention/education might be needed.

9.1.6 Do not elicit emergency response communications

While this can be thought of as part of the previous recommendation, it is important enough to emphasize separately.

The targeted emergency response component of Mission 4636 was very small, and very specific to the context: the existing reporting services had failed, the crisis-affected population was not predisposed to believing that a full 911-like service should exist, and the world’s largest military were already nearby and inviting emergency response requests with a mandate to respond. We expected the response to aid displaced populations (many of us had worked with refugee communities and knew the importance of citizen reporting as a first signal) but the targeted medical responses to reports like head-traumas and births were a surprise. A welcome surprise, of course, but beyond our initial expectations and very dependent on the particular context.

In general, relief organizations do not make targeted responses during large-scale disasters, except to specific time-critical emergencies like search-and-rescue operations. Even in the case of Mission 4636, we still carefully deliberated over whether to pursue this part of the initiative and purposefully did not publicize Mission 4636 as an emergency response service within Haiti.

This is in stark contrast to a number of cases since where people have launched public ‘crisis-maps’ and begun inviting emergency response requests without a connection to those who are able to respond—a disturbing trend that needs to be curtailed.

9.1.7 Use social media to encourage the centralization of information

In all current popular social media platforms, it is difficult to date or verify information, let alone filter those with firsthand information from those reporting information from other media. In some cases, social media will be a useful tool for finding people with specific knowledge and skills. In other cases, including some hostile conflicts, the best strategy might simply be to tell people not to use social media.

If people use social media as part of their everyday communications it is natural that they will share information about a crisis through the same channels. However, rather than attempting the complicated task of trying to analyze social media communications en masse, it might be simpler in many contexts to use the same channels to let people know about the appropriate methods to report information to, as we attempted in Mission 4636. It may be ambiguous as to whether someone actually witnessed some event or are simply repeating information heard elsewhere, but when invited to fill out a form that explicitly asks someone to state how they learned about an event, most people can be expected to be honest. By centralizing the data, that person can also take on the responsibility of deciding whether or not a given report is already known (as was often the case when adding people to Person Finder) thus taking care of part of the duplicate detection problem.

If reporting channels do not exist, then social media can be used as an opportunity to more generally educate people about the implications of sharing potentially sensitive data in highly-visible platforms.

9.1.8 Establish partnerships with technology companies

There is no need for aid agencies to mediate information management via technical NGOs, or for information management professionals to engage technical NGOs rather than working directly with aid organizations. Just because an NGO puts the word ‘crisis’ or ‘disaster’ into their title or software, it does not guarantee that it will be the appropriate software.

As this article shows, the majority of the technical support for Mission 4636 was by Silicon Valley-based engineers, not the technical-NGO community as erroneously reported in almost all previous reports about Mission 4636, so it is a myth that technical-NGO’s played a large role in the first place.

9.1.9 Avoid partnerships with media organizations and citizen journalists

The need to keep information private seems to be too much in conflict with the desire of media organizations to publicize their involvement in response efforts. Footnote 19 The two organizations partnering with Mission 4636 who were most closely tied to mainstream media, Ushahidi and Thomson Reuters Foundation, were the only partner organizations who published 4636 messages online that contained personally identifying information, but at the same time apologizing for being too busy to provide people to help with the actual management of Mission 4636. The latter is understandable—to the extent that a crisis is a ‘media event’, it can be expected that humanitarian-focussed journalists are already at their busiest.

For these reasons, journalists who brand themselves as ‘humanitarian’ might actually be among the least suitable people to engage in information processing efforts for crisis response. First-hand reports from the crisis-affected population are likely to make for very compelling material for the popular media, so aid organizations might have to proactively try to keep these kind of journalists out of the crowdsourced relief efforts that tend towards open engagement.

The same is true for citizen journalists, whether they refer to themselves as ‘journalists’, ‘bloggers’, ‘crisis-mappers’, ‘independent documentary makers’, or any other popular label at the time. The desire to publicize their own efforts might not align with the objectives of processing information about at-risk populations as securely as possible, and while well-meaning, their dual-objectives could ultimately result in more harm than good. As with the media organizations, it might be necessarily to proactively monitor for individuals like these among volunteer populations and try to keep them out of the critical information processing efforts.

9.1.10 Integrate, don’t innovate or disrupt

Some commentators have championed how new technologies are ‘disrupting’ crisis response, pointing to Haiti and Mission 4636 as an example. Again, this is an erroneous report that seems to originate with technical NGOs that are trying to market their own technology.

The terms ‘innovative’ or ‘disruptive’ are borrowed from the software development community and refer to changing the market conditions rather than the technologies themselves. As this article shows, the technology we used was almost all 10 years old at the time. Within Haiti, we tried to keep the system as inline as possible with the reporting systems that the radio stations were already running. When interfacing with the response organizations, we attempted to create data and reports that met their requirements as closely as possible.

The microtasking approach was certainly novel (for humanitarian work) but this was a black-box for the relief community—unstructured information entered the system in Haitian Kreyol and came out structured with an English translation. We were not asking relief workers to take time to understand and interface with any internal processes. In short, we used technology to integrate, not disrupt.

9.1.11 Employ people with close ties to the crisis-affected region

By employing the FATEM workers within Haiti, we overcame many of the concerns about crowdsourced information processing: it gave us a reliable workforce which meant that we could accurately predict the throughput; they worked at one physical location and could therefore more easily exchange information with each other; the identities of all the workers were known which addressed some of the security issues.

As a professional group working together, the workers in Haiti also required less management, which means that it can be a cheaper option. The rate we paid FATEM workers came to about $0.50 for every message translated, categorized and geolocated. If a crowdsourcing initiative was only processing a few thousand messages/reports over series of weeks, then it would be more cost-effective to pay professionals to do this rather than commit an additional person to volunteer management. In addition to saving time, it would address some of the reliability, information exchange, and security concerns, ensuring paid work for the people that needed it most.

9.2 Closing remarks

As with the initiative itself, this paper is in debt to the hard work and selflessness of the volunteers and workers of Mission 4636. I thank all of them for their careful thoughts and contributions. I have not seen a single person among them complain about attribution in light of the misleading reports to date, only expressing gratitude for the efforts made by international partners. This has only increased my motivation to highlight their crucial role.

In addition to the people mentioned in the extracts here, there were dozens more who deserve equal recognition: Aline, Gwyn, Jackie, Jean-Robert, John, Karina, Karine, Maeva, Raphaël, Shell and no doubt many others. As with any crisis, the most important actors will be people like these, helping their communities bootstrap their own recovery. Hopefully, the proliferation of digital technologies will continue to give us more ways to help them.