Information and Communication Technologies (ICTs) to Provide Agricultural Advice to Smallholder Farmers: Experimental Evidence From Uganda

Agricultural advisory services generally rely on interpersonal knowledge transfers in which agricultural extension agents visit farmers individually or in groups to provide information and advice. This approach is not always effective and has often proved hard to bring to scale, particularly in highly dispersed smallholder farming systems. Information and communication technologies (ICTs) have been advanced as a promising way to overcome these problems associated with information delivery. We evaluate the effectiveness of an ICT mediated approach to deliver agricultural information in a field experiment conducted among small-scale maize farmers in eastern Uganda. The approach consists of three complementary technologies: First, we investigate the effectiveness of video as a means of delivering information, transferring knowledge, increasing technology adoption, increasing productivity, and improving well-being. Second, we quantify the additional impact of augmenting video with interactive voice response (IVR) technology — a more demand-driven approach to information provision. Third, we estimate the additional effect of time-sensitive short message services (SMSs) that remind farmers about key agronomic practices and technologies. We find that video is effective in delivering information, with households that were shown short videos on how to become a better maize farmer performing significantly better on a knowledge test, applying more of the recommended practices, and using inputs more effectively than households that did not see this video. These same households also reported maize yields about 10 percent higher than those that did not see the video. However, the incremental effects of IVR and SMS technologies were found to be limited.


Information and communication technologies
(ICTs) to provide agricultural advice to smallholder farmers: Experimental evidence from Uganda

Introduction
With new possibilities offered by information and communications technology (ICT), an abundance of pilot projects and commercial products have entered the market with the promise of revitalizing agricultural extension in developing countries. However, a growing body of evidence suggests that not all ICT-mediated extension approaches are equally effective in achieving outcomes such as increased knowledge and awareness, higher rates of technology adoption, better crop yields, higher farm incomes, or improvements in household welfare. This may be because seemingly small design attributes affecting the channels through which the information is delivered can significantly influence these outcomes. This paper investigates the effectiveness of an ICT-mediated extension approach designed to provide information about improved maize cultivation practices to smallholder farmers in Uganda. We consider the specific ICT channel through which the information is delivered to the farmer, and differentiate between (i) short audiovisual messages viewed on tablet computers that provide detailed information on how to improve maize cultivation; (ii) an interactive voice response (IVR) service that farmers call into if they need additional information on how to improve maize cultivation; and (iii) short message services (SMSs) that remind farmers at particular points during the maize growing season about key farming practices they should apply, and encourage them to seek more information through the IVR service. The outcomes used to assess the relative effectiveness of these different ICT channels include increases in individual-level knowledge about improved maize cultivation, and changes in household-level indicators such as the use of improved inputs, technologies, and practices. We also look at the effect of the three different ICT channels on farm production outcomes, maize utilization, and household-level welfare. We draw on a field experiment in which farmers are randomly assigned to a group that received a particular combination of ICT-mediated information interventions, and are then compared to a group that did not receive that particular information intervention (de Janvry, Sadoulet, and Suri, 2017). We work with approximately 4,000 farm households sampled from the population of households cultivating maize across five districts of eastern Uganda. The first intervention takes the form of a short video that explains simple yet effective ways to increase maize productivity and profitability, such as recommended practices on row spacing and seed rate, the importance of quality seed, optimal soil fertility management, and farm budget management. To test the overall effectiveness of the information provided through the video medium, we compare outcomes of farmers who were shown the informational video to outcomes of farmers who were exposed to a placebo video. In addition, we provide complementary access to an IVR system to about twothirds of the sampled households who were shown the informational video and, within that group, we send about half of the sampled households a series of eight follow-up SMS messages at strategic points in time.
We find that providing information to farmers through short videos significantly increased their knowledge about modern inputs, improved technologies, and recommended practices. In addition, farmers who were shown the video were also more likely to adopt a range of inputs, technologies, and practices that were promoted in the videos. We also find clear effects on production, with maize yields 10 percent higher than in control households. However, these results do not translate into greater well-being. We find that the IVR generated no additional impact on knowledge, nor did it have a strong influence on technology adoption. We only find a positive effect of IVR on the uptake of hybrid seed. While the SMS reminders did increase the likelihood that farmers used the IVR service, there were no additional effects on other outcomes.
The remainder of the paper is organized as follows. The next section lays out the main research questions and explains how the study fits into the broader literature. Section 3 explains the experimental design and corresponding sampling frame. Section 4 describes the study site and context, while Section 5 describes the interventions used as the treatments in the field experiment. We then turn to the results in Section 6 with a discussion of baseline characteristics and sample balance. This is followed by analysis of the impact of the three technologies tested, with subsections for impact on knowledge, practices adopted and inputs used, production, utilization, and welfare. A final section provides concluding remarks and recommendations for future program and research investments in ICT-enabled agricultural extension and advisory services.

Research Aim
The primary focus of this study is to test alternative ways in which information is provided to farmers through ICT channels to improve the effectiveness of agricultural extension services. In doing so, we aim to contribute to the literature on how ICTs can increase the adoption of modern inputs and improved crop management practices among smallholder farmers-a topic that has received considerable attention in recent years but has lacked sufficient evidence on which approaches work and for whom (Nakasone and Torero, 2016;Aker, 2011). The role of ICTs in improving access to information on crop management-as compared to improving access to market and price information-is a particularly challenging topic. Information of this kind is often non-excludable and non-rival in nature, therefore often under-supplied by the private sector and requiring public sector intervention . Public sector intervention in this area typically takes the form of information campaigns, extension programs, and advisory services, many of which have been met with mixed success (Feder et al., 2010;Birner et al., 2009;Haug, 1999;Davis, 2008). These ambiguous outcomes are often associated with specificities of the technology, practice, or principle being promoted; the agroecological and socioeconomic context in which it is being promoted; and other confounding factors that may be poorly understood or too costly to address in program design and implementation. For example, the fact that many of the potential beneficiaries of an extension services live in highly fragmented settlements in remote areas that poorly connected by road infrastructure often makes extension approaches such as the Training and Visit (T&V) model or Farmer Field School (FFS) model too costly (Bindlish and Evenson, 1997;Feder, 2006;Waddington et al., 2014). As a result, ICT solutions for extension such as SMS reminders, purpose-built smartphone apps, IVR systems, callin hotlines, portable tablets and projectors, and other hardware and software applications have received considerable attention in recent years.
Several studies have reported positive impacts of ICTs applied to exten-sion services. For example, Cole and Fernando (2016) found that the introduction of a toll-free hotline, through which farmers can ask questions to agricultural experts, significantly increased cumin and cotton yields among farmers in Gujarat, India. Casaburi et al. (2014) found that in Kenya, sending SMS messages with agricultural advice to smallholder sugarcane farmers increased yields by 11.5% relative to a control group with no messages. Fu and Akter (2016) found that a multi-media mobile phone-based product linked to expert advisory services increased farmers' awareness and knowledge about specific solutions to their production constraints in Madhya Pradesh, India. Maredia et al. (2017), on the other hand, found that while mobile phonebased animated videos shown to farmers in Burkina Faso induced learning and understanding, it was no more effective in encouraging adoption than conventional approaches to information provision. In Uganda-the site of our study-Grameen Foundation had some success using smartphones to provide agriculture-related information to farmers through ICT-empowered community knowledge workers (Van Campenhout, 2017). Our study compares three technologies, each with their own characteristics. In our context, video is an ICT intervention through which information is made available to the participating farmer on a limited number of occasions (we showed the video only twice) and at moments that are decided upon by the provider of the information (we showed the video before the maize planting season and early during planting). The strength of video is that it can combine audio and visual information in an attractive way that is recognizable to the farmer. For instance, we use local farmers as it has been found that information may be more effective if it is brought by someone one can identify with. We hypothesize that the video-mediated approach may be particularly effective in disseminating information about new technologies and practices. However, we also note that the supply-driven nature of video and the management of its use by the information provider rather than the farmer, is a potential constraint to its effectiveness.
Next, we consider an IVR service. IVR services and hotlines are demanddriven approaches based on the assumption that farmers can identify their information needs and are capable of actively searching for additional information 1 . One of the main advantages of this approach is that the informa-tion is supplied in a channel that is continuously accessible to the farmer, and can be used on a more timely and relevant basis. Such services also tend to be less expensive than video, since an extension agent does not necessarily have to physically meet and interact with a farmer to transfer information 2 . Uganda has considerable experience with demand-driven extension: its National Agricultural Advisory Services (NAADS) was build on a consulting business model that sought to create a more responsive advisory service (Benin et al., 2011). This extension approach may be more effective when farmers' key information constraints are associated with unanticipated shocks such pests, disease, or adverse weather conditions (Cole and Fernando, 2016).
Finally, we consider SMS reminders. SMSs are often framed as a supplydriven behavioral "nudge" designed to remind people (with some considerable degree of persistence) to make decisions or take actions toward some particular objective. Timely reminders have often been found very effective in overcoming inertia, procrastination, competing obligations, or simple forgetfulness of human beings (Sunstein, 2014). In our context, SMSs are designed to remind farmers on a well-timed recurrent basis about important aspects of maize farming. However, SMS messages are short and can not be used to provide new and complex information: in the present study, they are only used to make the information that was provided in the video more salient and to encourage farmers to use the IVR.

Experimental Design and Corresponding Sampling Frame
The three technologies described above are tested within the framework of a field experiment 3 . The experiment has four treatment arms, and the IVR and SMS treatments are incremental in design. The experimental unit is 2 In fact, it may also be possible to simply stream video to mobile phones of farmers. That said, this would likely be more expensive than IVR since more data need to be transferred.
3 This study is part of a larger study that also looked at the role of gender in video mediated agricultural extension. The overall study took the form of a 3 3 factorial design (plus a separate pure control group), where one factor corresponds to the technology used and the other two factors varied the gender of the person to whom the video was shown within the household and the gender of the person who provides the information in the video. In this study, we restrict attention to the first factor. More information on the 6 Electronic copy available at: https://ssrn.com/abstract=3305341 the household. A total of 3,703 households were shown a video, while 256 households (our control group) were shown a placebo video. From those 3,703 households that were shown a video, 2,414 also received an IVR starter kit: a flyer containing the IVR number and instructions on how to use it. From those 2,414 households that were shown a video and received the IVR encouragement, 1,113 households were also allocated to the SMS treatment. This design, together with the sample size in each treatment category, is illustrated in Figure 1.
To evaluate the effectiveness of these different ICT channels, we start by comparing average household-level outcomes among households in the control group to average outcomes of households that were shown a video. Ex ante, we expected the largest effects to be found between the video group and the control group and power calculations indicated that we only needed about 250 observations in each group to detect such effects 4 . This gives us the average treatment effect for the video intervention. To obtain the additional effect of the IVR treatment, we compare average outcomes of households that were shown a video and received the IVR intervention to households that were shown only a video. Finally, the additional effect of the overall study can be found in the pre-analysis plan, which is pre-registered and publicly available from the American Economic Association's registry for randomized controlled trials (AEARCTR-0002153).
4 Power calculations were bases on an elaborate set of comparisons using different outcomes to power the complete 3 3 factorial design. We used simulation techniques that allowed us to to sample from actual data on outcome variables (maize yields obtained from Uganda National Household Survey of 2005/06) instead of from a theoretical distribution with an assumed mean and standard deviation. Apart from the sample size in the control group, sample size in other treatment arms are the result of binding constraints for minimal sample size needed to test differences in two other factors of the design. Detailed information on the power calculations can be found in the pre-analysis. SMS campaign is estimated by comparing outcomes of household that were shown a video, received IVR information and also received the eight SMS reminders to outcomes of households that are shown a video and received the information about the IVR system, but did not get the SMS reminders.

Context
We conducted the field experiment among smallholder maize farmers in Uganda. Maize is widely consumed throughout much of Uganda, yet its value-to-weight ratio is sufficiently high to also make it an important traded commodity. Therefore, efforts to increase maize productivity at the farmhousehold level are an important dimension of Uganda's strategy to increase food security and reduce poverty through both consumption and income channels. Yet, maize yields in Uganda are low when compared to neighbouring countries and global averages: While research station trials conducted in Uganda report potential yields of about 1.6 metric tons per acre (using only improved varieties without fertilizer application), data from the Uganda National Household Survey (UNHS) 2005/06 indicate that average maize yields are much lower at about 618 kg per acre for the main growing season (Fermont and Benson, 2011). Moreover, the UNHS data shows considerable variation in productivity across farmers, with the top 10 percent of farmers achieving yields in excess of 1.1 metric tons per acre. At the same time, the use of modern inputs such as inorganic fertilizer and adoption of modern management practices such as row planting is very low in Uganda. For example, the use of inorganic fertilizer is on average just 2.4 kg of nutrient per acre per year, compared to Kenya (75 kg/acre), Rwanda (70 kg/acre), and Tanzania (15 kg/acre).
Maize is especially important in eastern Uganda. We sampled from five districts in eastern Uganda known for their maize production: Bugiri, Mayuge, Iganga, Namayingo, and Namutumba. From this, we removed from our sample town councils and also two sub-counties that consisted of islands in lake Victoria. We used two-stage cluster sampling to obtain a representative sample of this population. In particular, we first selected parishes randomly and in proportion to the number of villages within each parish. In the selected parishes, all villages were included in the study. Within each village, we then listed all households, from which we randomly assigned households to be included in the study.
In eastern Uganda, there are two maize cropping seasons. For our study, we concentrated on the second maize-growing season of the year 2017, which ran from approximately August 2017 to January 2018 and is characterized by a shorter period of rainfall than the first maize-growing season. During this second season, the complete cycle from planting to harvest requires 3 to 3.5 months, and farmers tend to cultivate early-maturing but loweryielding maize varieties as a result. Fields are prepared in August, planted in September, an harvested beginning in mid to late December and, at higher elevations, through mid January.

Interventions
The first intervention used in the study was a video that was shown on 10-inch Android tablet computers. Videos were screened by a trained field enumerator during a one-to-one meeting with either an individual farmer or the male and female co-head 5 . The control group received a placebo treatment, which was a music video of traditional dancing that contained no information related to farming or maize. Videos (treatment or placebo) were screened twice to the households in the sample, once before the time for planting maize (July 2017) and once around planting time (August 2017).
The information contained in the treatment groups' video is expected to positively influence maize yields by encouraging the adoption of several improved technologies and practices which, in turn, are expected to positively affect farm-household income. The topics included in the video script were obtained from qualitative interviews with key informants that were conducted in May 2017. The key informants included maize farmers, traders, maize breeders, extension workers, district agricultural officers, and other government staff and experts.
The main factors affecting maize productivity that were identified by these key informants were related to pests, poor soils, and poor seed quality. The three most important pests that affect maize farming are striga (in particular the Striga hermonihica variety), maize stalk borer (Busseola fusca), and, more recently, fall armyworm (Spodoptera frugiperda). We focus on striga management in the videos because it was deemed more immediately problematic to our population of interest than maize stalk borer, and because, at the time this field experiment was conducted, there was no consistent strategy being promoted to address fall armyworm. Striga, also called witchweed, is a parasitic plant that feeds off the roots of maize, leading to severe stunting of the maize plant. Because the nutritional needs of maize are greatest at the early stages of growth, weeding of striga becomes less effective in curbing losses as the season progresses. During the qualitative interviews, experts estimated that striga can reduce production from about 30 percent up to total loss of the crop.
No single method is effective in controlling striga. Progress has been made in developing varieties that are resistant to the herbicide imazapyr and can be supplied with a imazapyr seed coating to contain striga before it can damage the host plant. However, the seeds of this variety on offer in Uganda are expensive at 10,000 Ugandan Shillings (UGX) per kg, while seeds of other common or popular varieties cost between 6,000 and 8,000 UGX/kg, 6 and does poorly under dry conditions and in areas where the rains are short such as in eastern Uganda. 7 Crop rotation with sweet potato or beans can also reduce striga infestation, as can applications of phosphatic fertilizer to make up for the nutrients drained by the parasitic weed. Weeding is also a strategy to control striga, and is most effective in the weed's earliest growth stages, before it establishes itself on the maize plant's root system and before it has a chance to spread via seed dispersal.
Poor soils, exhausted by continuous cultivation with little rotation, is the second main limiting factor. Soils often lack macro-nutrients such as nitrogen (N), phosphorous (P) and potassium (K), and farmers in our study area rarely use organic or inorganic fertilizers in sufficient quantities to ameliorate these deficiencies, either at the planting stage (when diammonium phosphate (DAP) or nitrogen-phosphorous-potassium (NPK) applications are recommended), or at the vegetative growth stage (when top dressings of nitrogenous fertilizers such as urea are recommended).
Throughout eastern Uganda, maize farmers mainly rely on seed saved from harvest, which represents the third limiting factor identified by key informants. The reliance on saved (recycled) seeds tends to negate the significant yield advantages conferred by high-yielding maize hybrids, which require that fresh seed be purchased at the start of each season to realize the genetic gains conferred by heterosis or hybrid vigor. And even where farmers rely on recycled seed from improved open-pollinated varieties (OPVs) that do not require seasonal replacement, the gains from improvement tend to depreciate over three to four seasons, again requiring farmers to purchase fresh seed at regular intervals. Experts during the qualitative interviews differed in opinions with respect to the potential gains from the improved genetic qualities of both maize hybrids and OPVs: Yield effects were estimated between 30 and 200 percent, depending on climatic conditions and the complementary use of modern inputs and intensive management practices.
Finally, there are also crop management practices that affect yields, but the effects are generally smaller than those related to improved varieties, fresh seed, and inorganic fertilizer use. The most important are timely planting, optimal plant spacing, and timely weeding.
Timely planting in eastern Uganda implies that maize should be planted at the onset of the rains. However, many farmers delay planting for several reasons. For instance, they may not be certain that the rainy season has really started. There may also be bottlenecks in land preparation, where poorer farmers are often the last in line for the use of oxen or tractors. Experts estimate that for each day a farmer delays planting, yields are reduced by 1.5 percent.
Optimal plant spacing implies that sowing should be structured so that plants are not too close to each other such that they compete for light and nutrients which, in turn, reduces yields. Plants that are too far from each other means space is wasted, also leading to lower yields. Related is the number of seeds that farmers plant in each hill. Farmers want to be sure of germination and put more than one seed, sometimes up to 5. If they all germinate, there is competition for nutrients and sun, resulting in stunted growth. Usually there is also not enough thinning, or this is done too late when competition has already occurred. One expert told us that standard spacing is 75cm x 60cm with two plants per hill. However, several experts mentioned a new way of plant spacing of 75cm x 30cm with a reduced seed rate of 1 seed per hill which leads to a 35 percent increase in yields.
Timely weeding implies that the first weeding should be done at 18 to 20 days after planting, with a second weeding at two to three weeks after first weeding. From emergence to the "eight-leaf" stage (when the maize is at about knee height), the maize plant is a very poor competitor and unwanted stress such as weed competition should be prevented. A third weeding at the tasseling stage is optional, and depends on observed weed pressure after the second weeding. Given the above, a significant portion of the videos focuses on providing technical information on seed choice, soil nutrient management (including the promotion of both organic and inorganic fertilizer application), weeding (with particular attention on fighting striga), timely planting, and plant spacing. 8 We made sure to include information that is likely to be unknown to the farmer, based on findings from other studies that information is most valuable when individuals learn about a new technology or institutional innovation (eg. ?). However, other studies also provide evidence of behavioural change occurring through the compounding or re-emphasis of common knowledge that, through repetition, becomes more salient to the individual (Duflo, Keniston, and Suri, 2014). Therefore, the videos also contain information that farmers are assumed to know but do not seem to act upon.
However, not all constraints to maize productivity improvement are information deficiencies directly related to the use of inputs, technologies, and crop management practices. Often, missing information problems manifest indirectly, as uncertainty about the distribution of the farm-household's profit function. This includes uncertainty about the correlations between expected and actual returns, the inter-temporality of income streams, estimates of fixed and variable costs, hidden transactions costs, probabilities of adverse events, and a range of other variables that farmers may simply be unable to conceptualize or measure. Thus, a significant portion of the videos also focuses on the costs and benefits of the different technologies and practices being promoted. In addition, the videos encourage long-term thinking, advising farmers to (a) start small and grow their farm enterprise over time, and (b) combine technologies and practices together rather than investing all of their money and effort into one single input, practice, or technology. In short, the videos encourage farmers to experiment with combinations of technologies and practices on a small scale in the initial season before reinvesting their gains for larger-scale replication in subsequent seasons.
We also pay attention to how the information is packaged. For instance, prior studies have found that farmers find communicators who face agricultural conditions and constraints most comparable to their own to be the more persuasive than other communicators (BenYishay and Mobarak, 2014). Several studies point out the importance of role models on aspirations and future-oriented behavior (Bernard et al., 2015). A growing strand of the literature investigates how non-cognitive farmer characteristics such as aspirations, locus of control and self-esteem can lead to behavioural change such as technology adoption (Abay, Blalock, and Berhane, 2017). Therefore, in our video, the message is conveyed by individuals who are readily recognized as "peer farmers" and who provide information that is framed as a success story.
The video starts with the farmer-actors introducing themselves 9 . The farmer-actors talks about how they used to struggle with their maize production and how at one point in time, they decided things needed to change. It is shown how the farmers sell a hen, obtain a small loan from a friend, and use these proceeds to buy small quantities of improved seed and fertilizer from a local retailer. The video then shows how the farmers prepare the field by collecting and applying manure. Next, the video details how DAP is applied correctly, how maize seed are spaced at 75cm x 30cm intervals with 1 plant per hill, and what the correct timing of these activities should be. The video progresses to a field about 10 to 12 days after sowing when the maize has emerged from the soil, and the farmer-actors recommend to engage in gap filling to replace seeds that did not germinate, thereby preserving optimal plant density. The video proceeds to the field at 18 to 20 days after sowing, and the farmer-actors recommend how weeding should be conducted 9 As mentioned above, this study focuses on the analysis of only one of three factors in a 3 3 factorial design. A second factor in the study is related to the person who provides the information, differentiating between a singe man, a single woman, or a couple (man and woman). Therefore, the video intervention actually consisted of three videos that are essentially the same except for the fact that the person who provides the information varies according to the three factor levels and one of these videos was shown to the household. As in a factorial design, factors are orthogonal to each other, the likelihood that a household gets to see a particular video is the same across the three videos, and so we measure the average impact of the three videos. The videos can be found at https://is.gd/Vs8kUi. with particular attention to the identification of striga and the importance of follow-up weeding two to three weeks later. The video then advances to four weeks after planting, where the farmer-actor demonstrates how Urea top dressing is done and recommends one more round of weeding at the tasseling stage.
The video then turns to a discussion about investing to obtain higher yields over time. The farmer-actors point out that while it may be difficult to raise cash, one can always start small (e.g., on one tenth of an acre) and reinvest the returns. They explain that following recommended practices and using improved inputs led them to produce 2.5 bags of maize on a small plot, which they sold for UGX 125,000 with a profit of almost UGX 90,000-as compared to previously harvesting merely one bag from the same area at a profit of only UGX 40,000. They then explain that they reinvested the difference (UGX 50,000) to increase the area under intensive maize cultivation year after year such that, over time, they were able to cultivate an entire acre using modern inputs. In the final part of the video, they reiterate their experience and again encourage the viewer to try a similar approach.
Two other treatments were incrementally added to the video treatment. For these two treatments, the IVR system and the SMS reminders, we collaborated with Viamo, a social enterprise that aims to connect individuals and organizations using digital technology to make better decisions. We set up an IVR system that provided the same information as was recommend in the video. Farm households that were allocated the IVR treatment could call a toll-free number which explained the IVR system in their own language. The caller is then invited to select the number corresponding to the topic on which he or she wants more information (e.g., "Press 1 for seed selection, 2 for spacing and seed rate, 3 for soil nutrient management, 4 for advice on weeding, or 5 for pest management"). Depending on what number was selected, the IVR then played an audio message of a conversation between two farmers where one farmer is explaining the recommended practice to the other farmer.
For the SMS campaign, we recorded telephone numbers for mobile phones owned by the household head at the time of the experiment's rollout. Households that were allocated to the SMS treatment were sent eight SMS messages over the course of the two months following the first screening of the video. The messages all followed a similar structure: farmers were first reminded about the important technology or practice that was relevant at the particular time that the message was sent, followed by a reminder about the existence of the IVR service. For example, the first message, that was sent out around planting time, read "You will get much more maize if you use hybrid seed instead of recycled seed. Call the maize hotline on 0200522420 free for more advice!" About one month into the growing season, the following message was sent: "When your maize is knee high, apply 1 water bottle cap of urea around each plant. Call the maize hotline on 0200522420 free for more advice!". The other messages were similar in structure. All content was produced in the local language (Busoga).

Balancing Checks
While we did not conduct a dedicated baseline survey, we did ask questions prior to the experiment's rollout to investigate balance. The choice of variables was based on those variables that other researchers in similar studies used in their orthogonality tests. In particular, we looked at balance tables in studies that investigate the adoption of yield-improving technologies and practices using randomized controlled trials (RCTs), including Duflo, Kremer, and Robinson (2011), Karlan et al. (2014), Ashraf, Gine, and Karlan (2009) and Bulte et al. (2014). We collected household characteristics such as household size, age, and, education level of household head. We then asked more specific questions related to maize farming, such as acreage and quantities produced in the last season. Furthermore, we asked if the household received agricultural extension services, whether it used improved maize varieties, and whether it applied fertilizer to maize. We also collected data on housing conditions (number of bedrooms) and distance to the nearest agroinput shop. As mobile phone ownership is relevant to the IVR and SMS interventions, we also report balance on household access to and ownership of a mobile phone.
In Table 1, we provide descriptive statistics and balance tests for the comparisons between the three information delivery packages. Averages for the control group are reported in the first column. We observe that few households that were included in our study had access to agricultural extension in the previous year (about 11 percent). We also observe that only about 17 percent of households reported to have used any fertilizer in the previous season, and only about 34 percent reported to have used improved seed bought from  Note: First column reports control group means (and standard deviations below); Column 2 reports differences between placebo (control) and video treatment (and standard error below), column 3 between video only and video+ivr, column 4 between video+ivr and video+ivr+SMS; the last column is sample size; ***, ** and * denote that the difference is signficantly different from zero at the 1, 5 and 10 percent level, respectively.
a shop or agro-input dealer during the last cropping season. This suggests ample scope to increase intensification investments through extension. We also find that farmers produced on average only 268 kg of maize per acre in the first cropping season of 2017. This is substantially lower than the average yield of 618 kg per acre we find in data from the 2005/06 Uganda National Household Survey. The low yields may reflect the devastating impact of the fall armyworm outbreak and adverse weather conditions that plagued East Africa in 2017 (Stokstad, 2017). Balance is tested by calculating differences between treatment groups for each characteristic, and with a joint significance test (F-test) for each of the three ICT channels. The second column in Table 1, denoted "Video", compares baseline characteristics between households that were shown the placebo video (control group) and households that were shown an intervention video. For example, we see that yields prior to the intervention are about 17 kg per acre higher in the group that was shown the intervention video than in the group that was shown the placebo video. However, this difference is not significantly different from zero. In fact, for the placebo video versus intervention video comparison, none of the differences between baseline characteristics is significant at the 10 percent significance level, while the F-statistic can not be rejected. In the third column, differences in baseline characteristics between households that received the intervention video and households that received IVR on top of the video are shown (denoted "+IVR"). Here, we see that households in the latter group are significantly larger than households that only saw the video. They also have significantly more bedrooms and the household head is slightly older. However, we can again not reject the null that jointly, baseline characteristics are unrelated to the treatment group for this comparison. For the final comparison (comparing farmers that received video and IVR to those that additionally received SMS messages; reported in column four and denoted "+SMS"), we find that treatment household have slightly fewer bedrooms, but the figure is only significant at the 10 percent level and the joint test does not reject overall balance.
We find that about 84 percent of households have access to a mobile phone. This is encouraging, as the usefulness of IVR depends on access to a mobile phone. Further, we find that there is no difference in this percentage between the various treatment groups. The incidence of mobile phone ownership is also high, with about three quarters of households reporting they own a mobile phone. Again, this is important as the success of the SMS intervention depends on being able to receive the messages.
Following implementation of the field experiment, attrition within the sample was relatively low. Only 8.63 percent of the sample could not be tracked or persuaded to complete the endline survey. In addition, there were no indications that attrition rates were different between or among the different treatment groups: attrition was 7.03 in the control group; 8.74 percent in the group that was shown a video; 8.90 percent in the group that was shown a video and received the IVR encouragement; and 7.45 percent in the group that also received an SMS in addition to the IVR encouragement and the video. In Appendix Table A.1 we compare baseline characteristics of attritors and do not find any statistically meaningful differences in the observable characteristics of attritors by treatment status.

Impact on knowledge
We start with an elaborate analysis of the intent-to-treat (ITT) effects of the ICT channels. For the first channel, video, we had virtually no household that refused to watch, and so the ITT effect is equivalent to the average treatment effect of channel. This is not necessarily the case for the two other additional technologies. For instance, the IVR treatment consists of an encouragement to use the IVR system. We cannot ignore the fact that a large proportion of farmers that were encouraged to use the IVR ended up not calling the system, as they are likely to be a non-random subset of the households that were assigned to the IVR treatment, leading to a biased estimate of the IVR channel's impact. Also, for the third treatment, the SMS reminders, being allocated the treatment does not necessarily mean that the household also receives the SMS message. Here again, the probability that a particular household does not receive the intended treatment is likely nonrandom in nature. For instance, households that live further away from a mobile phone tower may be more likely to experience technical issues leading to delays or non-delivery of the SMS message. Even though we believe that the ITT analysis is the most informative when considering the experiment from a broader policy perspective, we also discuss local average treatment effects (LATE) below in an attempt to better estimate the effectiveness of the different channels itself.
We first estimate the ITT effects of the three different ICT channels on knowledge outcomes. Knowledge outcomes were measured with a short quiz (Feder, Murgai, and Quizon, 2004a,b;Masset and Haddad, 2015) consisting of four multiple-choice questions that were asked during the endline survey to each of the two spouses in the farm household separately. For each question, three possible answers were read out to the respondent, who was then asked to indicate which answer he or she thought was correct. The respondent was also allowed to indicate if he or she did not know the correct answer. The household was considered knowledgeable on a particular topic if at least one of the spouses indicated the correct answer.
The first question was related to planting. In our video, we recommended a spacing of 75cm x 30cm with 1 plant per hill, and this was the correct option. Other possible answers included a spacing of 75cm x 60cm with two plants per hill, which is standard for many farmers and recommended by many agricultural extension agents, and an intermediate alternative of 75cm x 30cm with 2 seeds per hill. Because our video recommended a technique that deviates from what is assumed to be standard spacing, we assumed that the recommended practice is new to most of the farmers. The second question is less related to technical knowledge, but more to viewing farming as a business. In the video, we paid ample attention to promoting an approach where farmers start small and grow over time by reinvesting, and we emphasized the benefits of combining inputs rather than investing only in e.g., improved seed. We wanted to see if farmers internalize this advice and ask what a successful farmer would do if he or she only has 40,000 shillings. The correct answer was to use this amount to purchase improved seed and fertilizer and start intensified farming on a small area. Alternative options were to: "use all the money to buy hybrid seeds, because without good seeds, yields will be low"; and "use all the money to buy fertilizer, because with poor soils, yields will be low". For the third question, we asked if farmers knew when weeding is most important. The video showed that weeding is most important during the first four weeks after planting, as maize is a poor competitor for light and nutrients. We assumed that most farmers would know the correct answer to this question given that weeding is part of well-established management practices in the study area. Alternative answers were: "when the maize is knee high" and "when the maize is at tasseling stage". Finally, we asked if farmers knew when spraying against fall armyworm is most effective. There was no information given about fighting fall armyworm in the intervention, so unless one of our interventions encouraged farmers to search for additional information, we do not expect an impact. For this question, the correct option was: "During the evening, as fall armyworm eats during night," while the other options were: "Early in the morning when it is still cool" and "At noon because sunlight increases chemical performance." To guard against over-rejection of the null hypothesis due to multiple inference, outcomes of the knowledge questions were combined into an index, constructed as the weighted mean of the individual standardized outcomes, using as weights the inverse of the co-variance matrix of the transformed outcomes (Anderson, 2008). However, we also see value in examining the impact on the questions individually, as they attempt to measure different aspects of the information intervention. To control the family-wise error rate (FWER) when examining scores for each question individually, we use re-randomization to construct the joint null distribution for the family of outcomes we are testing. From this family-wise sharp null, we can obtain the corresponding FWER-consistent significance thresholds by determining which cuttoffs yield e.g., 10 percent-, 5 percent-, and 1 percent significant hypothesis tests across all tests and simulations (Young, 2018;EGAP, 2015). Table 2 shows results for the three incremental levels of ICT-mediated information delivery using different dissemination technologies. The first column reports mean scores in the control group (with standard deviations reported below in parentheses). For the four individual questions, this is simply the proportion of households that answered correctly on the particular question. For instance, we find that in 16 percent of the households in the control group, at least one of the spouses indicated the correct option among the response alternatives to the question on optimal maize seed spacing. This relatively low rate of correct responses is due to the fact that this is a fairly new and potentially controversial recommendation. We find that in about 91 percent of households, at least one spouse knew inputs were best combined and in more than 95 percent of households it was known that weeding is most important during the first 4 weeks. In about one third of the households, at least one spouse knew how to fight fall armyworm. For the knowledge index, the mean is harder to interpret, as it is the result of a weighted mean after standardization of the individual components of the index.
In the second column, we report the impact of having been shown the video (with standard errors of the estimated coefficient reported below in parentheses). We find that having been shown the video increases the likelihood that at least one individual knows the recommended spacing by 13 percentage points and this difference is significantly different from zero at the 1 percent FWER-adjusted significance level (randomization inferencebased p-value are shown in the third column; asterisks denote significance as compared to FWER-adjusted thresholds). We also find that the video 20  Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between placebo and video treatment (and standard error) with its corresponding p-value in column 3; column 4 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 5; column 6 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 7; sample size is reported in column 8. Reported p-values are based on randomization inference (10,000 permutations); ***, ** and * denote that the difference is signficant at the 1, 5 and 10 percent level, respectively, after correcting for multiple hypothesis testing using a family-wise sharp null (10,000 permutations). All specifications control for the other orthogonal factors in the factorial design.

21
increased the likelihood that at least one spouse indicates that inputs are best combined for optimal results by 4.5 percentage points. This difference is statistically significant at the 5 percent FWER-adjusted significance level. For the question on weeding, the intervention does not seem to have had a significant effect. However, this result should be interpreted with care due limited variation in the outcome. 10 Finally, on the fall armyworm question, we find that households that were shown the video are no more likely than control households to know when one should spray to control the pest. This suggests that the videos did not encourage farmers to actively search information on important challenges that were not explicitly covered in the video. Overall, and as confirmed by the knowledge index, we conclude that the agricultural extension videos increased knowledge at the household level, and that this increase seems especially poignant for novel information provided in the videos. The fourth column (+IVR) shows the additional (ITT) effect of IVR (with standard errors of the estimated coefficient reported below in parentheses, and corresponding randomization inference-based p-value in the fifth column), on top of the video effect reported in the second column. We see that being provided with an IVR encouragement does not additionally affect knowledge about the new recommended spacing. There is some indication that the IVR treatment reduces the likelihood that at least one spouse responded correctly on the question about how inputs should be combined, but the reduction is not significantly different from zero after taking multiple inference into account. Similarly, there is no additional effect on knowledge related to the optimal time for weeding and also no additional effect on knowledge about the best way to fight fall armyworm. The fact that there is no supplementary effect of the IVR encouragement on knowledge is confirmed by the non-significant difference in the knowledge index.
The sixth column (+SMS) reports the additional effect of the SMS reminders on the various questions and the index (with randomization inferencebased p-value are reported in the seventh column). Similar to the impact of IVR, we do not find an additional effect of the SMS campaign on any of the questions. We also do not find an effect of the SMS campaign as judged by the knowledge index.
It may be that the null results for both IVR and SMS channels are due to the fact that we average over both households that have access to a mobile phone and those who do not. As the IVR and SMS treatments are likely to be most effective if households have access to or own a mobile phone, averaging over all household may dilute the effect, making it more difficult to detect a significant difference with the control group. Therefore, we take a closer look at conditional treatment effects. In particular, for the IVR campaign, we reestimate the impact on knowledge again, but only for the 3,002 households that reported to have access to a mobile phone (either because someone within the household owns a mobile phone or because they can use the phone of a neighbor or relative if necessary). For the SMS reminders, access to a mobile phone is likely not sufficient. To get the most benefit from the SMS reminders, households need to own a mobile phone. We therefor reran the analysis for the subset of 2,736 farm households that reported owning a mobile phone. However, results (Table A.2) are very similar to the results obtained from the analysis on the entire sample, with no additional effects from neither IVR nor SMS reminders on knowledge outcomes. Furthermore, especially for the IVR encouragement, the analysis above estimates the effectiveness of the intervention in increasing knowledge, adoption, and production outcomes. It does not provide an estimate of the efficacy of the IVR treatment itself. To get to the effect of the IVR system itself, we need to consider compliance: did farmers also use the IVR system? For the IVR treatment, compliance was measured by comparing the phone numbers extracted from the IVR call log with the phone numbers that were given to us by the farmers at baseline. We find that only a small number of households that were encouraged to use the IVR system also actually called in (8.9 percent or 214 households) 11 . In addition, we also found that 22 households that did not receive the encouragement called the IVR system. Compliance was also imperfect for the SMS treatment. For the SMS reminders, the system reported which households received the message. Of the 1,113 households that were supposed to receive the reminders, only 860 (77.3 percent) did. We therefore also generate LATE estimates , where the IVR encouragement is used as an instrument for having called into the IVR system. Similarly, we report two stage least squares results where allocation to the SMS treatment is used as an instrument for having received the SMS message. Note that for the former, due to the potential for two-way non-compliance, we need both excludability and monotonicity, while for the latter, there is only one-way non-compliance, hence only the exclusion restriction is needed.
In Appendix Table A.3 we repeat the average scores for the control group in the first column for reference purposes. In the second and third columns, we report results of a two stage least squares (2SLS) regression where the endogenous variable, an indicator that takes the value of one if the household called the IVR, is instrumented by the IVR treatment allocation. We conclude that, based on the LATE estimates, calling the IVR system on top of having seen the intervention video did not affect knowledge outcomes. In the fourth and fifth columns, we report LATE estimates for the SMS reminders as results of a 2SLS regression where the endogenous variable, an indicator that takes the value of one if the household received at least one SMS message, is instrumented by the SMS treatment allocation. As for the ITT effect, we also find no additional effect of receiving SMS on top of the intervention video and invitation to the IVR as evaluated by LATE estimates.
There may be different reasons why we do not find additional knowledge effects of the IVR encouragement and the SMS reminders. First, the lack of additional effects may be due to the way the experiment was designed. As we wanted to rule out any possibility that the effects were driven by differences in the content, we made sure that no new information was given in the IVR system and in the SMS campaign. In fact, we were interested in testing if IVR is effective in mitigating the fact that video is a one-off intervention and farmers may forget the details related to new and complex agronomic practices. Also for the SMS campaign, the messages are too concise to transfer new knowledge. The aim here was to simply remind farmers that the moment was there to implement what they learned in the video, and encourage hotline, representing about 18 percent of encouraged households. Unfortunately, we did not enquirer if farmers called the hotline during endline data collection. them to call the hotline. In light of this, it may be less surprising that we do not find additional knowledge effects from the two interventions. The null result for the IVR treatment on knowledge is also consistent with Cole and Fernando (2016), who also do not observe differences between the treatment and control groups in agricultural knowledge outcomes from a hotline in either their midline or endline survey data. They argue that this was, to some extent, to be expected: as a demand-driven service, the types of knowledge that respondents gain reflect their actual demand for information. Also in our case, we did not test knowledge on the topics that were chosen most frequently by the farmers that used the IVR system (pests, diseases, land preparation, and seed selection) 12 . The results are also consistent with results from Casaburi et al. (2014), who find only an impact from their SMS messages if it is more likely that the farmers do not already know the information that is contained in them.

Adoption effects
We continue our analysis by examining the effect of our incremental treatments on household-level adoption of recommended farming practices. During the endline survey, we collected detailed information on practices employed on the household's maize plots. Results of mean adoption rates in the control group for different practices are reported in the first column of Table  3. For instance, in the video, we recommend to start planting maize immediately after the start of the rains. We find that 37 percent of households in the control group reported that they started planting one day after the start of the rains on at least on plot-a relatively high rate of adoption for this practice that indicates the extent to which it is likely known among farmers. We also find that only 2.6 percent of households in the control group used the recommended way of plant spacing of 75cm x 30cm with a reduced seed rate of 1 seed per hill. This low rate in the control group is not surprising given that this was a new and somewhat controversial recommendation. Almost 69 percent of control households report removing striga before it flowered to reduce damage early on and prevent the weed from spreading. Finally, we find that about 43 percent of control households reported first weeding after 18-20 days as recommended in the video.  Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between placebo and video treatment (and standard error) with its corresponding p-value in column 3; column 4 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 5; column 6 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 7; sample size is reported in column 8. Reported p-values are based on randomization inference (10,000 permutations); ***, ** and * denote that the difference is signficant at the 1, 5 and 10 percent level, respectively, after correcting for multiple hypothesis testing using a family-wise sharp null (10,000 permutations). All specifications control for the other orthogonal factors in the factorial design.
The second column in Table 3 again reports the difference in the adoption of practices between households that were shown the intervention video and households that were give the placebo treatment (with standard error of the estimate in parenthesis below and corresponding randomization inferencebased p-values in column 3). We find that for the first recommended practice, early planting, there is no impact from the video treatment. However, the likelihood that households adopt the recommended 75cm x 30cm spacing with a reduced seeding rate increases significantly after having been shown the video: while only 2.6 percent of households in the control group report adopting this practice on at least one plot, this figure increases to 8.9 percent among households that were shown a video (and this difference is significant at the 1 percent FWER-corrected significance level). Similarly, we find that the proportion of households that removed striga early on increased from 68.5 percent to 74.7 percent as a result of being shown a video (randomization inference-based p-value of 0.020, significant at the 10 percent FWER-corrected significance level). Finally, while the proportion of households that reported having started weeding after 18-20 days is almost 2 percentage points higher among households that were shown the video, the difference is not significantly different from zero. Estimation results for the index that summarizes the different practices confirms that overall, we can conclude that the videos significantly increased the adoption of recommended practices. Results suggest this is particularly the case if it concerns practices that are new and less likely to be known by farmers.
The fourth and fifth column of Table 3 report results for the additional effect of the IVR encouragement, with ITT estimates reported in column 4 and the corresponding randomization inference-based p-values in column 5. We do not find any significant additional effect of the IVR treatment on any of the recommended practices. The sixth and seventh column report results for the additional effect of the SMS campaign, with ITT estimates reported in column 6 and the corresponding randomization inference-based p-values in column 7. While we do find that the proportion of households that reports to be removing striga before flowering is 2.4 percentage points higher in the treatment group, and while we also find a small positive effect on seed spacing and seeding rate, the differences are not significant. The fact that the both the IVR encouragement and the SMS campaign has no impact on adoption of practices is also reflected in the index estimation results.
Similar to the analysis for knowledge outcomes, we repeat the analysis to assess the sensitivity of the results to potential treatment heterogeneity related to mobile phone access. In particular, we redo the analysis for the additional effect of the IVR encouragement, but only for households that reported having access to a mobile phone. Results are reported in columns 2-4 in Appendix Table A.4. In addition, we repeat the analysis for the additional impact of the SMS campaign, but restrict our sample to only the households that reported owning a mobile phone. Results for this are in columns 5-7 of Appendix Table A.4. Results are very similar to those reported in Table 3 and we conclude that our lack of results for the IVR and SMS treatments is not driven by treatment heterogeneity with respect to mobile phone access or ownership. In Appendix Table A.5, we account for non-compliance in the IVR and SMS treatments when looking at adoption. As in Table A.3, we use 2SLS and instrument callers with the IVR encouragement and recipients of at least one SMS message with treatment allocation to the SMS reminder treatment. For the efficacy of both treatments on agronomic practices, there is no impact.
Next, we examine results related to the use of modern inputs. During the endline survey, we collected detailed information on input use on the household's maize plots, specifically on the use of inorganic fertilizer and improved seed. Results are reported in Table 4. In the top panel of the table, we look at the three types of fertilizer that were recommended in the video separately, and again use an index to assess changes in overall fertilizer use. The first column in Table 4 reports mean adoption rates in the control group. We find that 26.4 percent of control households report that they used DAP or NPK on at least one of their maize plots, while the use of urea is less widespread with only 5.1 percent of control households reporting urea use on at least one plot. Among control households, 15.7 percent reported using organic fertilizer on at least one plot.
The impact of showing videos on fertilizer use is reported in column 2 and 3 of Table 4. We see that the video treatment reduced the use of DAP or NPK by 4.7 percentage points. This difference has an associated randomization inference-based p-value of 0.088. However, it is not significant after controlling FWER. We find the opposite result for urea use with estimates indicating an increase of almost 5 percentage points (with a randomization inference-based p-value of 0.011, and significance at the 5 percent level after controlling FWER). We also find that the use of organic fertilizer increased by about 50 percent as a result of the video treatment. Summarizing the three types of fertilizer in a index results in a positive difference between treatment and control, with the difference significant at the 10 percent level.  Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between placebo and video treatment (and standard error) with its corresponding p-value in column 3; column 4 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 5; column 6 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 7; sample size is reported in column 8. Reported p-values are based on randomization inference (10,000 permutations); ***, ** and * denote that the difference is signficant at the 1, 5 and 10 percent level, respectively, after correcting for multiple hypothesis testing using a family-wise sharp null (10,000 permutations). All specifications control for the other orthogonal factors in the factorial design.
The opposing effects of the video on different types of fertilizer, the inorganic types in particular, may be due to the fact that in the video, we pay particular attention the benefits of combining inputs and technologies. The video may thus encourage farmers to reassess their mix of fertilizers, reducing the use of fertilizer types that are already used widely and increasing the use of fertilizer types that are less commonly used. In the light of this reasoning, the fertilizer index may need to be specified differently, as more is not necessarily better for each type of fertilizer. Specifically, and in the light of the imbalance in the control group, one may argue that DAP/NPK use is higher than optimal and urea use is lower than optimal. As such, a case can be made to include DAP/NPK use in the index with a negative sign. Doing so, we conclude that the video intervention has a significant effect on the fertilizer mix used by farmers (with a randomization inference-based p-value < 0.001).
The incremental impact of the IVR encouragement is reported in columns 4 and 5 of Table 4. While the difference between treatment and control is positive for both types of inorganic fertilizer, the effects are not significant. However, we do find that the IVR encouragement reduced the proportion of households that reported using organic fertilizer by about 25 percent. This negative impact may again be explained by a change in the mix of fertilizer. In the context of a more demand-driven ICT channel, farmers that used organic fertilizer may seek information on more efficient types of fertilizers. If we judge the effectiveness of the IVR encouragement by including organic fertilizer use in the index with a negative sign, we do indeed find a significant impact (with a randomization inference-based p-value = 0.009). The additive effect of the SMS campaign is reported in columns 6 and 7 of Table 4. We find that the SMS reminders reduced the likelihood of using urea and increased the likelihood of using organic fertilizer, although none of the differences is significant.
In the lower panel of Table 4, we report results for the use of improved seed. We differentiate between maize hybrids and open-pollinated maize varieties. In the first column, we see that use of improved seed is more common than fertilizer use: in the control group, about 30 percent of households report using open-pollinated varieties on at least on plot, and this percentage is only slightly lower for hybrids. We do not find that the video treatment changes these percentages (columns 2 and 3). However, we do find an effect from the IVR encouragement: the percentage of households that reported to have used hybrid seed on a least one plot was 4 percentage points higher 30 in the treatment group. Apparently, providing farmers with a tool that allows them to actively seek out information about a new input or technology increases the likelihood that they also adopt it. The impact of the IVR treatment on improved seed use is confirmed by the seed index. The addition of SMS reminders, on the other hand, does seem to reduce the likelihood that hybrid maize seed is used (columns 6 and 7). However, the effect ceases to be significantly different from zero after controlling FWER. The additional effect of the SMS campaign on the seed index is also insignificant.
We again investigate robustness of the results for the IVR and SMS interventions by rerunning the analysis conditional on mobile phone ownership. The results from this analysis are presented in Appendix Table A.6. We find that for the subset of households that have access to a mobile phone, the IVR encouragement still reduces the likelihood of using organic fertilizer and increases the use of DAP/NPK (but not urea). If we include organic fertilizer use with a negative sign in the index, we still conclude that the IVR treatment significantly affects fertilizer allocation (with a randomization inference-based p-value = 0.006). The addition of SMS reminders reduces urea use more in the subgroup of phone owners (with a randomization inference-based p-value=0.050), but the estimate is not significantly different from zero after controlling FWER. We also find that organic fertilizer use is slightly higher among households that were allocated to the SMS treatment. Based on estimates of the fertilizer index, there is also no additional effect of SMS on fertilizer use. The additional effect of the IVR treatment on seed use becomes insignificant if we restrict ourselves to the subset of farmers that have access to a mobile phone, while the negative effect of the SMS campaign on hybrid seed use now becomes significant at the 10 percent FWER-controlled significance level.
Results for the LATE estimates for inputs are somewhat more interesting than LATE estimates for agronomic practices (see Appendix Table A.7): We find a substantial reduction in the use of organic fertilizer if farmers make use of the IVR system. We also find a significant effect of having received at least one SMS message. The effect of the SMS reminder reverses the IVR effect somewhat, but the effect size is very low. Further, we find that farmers that called the IVR line are much more likely to use hybrid seed. Also here, the effect is reversed by the SMS treatment.

Production effects
We now turn to the intervention's effects on production-related outcomes. We first examine household-level maize production. During the endline survey, we asked both spouses separately to estimate how much maize was harvested from each plot. These quantities were then summed over the different maize plots assessed by each spouse and the average between the two spouses was taken as the final estimate of household-level maize production. We find that overall, a typical household in our sample produced about 500 kg of maize on an average of 1.16 acres, with yields estimated at 530 kg per acre.
The first column in Table 5 shows mean values for the production indicators for the control group. On average, control households produce (log(kg)) 5.814 or about 440 kg of maize. This was cultivated on (log(acre)) 0.018 or about 1.19 acres on average. It also shows that for the average household in the control, maize yields (log(kg/acre)) equal 5.850 or about 430 kg/acre. This is much higher than yields recorded at baseline (290 kg/acre) for the previous season, possibly because rainfall patterns were better during our intervention and farmers may have given greater attention to combating fall armyworm. Still, compared to figures recorded in FAOSTAT (1000 kg/acre) or figures from household survey data (typically around 600 kg/acre), yields were below what might be considered "normal" in Uganda. This is also confirmed by the fact that in less than 40 percent of households at least one spouse reported that yields were better than a typical year on at least one plot. We also look at labour use. The average household spent about 72 person-days on maize farming, which includes labour that was hired in. This translates in a labour productivity of 6.73 kg of maize per hour worked.
In the second column of Table 5 we report the impact of the video treatment (with corresponding randomization inference-based p-values in column 3) for the various production related outcomes. We see that there is no impact on (log) maize production. However, we do see that households in the video treatment produced this same amount of maize on an area that is about 9.6 percent smaller than the area used for maize production by control households. As a result, we also find that among households in the video treatment, yields are about 10.3 percent higher than among the control group, and this difference is significant at 10 percent after controlling for FWER. The fact that the video intervention has a clear effect on productionrelated outcomes is also reflected in the significant different in the production index between treatment and control. That said, we do not find that house- Table 5: Impact of ICT treatments on production outcomes Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between placebo and video treatment (and standard error) with its corresponding p-value in column 3; column 4 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 5; column 6 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 7; sample size is reported in column 8. Reported p-values are based on randomization inference (10,000 permutations); ***, ** and * denote that the difference is signficant at the 1, 5 and 10 percent level, respectively, after correcting for multiple hypothesis testing using a family-wise sharp null (10,000 permutations). All specifications control for the other orthogonal factors in the factorial design.
holds in the video treatment were more inclined to feel that yields are better than normal than control households. and with respect to labour, we do not find any effects at the extensive or intensive margins.
Results are again consistent with the content provided in the video. In particular, the video advised that farmers experiment on a small part of their field with modern inputs, following recommended practices, and advised against using improved seed on their entire field if this does not leave sufficient money for complementary inputs such as fertilizer. Further, the video advised farmers to pursue a more commercial mindset to maize cultivation, and paid ample attention to the idea of starting small and growing over time through re-investing.
Columns 4 and 5 report the additive effect of the IVR treatment on production-related outcomes. As with previous outcomes, there seems to be little impact from this treatment. The only exception is with respect to labour productivity, which is about 6.4 percent higher among households that received the IVR starter kit. However, while this difference has an associated p-value of 0.070, it is not significant after after controlling FWER. Similarly, we do not find additional effects on production related outcomes of the SMS campaign.
Next, we investigate if production-related outcomes differ if we restrict our sample to those households that have access to a mobile phone (for the IVR treatment) and if we restrict to households that own a mobile phone (for the SMS treatment). Results can be found in Appendix table A.8 with LATE estimates reported in Appendix table A.9. In both cases, we do not find additional effects of IVR nor SMS on production-related outcomes.

Impact on maize utilization
We next investigate if the intervention also have an impact on how households utilize the maize they produce. We look at three different ways that harvested maize can be used: for consumption by the household itself, for sale in local markets, or as seed saved for sowing in the next season. Results are presented in Table 6. The first column in the table shows that, not surprisingly, a large share of the households in the control group report positive amounts of maize production retained for consumed (83.5 percent). We also find a large share of household reporting that they sell part of their maize (64.3 percent of households). This is about twice the market participation rates commonly found in the literature, underlining the importance of maize for our study population as a source of income (Barrett, 2008). We also find that 85 percent of households save maize for use as seed in the next season, illustrating the prevalence of seed recycling and reliance on informal seed systems among smallholders (McGuire and Sperling, 2016).
The second column in Table 6 reports the impact of the video treatment (with corresponding randomization inference-based p-values in the third column). The video treatment's effect on the proportion of households reporting to have consumed maize is not significantly different from zero; neither is the effect on the proportion of households reporting have sold maize. However, the share of households that reported having saved maize for seed is 3.3 percentage points lower lower among households that received the video treatment, although the difference is not statistically significant. Despite this, the sign on the coefficient estimate here is consistent with the video's content that emphasizes the importance of using high quality seed and advises against recycling seed. When we combine the three types of utilization in an index, with maize seed saving entered negatively, we do find that utilization is affected by the video treatment (albeit only at a 10 percent significance level). The fact that the index is significant but the individual outcomes are not is due to the increased efficiency by ensuring that outcomes that are highly correlated with each other receive less weight, while outcomes that are uncorrelated, and thus represent new information, receive more weight (Anderson, 2008).
In the fourth and fifth column, we report the additional effect of the IVR encouragement. There are no additional effects of the IVR on home consumption, nor on the likelihood that households sell part of their maize production. We do find among households in the IVR treatment that the proportion that saved maize as seed for the next season was lower by an additional 2.4 percentage points. However, with a randomization inferencebased p-value of 0.113, this difference is not significant. The index also confirms that we can not reject the null of no additional effect from the IVR treatment. In the sixth and seventh column, results are presented for the additional ITT effect of the SMS reminders. We do not find any additional effect on maize utilization outcomes. In Appendix Table A.10, we again estimate conditional treatment effects, but results for the IVR and SMS treatments are not sensitive to phone access and ownership. Appendix Table  A.11 reports LATE estimates for the effect of calling the IVR help line and receiving SMS reminders on maize utilization. Also here, we do not find an effect on maize consumption, the probability that maize is sold or the  Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between placebo and video treatment (and standard error) with its corresponding p-value in column 3; column 4 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 5; column 6 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 7; sample size is reported in column 8. Reported p-values are based on randomization inference (10,000 permutations); ***, ** and * denote that the difference is signficant at the 1, 5 and 10 percent level, respectively, after correcting for multiple hypothesis testing using a family-wise sharp null (10,000 permutations). All specifications control for the other orthogonal factors in the factorial design.
likelihood that maize is saved for seed, nor on the index.

Impacts on household welfare
Finally, we examine the impact of the ICT interventions on measures of wellbeing at the household level. Welfare is captured though self-assessment and through consumption expenditure. We asked welfare-related questions to the main adult woman decision-maker within the household, as we assume she has a good idea of both consumption expenditure within the household and the general level of well-being. First, we asked if they feel that their household is better off (as opposed to equally well off or worse off) in terms of income and consumption than the average household in their community and relative to six months earlier. We find that among control households, 39.7 percent felt they were better off than the average household in their community and 38.8 percent felt they were better off than six months ago (Table 7). We also include two standard self-assessed food security questions taken from the Household Food Insecurity Access Scale (HFIAS) (Coates, Swindale, and Bilinsky, 2007). The first asks whether the household was generally able to eat its preferred food, and 41.8 percent of households responded affirmatively. The second asks whether the household is generally able to eat enough food, and 60 percent of households responded affirmatively. As a less subjective way to asses changes in well-being, we collected data on consumption. In particular, we asked the adult woman decision-maker in the household about the value of the most-consumed items in the household, including household own-consumption of farm production, over the period of one week. These questions included consumption of maize, cassava, sorghum, millet, rice, sweet potatoes, beans, fruits, groundnuts, vegetables, salt/sugar, cooking oil/ghee, soap, and airtime 13 . From this we then took the logarithm and trimmed at 5 percent. We find that log consumption expenditure is about 10.92 in the control group, which corresponds to about UGX 65,000 or just under USD 20 per household per week.  Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between placebo and video treatment (and standard error) with its corresponding p-value in column 3; column 4 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 5; column 6 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 7; sample size is reported in column 8. Reported p-values are based on randomization inference (10,000 permutations); ***, ** and * denote that the difference is signficant at the 1, 5 and 10 percent level, respectively, after correcting for multiple hypothesis testing using a family-wise sharp null (10,000 permutations). All specifications control for the other orthogonal factors in the factorial design. Columns 2 and 3in Table 7 reports average treatment effects for the video intervention. We find no evidence of impact on any of the welfare indicators. In the next two columns, we report ITT effects for the IVR encouragement. Also here, we find no effects. Finally, columns 6 and 7 report ITT effects for the addition of the SMS treatment. Again, we find no effects. We also report conditional effects for subsets of farmers with mobile phone access or ownership in Appendix Table A.12 and LATE estimates for the IVR and SMS treatments in Appendix Table A.13. Again, we do not find any additional welfare effect.

Impact of SMS campaign on IVR
Although the SMS treatment does not appear to have any effect on many of the outcomes discussed above, it is worth exploring whether the treatment had an effect on the use of the IVR encouragement. Recall that an important aim of the SMS treatment was to remind farmers that they can call the IVR system to get additional information on the practices they were being reminded about: each message ended with the instruction to "Call the maize hotline on 0200522420 for free to get more advice!". To this end, Table 8 shows that the SMS messages did seem to encourage participants to use the IVR. Restricting our attention to households that received the IVR encouragement, we see that among households that received the IVR encouragement but were not allocated to the SMS treatment, only 6.5 percent actually called the IVR line (column 1). This percentage was 6.1 percentage points higher among households that received the IVR encouragement and were also allocated to the SMS treatment (column 2 and 3). If we use SMS allocation to the treatment as an instrument for actually receiving the SMS messages (columns 4 and 5), the effect increases to 8.1 percentage points.

Conclusion
In this study, we evaluated the effectiveness of three complementary ICTmediated agricultural extension approaches that can be brought to scale relatively easily when compared to many of the more conventional extension approaches in use among smallholder farmers today. The first approach-the use of short, appealing video messages animated by farmer-actors who farmers can relate to-is probably the approach containing the most substantial quantity of relevant information. While appealing because of its combination of both audio and visual information, the approach is also somewhat supplydriven, assigning farmers to a passive role as the recipient of information. 14 The second approach-an interactive voice Response (IVR) service-allows the farmer to play a more active role in information acquisition by allowing the farmer to call into a system that offers menu-based choices leading to prerecorded message. The third approach-a series of SMS messages that reminds the farmer of particular key inputs or practices-provide a nudge to farmers that makes available information more salient or encourages them to seek out additional information, and has been shown to be effective in a variety of other contexts (eg. Johnson et al., 2016).
We test the relative effectiveness of these three ICT channels using a field experiment that involved nearly 4,000 smallholder maize farmers in eastern Uganda. To assess the effectiveness of the first channel,we compared outcomes of a random subset of farmers that was shown an informational video to a random subset of farmers that was shown a placebo video. From this initial treatment group, two-thirds of the farmers that were shown the video were randomly assigned to receive an IVR starter kit which encouraged them call into the IVR service to obtain information on maize cultivation that was consistent with the video's content. From this second treatment group, half were randomly assigned to receive a series of eight time-sensitive SMS reminders related to the recommended practices and technologies, along with a reminder to use the IVR service to obtain additional information. The design of this experiment allowed us to estimate the effect of the video treatment as well as the additional effects of the IVR encouragement and SMS treatments. Effectiveness was measured in terms of knowledge gained, practices and technologies adopted, production increased, utilization affected, and welfare improved. By taking into account non-compliance to being offered the services, we could estimate the additional treatment on the treated effects. We also considered treatment effects conditional on access to and ownership of a mobile phone.
Findings indicate that in our study site and context, video-enabled agricultural extension approaches affect a range of outcomes. Showing agricultural information videos to maize farmers increased knowledge outcomes, particularly on new practices and technologies, and increased the adoption of recommended practices, particularly those that were new and otherwise unknown to farmers. Videos also led to adjustments toward more balanced fertilizer use, with farmers increasing the use of certain types of fertilizers (urea and organic) and reducing the use of others (DAP, NPK). As a result, farmers that were shown agricultural information videos increased their maize yields by 10 percent . These results are thus the result from increases on the intensive, not extensive, margin. However, we did not find an impact on several measures of household well-being.
The IVR treatment offered in addition to the video treatment did not seem to additionally increase knowledge or adoption outcomes, although there is some evidence to suggest that farmers who received the IVR encouragement were more likely to use hybrid maize seed. We found no additional effect of being offered IVR on production, utilization, nor well-being. The SMS reminders provided in addition to the video treatment and IVR encouragement also did not have any additional effect. The results (or lack thereof) for the IVR and SMS treatments are robust to incorporating non-compliance and persist if effects are estimated conditional on mobile phone access and ownership.
One area that needs further attention is related to spillovers. It is well known that experimental studies of information treatments are especially prone to non-interference violations (Coppock, 2014). As such, our estimates are likely to underestimate the true effects of ICT-mediated extension approaches. On the other hand, there is considerable evidence that much of the learning among smallholders happens through peer learning (Bandiera and Rasul, 2006;Conley and Udry, 2010). Some studies have tried to understand how peer learning can be leveraged to accelerate agricultural technology adoption and diffusion (eg. Beaman et al., 2018;Magnan et al., 2015). The question then becomes which ICT-mediated extension approach maximizes spillover effects, for instance through ease of sharing of content. Future studies should be designed with such questions in mind, by for example en-suring exogenous variation in treatment saturation or by using a "multilevel" design where different levels correspond to different spillover networks and treatments are assigned so as to create variation in different levels (Baird et al., 2017).
Young, A. 2018. "Channelling Fisher: Randomization tests and the statistical insignificance of seemingly significant experimental results*." The Quarterly Journal of Economics qjy029. Note: First column reports control group means (and standard deviations below); Column 2 reports differences between placebo (control) and video treatment (and standard error below), column 3 between video only and video+ivr, column 4 between video+ivr and video+ivr+SMS; the last column is sample size; ***, ** and * denote that the difference is signficantly different from zero at the 1, 5 and 10 percent level, respectively. Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 3 and number of observations in column 4; column 5 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 6; sample size is reported in column 7. Reported p-values are based on randomization inference (10,000 permutations). All specifications control for the other orthogonal factors in the factorial design. Note: In the first column, means (and standard deviations) in the comparison group are presented for each variable. Column 2 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 3; column 4 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 5; sample size is reported in column 6. All specifications control for the other orthogonal factors in the factorial design. Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 3 and number of observations in column 4; column 5 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 6; sample size is reported in column 7. Reported p-values are based on randomization inference (10,000 permutations). All specifications control for the other orthogonal factors in the factorial design. Note: In the first column, means (and standard deviations) in the comparison group are presented for each variable. Column 2 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 3; column 4 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 5; sample size is reported in column 6. All specifications control for the other orthogonal factors in the factorial design. Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 3 and number of observations in column 4; column 5 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 6; sample size is reported in column 7. Reported p-values are based on randomization inference (10,000 permutations); ** and * denote that the difference is signficant at the 5 and 10 percent level, respectively, after correcting for multiple hypothesis testing using a family-wise sharp null (10,000 permutations). All specifications control for the other orthogonal factors in the factorial design. Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 3 and number of observations in column 4; column 5 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 6; sample size is reported in column 7. Reported p-values are based on randomization inference (10,000 permutations); ***, ** and * denote that the difference is signficant at the 1, 5 and 10 percent level, respectively, after correcting for multiple hypothesis testing using a family-wise sharp null (10,000 permutations). All specifications control for the other orthogonal factors in the factorial design. Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 3 and number of observations in column 4; column 5 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 6; sample size is reported in column 7. Reported p-values are based on randomization inference (10,000 permutations). All specifications control for the other orthogonal factors in the factorial design. Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 3; column 4 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 5; sample size is reported in column 6. All specifications control for the other orthogonal factors in the factorial design. Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 3 and number of observations in column 4; column 5 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 6; sample size is reported in column 7. Reported p-values are based on randomization inference (10,000 permutations). All specifications control for the other orthogonal factors in the factorial design. Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 3; column 4 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 5; sample size is reported in column 6. All specifications control for the other orthogonal factors in the factorial design. Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 3 and number of observations in column 4; column 5 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 6; sample size is reported in column 7. Reported p-values are based on randomization inference (10,000 permutations). All specifications control for the other orthogonal factors in the factorial design. Note: In the first column, means (and standard deviations) in the control group are presented for each variable. Column 2 reports differences between video only and video+ivr (and standard error) with its corresponding p-value in column 3; column 4 reports differences between video+ivr and video+ivr+SMS (and standard error) with its corresponding p-value in column 5; sample size is reported in column 6. All specifications control for the other orthogonal factors in the factorial design.