Do Offline and Online Go Hand in Hand? Cross-channel and Synergy Effects of Direct Mailing and Display Advertising

Despite the rise of digital, direct mailing as a marketing communication tool remains relevant and widely applied in practice. Nevertheless, research into the effectiveness of direct mailing in the online environment is scant. Key questions that remain entail how direct mailing affect different online and offline consumer activity metrics throughout the purchase funnel and how they interact with digital marketing communication tools. The current paper, therefore, investigates these two questions by conducting two studies. First, we focus on the effect of direct mailing on zip-code level upper, middle, and lower funnel performance metrics over time by analyzing quasi-experimental data from a large European insurance firm. The results reveal that direct mailing significantly influences consumer activity metrics in the online channel (i.e., online search and clicking behavior), in support of cross-channel effects of direct mailing. Moreover, direct mailing is shown to be effective throughout the purchase funnel, both directly and indirectly, with a positive net sales effect. Second, we study the joint effect of direct mailing and display advertising by analyzing field experiment data from the same insurance firm. The results show positive synergy between direct mailing and display advertising. Therefore, despite the rise of digital, direct mailing still serves as an effective marketing tool, both by itself and in combination with digital marketing.

In line with prior research, we acknowledge that direct mails can have direct effects on sales as well as indirect effects by inducing consumer activities which ultimately can lead to a purchase (see e.g., Naik & Peters, 2009). This notion of consumers moving through different preliminary stages before eventually conducting a purchase is also called the search-purchase funnel 1 (Verhoef et al., 2017). Hence, beyond insights into its lower-funnel sales effects, a better understanding of the effects of direct mailing alone and in the interplay with digital marketing communication on different upper-and middle-funnel (online) performance metrics on the aggregate level is required (Srinivasan, Rutz, & Pauwels, 2016).
Furthermore, we acknowledge that firms do not use a single medium to communicate their brand message (e.g., AdNews, 2017) and hence have to manage multiple different marketing media simultaneously (e.g., De Haan, Wiesel, & Pauwels, 2016). Conforming to prior research, these different marketing media interact with one another (see e.g., Danaher & Dagger, 2013) which can also be termed media synergy. Current literature largely neglects the synergy effects between different types of marketing communications, particularly between direct mailing and digital marketing communication. However, these synergy effects should be taken into account when determining the actual effectiveness of specific marketing media, in our case direct mailing. This is also suggested by cross media studies by Kantar Millward Brown which identify that globally 25% of media effectiveness can be assigned to media synergies (see e.g., AdNews, 2017). Therefore, insights into whether there is synergy between direct mailing and display advertising are needed.
To address our research questions, we conduct two studies. With the first study, we aim to investigate how direct mails affect consumer activity metrics in the different stages of the purchase funnel both online and offline over time by analyzing quasi-experimental data from a large European insurance firm. We find that direct mails affect all stages of the purchase funnel, 1 In the following, we will use the simpler term "purchase funnel".
in accordance with a cross-channel effect of direct mailing on online consumer activity metrics on the zip-code level. In particular, direct mailing yields a lift in the number of online searches for generic keywords as well as in the number of purchases. Direct mailing negatively impacts the number of online searches for branded keywords and the number of clicks on sponsored search ads. Overall, direct mailing seems to positively influence consumer activity in upper funnel stages by putting the general topic of the direct mail at the top of consumers' minds. We also find support for a positive indirect sales effect of direct mailing through consumers' search and subsequent clicking activity in the online environment. Taken together, the total effect (including the direct and indirect effects) of direct mailing on purchase behavior is positive. In study 2, we find evidence for a synergy effect between direct mailing and display advertising suggesting that these marketing communication tools complement one another and when used jointly, even exceed their individual effects. In sum, our findings support direct mailing as an effective tool to positively influence consumer activity metrics throughout the purchase funnel.
These effects also establish in the online environment and in combination with other digital marketing tools.
By addressing our research questions, we aim to contribute to both theory and practice. We build on research regarding direct mailing (e.g., Danaher & Dagger, 2013;Naik & Peters, 2009) and attribution modeling (e.g., De Haan, Wiesel, & Pauwels, 2016), and offer several contributions (see table 1).
First, we provide insights into the effectiveness of an offline marketing communication tool.
Digitalization trends have encouraged this focus on digital marketing channels, yet many massive advertisers (e.g., Procter & Gamble, Unilever) continue to reevaluate their marketing spending and have cut digital advertising spending, which even increased their media reach Marketing Science Institute Working Paper Series (AdWeek, 2018). Hence, firms need to manage and allocate their marketing budgets strategically across both online and offline media (De Haan, Wiesel, & Pauwels, 2016;Dekimpe & Hanssens, 2007), so insights into the effectiveness of offline marketing instruments are also of strong practical interest.
Second, we study the cross-channel, i.e., offline-to-online, effects of direct mailing and also take the indirect path into consideration. With the rise of digital media and the prevalence of studies on digital marketing channels, knowledge about the effectiveness of direct mailing (and in general offline marketing actions) on upper and middle funnel performance metrics is limited (cf. Dinner, Van Heerde, & Neslin, 2014). With a sole focus on purchase outcomes, one might miss the supporting effect of marketing activities which have led (up) to this purchase (see e.g., Srinivasan, Rutz, & Pauwels, 2016). Hence, to capture the complete effect of marketing activities, their indirect effects should also be considered.
Third, we are one of the first studies to show synergy between direct mailing and digital marketing communication (i.e., display advertising). Current literature generally neglects the interaction of multiple marketing actions, in particular online and offline marketing efforts, although research suggest that using both in combination is best due to possible synergy effects (e.g., Danaher & Dagger, 2013).
Lastly, we contribute by investigating how direct mailing affects the different stages of the purchase funnel over a considerably long period of time. In current direct mailing and attribution studies, dynamic time effects are largely neglected, preventing any sense of whether the effects might wear out over time or continue to have a long-run impact (Kannan, Reinartz, & Verhoef, 2016). Our extended timeline is also critical for direct mails, because customers respond through multiple steps, including opening the mail, keeping it, and responding to it (Feld et al., 2013).
----Insert Table 1 about here ----In the next section, we present our conceptual framework, review relevant studies pertaining to the purchase funnel and the (cross-channel and synergy) effects of direct mailing, and formulate our expectations. Then, we describe the unique data from both our studies and develop our models to answer our research questions. Thereafter, we present the empirical results of our analyses of both studies and conclude with implications for research and practice.

Conceptual framework
In line with Gopalakrishnan and Park (2019), we focus on the purchase funnel where the upper funnel stage refers to the share of consumers who become aware of their need and are induced to search for a product or service (i.e., awareness and search stage). This stage is followed by the middle funnel stage in which consumers interact with ads by clicking on them and eventually visit the advertising firm's website (i.e., consideration stage) (e.g., De Haan, Wiesel, & Pauwels, 2016). Lastly, in the lower funnel stage, we observe whether a certain group of consumers decides to conduct a purchase or not. For both studies, we focus on different consumer activity metrics on the aggregate, zip-code level throughout the purchase funnel in a highly similar manner (see figure 1).
In study 1, we analyze the potential effects of direct mailing on the different funnel stages: (1) the number of (organic) online searches (both branded and generic), which functions as a proxy for the awareness and search stage, because it is a channel to search for information (Li & Kannan, 2014); (2) the amount of clicks on sponsored search ads as a proxy for the consideration stage, because clicks lead consumers to visit the firm's website (Mulpuru et al., 2011); and (3) the number of purchases to represent the purchase stage. Beyond the effect of direct mailing on the different stages of the purchase funnel, we also investigate the relations among the different stages, i.e., (4) search → visit and (5) visit → purchase, allowing us to uncover the indirect effects of direct mailing on sales (Pauwels, Aksehirli, & Lackman, 2016).
The conceptual framework in Figure 1 details our study process.
In study 2, we aim to provide further evidence for the causal sales effect of direct mailing by analyzing its sales effect using field experiment data and diff-in-diff analyses to establish causality. Furthermore, we explore whether there is synergy between direct mailing and display advertising by investigating the change in purchase behavior when combining both types of marketing communication (6).

Research background
Prior literature establishes that the effectiveness of a firm's digital (e.g., email marketing, display advertising) and offline (e.g., TV and print advertising) marketing communication efforts differ across the different stages of the purchase funnel (Abhishek, Fader, & Hosanagar, 2012;Pauwels, Aksehirli, & Lackman, 2016). De Haan, Wiesel, and Pauwels (2016) suggest that firm-initiated communication (e.g., e-mail, TV advertising) can reach consumers unaware of their need for the product (or category). Abhishek, Fader, and Hosanagar (2012) concur and show that firm-initiated online communication is usually most effective in the upper part of the purchase funnel, moving consumers from a disengaged to an engaged state. In our conceptual framework, the stages preceding a potential purchase constitute the upper (i.e., awareness and search stage) and middle (i.e., consideration stage) part of the purchase funnel. Furthermore, prior research reports that firm-initiated communication in the upper and middle part of the purchase funnel positively contributes to an increase in purchase probability in later stages of the funnel (Li & Kannan, 2014;Shankar & Malthouse, 2007).
3.1. Effects of direct mailing in the upper and middle part of the funnel. We adopt the definition of a direct mail proposed by Jonker, Franses, and Piersma (2002, p. 6): "an addressed, written, commercial message." A limited number of studies point to the effectiveness of direct mailing in the upper and middle part of the purchase funnel without providing empirical evidence; they can trigger interest in a product/service and eventually lead to purchase (Roberts & Berger, 1999). According to Krafft et al. (2007), stimulating interest is another advantage of direct mails. Danaher and Dagger (2013) cite direct mailing as an effective tool to reach unaware consumers and make them aware, by exposing them to advertising. Naik and Peters (2009) provide empirical evidence for the effect of direct mailing in the middle funnel stage by showing that direct mails directly affect online car configuration visits, which is used as a proxy for the consideration stage. Therefore, we expect that direct mailing influences upper and middle funnel performance metrics, but also eventually help to move consumers along the funnel to the purchase stage, in line with an indirect effect of direct mailing.

Effects of direct mailing in the lower part of the funnel. Direct marketing
communications seek to influence buying behavior (Rust & Verhoef, 2005). Prior academic research mainly studies the direct effects of direct mailing on purchase behavior. Past studies find that direct mailing has a positive effect on purchase (e.g., Beirne, 2008;Bawa & Shoemaker, 1987;Verhoef, 2003;Gázquez-Abad, De Canniére, & Martínez-López, 2011) and adoption of a new (technological) product (e.g., Prins & Verhoef, 2007;Risselada, Verhoef, & Bijmolt, 2014). In their comparison of the relative effectiveness of multiple marketing tools, Danaher and Dagger (2013) determine that direct mailing is among the seven communication instruments that significantly influence purchase outcomes (i.e., dollar sales and profits).
Specifically, they identify direct marketing as the second most effective tool when considering dollar sales as the focal outcome and the most effective if profit is the focal outcome. Recently, Valenti et al. (2018) find positive effects of direct mails on purchase behavior in a retail context for prospective customers. Overall, direct mailing appears to have a strong, positive, direct effect on purchase behavior, and we include this expected effect in our framework.
Marketing Science Institute Working Paper Series 3.3. Cross-channel effects of direct mailing. Current literature largely neglects offline-toonline effects when investigating the effectiveness of marketing communication, focusing more on the online-to-offline effects. For example, Lewis and Reiley (2014) cite an increase in offline sales for a group of consumers exposed to banner ads, though Danaher and Dagger (2013) do not find any evidence for this cross-channel effect of display advertising. Lobschat, Osinga, and Reinartz (2017) extend this research by including the effects of online touchpoints (i.e., banner, sponsored search, and contextual advertising) on customers' online and offline (purchase) behavior. Their findings reveal an indirect effect of banner advertising on offline purchase likelihood, through website visits, for consumers who have not visited the advertiser's website recently. Srinivasan, Rutz, and Pauwels (2016) show effects of online customer activity in paid, owned, earned, and unearned media on (aggregate) sales and their interdependencies with traditional marketing mix elements. Despite the key insights these studies offer, they focus on the effect of digital marketing communication on offline consumer responses and neglect the effects of offline communication on online behavior.
There are a few notable exceptions which study offline-to-online effects with a distinct focus on TV advertising. Joo, Wilbur, and Zhu (2016) investigate the effect of TV advertising on online search behavior and find that TV ads for a financial services brand increase the total number of online searches as well as the number of online searches with a branded (vs. generic) keyword. In further support of an offline-to-online effect in the upper and middle part of the funnel, Fossen and Schweidel (2017) explore the impact of TV advertising on online word-ofmonth (WOM) and find a significant positive effect on WOM volume for the advertising brand. Liaukonyte, Teixeira, and Wilbur (2015) analyze the direct (and indirect) effects of TV advertising on online website transactions for five different product categories and find support for positive indirect effects of TV ads through consumers' direct visits to the advertising firm's website as well as referrals from search engines.
In sum, research suggests a positive cross-channel effect of TV advertising. However, research considering the specific cross-channel effects of direct marketing is scant. One notable exception is Naik and Peters (2009), who examine the effects of online display advertising, offline advertising, and direct mailing on online and offline consideration metrics for a car brand. They find significant cross-channel effects, such that online advertising affects the number of offline dealership visits, and direct mailing affects the number of online car configurator visits. They only consider the upper and middle funnel stages though. Mark et al. (2019) also find evidence for an offline-to-online effect by showing a positive influence of catalogues on purchase behavior in the digital channel.
Even given these prior research efforts, the effects of direct mailing throughout the full purchase funnel have not been taken into account. To address this gap, we study the crosschannel effects of direct mailing on upper and middle purchase funnel metrics on the zip-code level, with the prediction that these effects are notable, and also explore whether these earlier funnel outcomes also significantly impact the lower part of the funnel, i.e., help to increase sales.
3.4. Synergy effects of direct mailing. The interactions of multiple marketing actions, in particular online and offline marketing efforts, are generally neglected in current direct mailing as well as attribution modeling literature. Media synergy is "the added value of one medium as a result of the presence of another medium, causing the combined effect of media to exceed the sum of their individual effects" (Naik & Raman, 2003, p. 385). Jagpal (1981) was among the first to find empirical support for synergy in multimedia advertising by studying the synergy between print and radio advertising. Also, Naik and Raman (2003) find synergy between offline marketing actions, whereas other studies find synergy between offline marketing actions (e.g., TV or print advertising) and digital marketing actions (i.e., Internet advertising) (e.g., Chang & Thorson, 2004;Reimer, Rutz, & Pauwels, 2014). Stammerjohan et al. (2005) provide different theoretical explanations for the existence of synergy: Encoding variability theory states that if consumers are exposed to a (marketing) message in different media, encoding will result in a "stronger, clearer, more accessible information network in the brain" (p. 56). This, in turn, fosters the recall likelihood of the respective marketing message. Additionally, selective attention theory suggests that using multiple media increases familiarity with the marketing message, but also increases the complexity of the marketing campaign (Kahneman, 1973). This combination (i.e., a familiar but complex stimuli) is shown to increase consumer attention in line with a positive synergy effect (for an elaborate discussion on the theoretical explanations for media synergy, please see Stammerjohan et al., 2005). However, only a limited number of studies exist, which consider direct mailing when looking into the synergy effects of multimedia communication. Naik and Peters (2009) consider multiple offline (e.g., print, radio, television) and online media (e.g., banner and search ads) and find synergy effects among them. Also, they consider direct mailing, but do not find synergy effects among direct mailing and online or offline media. Similarly, Danaher and Dagger (2013) also examine direct mailing and do not find synergy effects. Lastly,  are the first to show synergy between online paid search and direct mailing.
Despite the efforts of current studies, research into the synergy effect of direct mailing with digital marketing communication is limited and yields mixed results. To address this gap, we study the synergy effects of direct mailing with display advertising.

Study 1: Cross-channel effects of direct mailing
In our first study, we aim to study how direct mailing affects consumers in the different stages of the purchase funnel. For this purpose, we analyze quasi-experimental data on the zip code level from a large insurance firm. In the following, we describe the data as well as our modeling approach and present our key findings on the effectiveness of direct mailing.

Data study 1
4.1.1 Quasi-experimental data. We have access to data from a large German insurance firm, which serves us adequately to answer our first main research question. The insurance firm is a well-known company that belongs to a worldwide insurance group with more than 50,000 employees in 200 countries. The firm's well-established, multichannel distribution system includes an online presence, owned agencies, and partners. For confidentiality, we cannot disclose its name. The data that this firm provided pertain to a campaign to promote car insurances, for which it sent out direct mails to potential new customers of the insurance firm.
The overall campaign ran from September 7 to October 24, 2015. For this campaign, the direct mails were sent out in week 43 (i.e., October 19-24) whereas all other campaign-related activities (i.e., TV advertising, online video advertising, social media marketing) were stopped 3 weeks before (i.e., September 7-28, 2015); these ended in week 40 2 . Hence, there is a time gap of 3 weeks between all non-direct mailing campaign activities and the direct mailing campaign.
The data cover 609 German zip codes (5-digit level) and are quasi-experimental (cf. Harmeling et al, 2017;Liaukontye, Teixeira, & Wilbur, 2015), such that they reflect a treated (n = 596) and a control group (n = 13), for which only the treated group received direct mails from the insurance company. For both groups, we have information over an 11-week period 2 Previous research suggests that these marketing activities should not influence our results given that their effects do not prolong for such a long period. Guitart and Hervet (2017) study the effects of TV advertising on online conversions and find that the effects of TV ads (including for a car insurance) level out after only 15 hours. Given this, we are confident that TV advertising did not bias our results. Same holds for online video ads and the firm's social media activities (see e.g., De Haan, Wiesel, & Pauwels, 2016).
The selection of the treatment (and control) group reflected the households' purchase potential in a specific zip code region, based mainly on age and income, though the firm's exact algorithm is unknown. The control group comes from similar zip code regions with the same household potential that ultimately did not receive any direct mails. The insurance company We also ran t-tests where we compare the treatment and the control group in the period (average mean across weeks) before the overall (direct mailing) campaign starts and find no evidence for significant differences between the two groups. In sum these results establish confidence in the composition of our experimental groups.

Variable operationalization.
The unit of analysis is customer behavior at the zip code level, measured on a weekly basis. The German zip codes are on the 5-digit level, which is the most granulated level of zip code level data for Germany. We aggregate daily data to a weekly level, because the variation per day is limited. Such a weekly aggregation is relatively common for research into direct response media, due to their low response rates (e.g., Danaher & Dagger, 2013;Naik & Peters, 2009;Srinivasan, Rutz, & Pauwels, 2016). Consumers often take some time to respond to (direct) mails, including the steps of opening, keeping, and responding to them, so analyzing daily data seems less useful (Feld et al., 2013). The data of interest are observed consumer activity metrics linked to the focal direct mailing campaign, so we only use data collected after October 24, 2015. As a cutoff date, we use January 3, 2016, or eleven weeks after the direct mails were sent out. This period should be sufficient, because direct mails have a peak effect one month after they have been sent out (Montgomery & Silk, 1972). Moreover, the data cover the start of a new calendar year, when consumers often decide whether to switch their insurance policies or not (Frankfurter Allgemeine Zeitung, 2015). In the following sections, we elaborate on the operationalization of our focal variables.

Direct mailing.
The direct mail we study is informational, mainly featuring information about car insurances and its relevance in general. The design was not personalized, so it was the same for all consumers, including images, a brief description of the insurance highlighted by the campaign, and the firm's logo (Web Appendix D; logo is hidden to maintain confidentiality). For the entire campaign, 450,000 direct mails were sent.

Organic search behavior.
We have information about the number of online search queries in response to the campaign on the search engine of Google at the zip code level and can distinguish two search query categories: branded and generic. Branded search queries contain our focal brand name in the list of keywords used to search (e.g., "State Farm insurance," "Allstate car insurance"); generic search queries do not (e.g., "car insurance," "how to insure my car") (cf. Ghose & Yang, 2009) and include product-category related search queries, but exclude competitor brand names. The data provider indexed the absolute query volume, so all absolute values are divided by a random base value. This indexing does not create any issues; it still allows us to see the movements and ratios between data points and thereby check the relative differences among data points. The data encompass the indexed number of generic (branded) online search queries for all zip codes over eleven weeks, which range from 0.25 to 278 (0.50 to 11), with average values of 3.07 (0.79) per zip code region. 4

Clicks on sponsored search ads.
With sponsored search advertising, the firm pays a fee to a search engine operator (e.g., Google) to display its ads, alongside the organic search results (Ghose & Yang, 2009). We have information on the number of clicks on our focal firms sponsored search ads that lead to website visits by consumers in each zip code region. The data comprise 3,217 clicks on sponsored search ads, ranging from 0 to 18 with an average of 0.48 clicks on sponsored search ads per zip code region per week over our 11-week period.

Purchase behavior.
The insurance firm records purchase behavior, including the number of purchases per zip code region, both online and offline. These data do not allow us to assign purchases of a zip code region to offline or online channels, but from additional data provided by the insurance firm, we determine that approximately 60% of total purchases take place online and 40% offline. The data cover purchase behavior from 609 zip codes, with a total number of 16,059 purchases, ranging from 0 to 107 with an average of 2.41 purchases per zip code region per week over our 11-week period. Of those purchases, 302 were conducted by existing customers of the focal insurance firm, i.e., 1.87%.

Missing data.
Our data come from a large quasi-experiment, including many zip code regions which we observe over a long observation period, which increases the chance for missing data. In our study, we have to deal with data, which is missing due to a technical matter on the side of the data provider. For search queries and clicks on sponsored search ads, the data contain entries if a search query or click on sponsored search ad is being conducted by a specific zip code in a certain week. However, the "no entries" can be due to (a) no search queries or clicks in a given week for a given zip code or (b) missing roll-up of non-attributed query or clicks data. This data is missing at random. Therefore, it is appropriate to treat the missing data points with multiple imputation 5 (Schafer & Graham, 2002). Analyses with incomplete data would likely result in inaccurate and/or biased predictions. Therefore, we rely on imputation of missing data to make use of all available information in the data (Schafer, 1997). Specifically, the multiple imputation method applies available data to predict missing data points. More details on the missing data and imputation process can be found Web Appendix C. Moreover, we tested the robustness of our results by analyzing our models with only non-missing data and find our results to be consistent. For more information, please refer to chapter 4.3.4. on robustness checks.

Model development
We propose a model for each funnel stage with a simultaneous system of equations for the upper funnel stage (i.e., number of generic and branded online searches) and separate equations for the middle and lower funnel stages (i.e., number of clicks on sponsored search ads and number of purchases) to analyze the data. We estimate our models to show the main effects of direct mailing (i.e., main effect models), but also aim to study how these effects develop over time. For this purpose, in addition to our main effects models, we also estimate our models including a time interaction (i.e., time interaction models). We elaborate on our model development and the specific models we use in the following sections.

Model development of the main effects models.
For each of the purchase funnel stage models, we are interested in showing the effects of direct mailing and the impact of the previous funnel stage while controlling for a set of additional variables. Therefore, in the following, we show how we include the treatment effect of direct mailing as well as the control variables before going into the model specifications.

Direct mailing.
Our main interest is studying the direct and indirect effects of direct mailing throughout the purchase funnel. However, we are aware that the effects of direct mailing might level off over time (East, 2003). Therefore, we include a decay effect, by changing the treatment effect to 0 for the treatment group six(seven) weeks after the direct mails have been sent out, in line with research showing that direct mailing has peak effects one month after they have been sent out (Montgomery & Silk, 1972) 6 . We also tested our models with two alternative specifications for the decay effect (i.e., quadratic and cubic). The models with our alternative specifications show highly similar results compared to our focal model. Moreover, a comparison of model fit of the different decay specifications reveals that our proposed specification with the treatment effect set to zero after 6(7) weeks has the best model fit 6 A comparison of the model fit of different specifications, in which we set the effect of direct mailing to zero after 4, 5, 6, and 7 weeks (including the model without decay), reveals that the model with the treatment effect set to zero after 6 weeks had the best fit for the upper and middle stage funnel models and 7 weeks for the lower funnel model (see Web Appendix A), so we adopt this specification.

Control variables.
In order to control for multiple aspects which might serve as confounds, we also include control variables in our models (see also table 2). First, we include lags of the dependent variables in the equations (e.g., including PurchasePrev it in the purchase equation), as a general way to control for unobserved variables (Wooldridge, 2012). Besides the lagged dependent variables, we also control for socio-demographics of the zip code regions (time-invariant) as well as own and competitive marketing activities in a given week 7 . In order to control for own and competitive marketing activities, we include the indexed weekly total industry advertising spend in our models as well as the average rank of the sponsored search ads of the focal firm. Lastly, we control for the number of direct mails send to each zip code region in the treatment group 8 . After adding the control variables in the models, we obtain our final main effect models for each purchase funnel stage. specify the relations and therefore, a simultaneous system of equations can be used (as indicated by Leeflang et al., 2015).

Upper
7 Given that some socio-demographics are highly correlated, we opted to only include those sociodemographics, which are not highly correlated, in order to circumvent potential multicollinearity issues. The results of the models with and without the highly correlated socio-demographics are robust, except for the correlation between ad spending and rank of the sponsored search ads. Therefore, we present the models with all covariates in the main text. Only for the middle funnel stage model, we exclude "ad spending" in favor of "rank" as covariate. 8 For some zip code regions in the treatment group, we were not provided with the number of direct mails send.
Therefore, we also analysed our models without these zip code regions and results are robust.
We simultaneously estimate the system of equations model for all zip code regions using three-stage least squares (3SLS) (Zellner & Theil, 1962) 9 . The model is specified as follows: In this system of equations, DMit represents whether the zip code region i is in the treatment group (1) or in the control group (0)

Middle funnel stage model.
In order to analyze the effects of direct mailing in the middle funnel stage, we model the number of clicks on sponsored search ads. We estimate the 9 Given the simultaneous estimation of our model and our initial analyses showing that the regressors of one or more equations are correlated with the disturbances, a 3SLS method leads to consistent and asymptotically more efficient estimates than a two-stage least squares (2SLS) method if the disturbances might be collectively correlated (Zellner & Theil, 1962). A Hausman (1978) test confirms that the 3SLS estimation is best for our models (null hypothesis accepted, p > .05). 10 We also ran both equations separately with all covariates included and results are similar, although the treatment effect on branded search turns insignificant. middle funnel stage model for all zip code regions using ordinary least squares (OLS). The model is specified as follows: In this model, we define the following additional variables: Clicksit is the average number of clicks on sponsored search ads for a zip code region i in week t, and Rankt is the average rank of the sponsored search ads for the focal firm in week t.

Lower funnel stage model:
In order to analyze the effects of direct mailing in the lower funnel stage, we propose a model for the number of purchases (i.e., Purchaseit -the number of purchases (both online and offline) for a zip code region i in week t). We estimate the lower funnel stage model using a zero-inflated Poisson model (Leeflang et al., 2015). Our dependent variable (i.e., number of purchases) is a count variable, which requires a Poisson distribution (Wooldridge, 2012;Leeflang et al., 2015). Moreover, we have to account for excess zeros in our data, as there is disproportionately high number of non-purchase incidences in our data.
The model is specified as follows: In this model, we define the following additional variables: Purchaseit is the number of purchases (both online and offline) for a zip code region i in week t, PurchasePrevit represents whether there was a purchase in the previous week(s) (1) or not (0) for a zip code region i in week t, and PPi represents the purchasing power per household index of Germany for a zip code region i. We also used the actual count of purchases per zip code in previous weeks in our model and find our results to be similar in size and sign.

Time interaction effect models.
In line with Kannan, Reinartz and Verhoef (2016) indicating the importance of understanding the dynamic time effects, we concur that the effects of direct mailing might change within the first 6 weeks after the direct mailing is send out.
Therefore, we explore how the effect of direct mailing develops over time by estimating an additional model for each purchase funnel stage that explores the effects of direct mailing on the different funnel stages in more depth. Here, the main effects models (equations 1a, 1b, 2 and 3) serve as a base, and we add an interaction term for the treatment variable and an elapsed time variable, which represents the weeks since the direct mails were send out (Konus, Neslin, & Verhoef, 2014) 11 . Given that the indexed total industry advertising spending variable causes potential multicollinearity issues when the time variable is included, this variable was excluded from the models. The detailed model specifications (equations C1a, C1b, C2, C3) can be found in Web Appendix B. We estimate the time interaction models for each of the purchase funnel stages in a similar way as the main effect models.

Results
Our main results are provided in Table 2  Beyond these direct effects, we also want to uncover potential indirect effects of direct mailing throughout the purchase funnel. Results reveal that organic online searches with a generic or branded keyword also influence each other. In the same week, online searches for a generic keyword yield a lift in the number of branded online searches (ζ2 = .28, p < .001) and branded online searches increase the number of online searches for a generic keyword (δ2 = 3.61, p < .001). Regarding our control variables, we find that weekly industry advertising spending and the share of male inhabitants of a zip code region have a significant influence.
Results indicate that both higher spending on advertising by the industry and a higher share of male inhabitants decrease the number of generic online searches, whereas they boost the number of branded online searches (δ7 = -1.66, p < .001; ζ 7 = .46, p < .01; δ9 = -34.48, p < .001; ζ 9 = 9.53, p < .001).

Middle funnel stage.
The results indicate that the number of clicks on sponsored search ads for our focal firm slightly diminishes after receiving (vs. not receiving) direct mails (η1 = -0.08, p < .01). The interaction effect model shows that the treatment and the interaction of the treatment and time variable for the middle funnel stage are not significant for clicks on sponsored search ads (ρ1 = 0.02, p > .10; ρ3 = -0.00, p > .10).

Robustness checks.
In addition to our focal analyses, we examine whether our results are robust to different model specifications. We test our models (1) without control variables, where we include the direct transition from the awareness & search stage to the purchase stage as we acknowledge not all consumers might follow the strict order of the funnel. We find that all effects remain the same (see Web Appendix G) as well as the net effect when taking into account this additional path.
equations model for upper and middle funnel stages, (6) where we only use non-missing data for the estimation, (7) after matching the groups using propensity score matching, and (8) with the direct path from the upper funnel level to the lower funnel level. All specifications provide directional support for our findings and most are similar in significance (see Web Appendix G).
Additionally, we ran multiple diff-in-diff analyses in which we estimate four separate diff-indiff analyses for our four focal DVs. We find our treatment effect across the different stages to be similar in size and sign, although the diff-in-diff estimators turns insignificant. We also ran our model with all stages estimated simultaneously and results are similar to a large extent.
Across all robustness checks, we find that our results are robust establishing confidence in our findings.

Conclusion study 1
In today's digital media age, questions remain about how direct mailing affects consumer online (and offline) activity metrics throughout the purchase funnel. Therefore, we assess the impact of a direct mailing campaign on different upper, middle, and lower performance metrics, using quasi-experimental data on the zip code level from an insurance firm. The results indicate that direct mailing exerts effects on all stages of the purchase funnel. We find significant crosschannel effects on the number of online searches and clicks on sponsored search ads.
Nevertheless, whereas the effect of direct mailing on the number of generic online searches is positive, it turns out to be negative for the number of branded online searches and clicks on sponsored search ads. Hence, it seems that the focal direct mailing campaign put the general topic of car insurances on top of consumers' minds which triggered further category-specific (rather than brand-specific) search behavior. On the other hand, the direct mail might have already fulfilled consumers' need for brand-specific information, which might explain the negative effects on consumers' branded search and clicking behavior. However, in combination with the positive impact on the number of purchases, the results (across the different purchase funnel stages) provide evidence for a positive indirect (sales) effect of direct mailing.
We acknowledge that there potentially might be debate about the causality of our claims.
Given the limitations of our data from study 1 (i.e., data sparsity issues), we are not able to properly conduct the required methodological approaches (i.e., diff-in-diff analysis) to establish causality. The field experiment in study 2 solves the data issues by having similar sized experimental groups and random assignment to the treatment and control group allowing us to execute diff-in-diff analyses. With this study we strive to confirm findings in the purchase phase of study 1 and to additionally study synergy effect.

Study 2: Synergy effects of direct mailing
With study 2, we thus aim to investigate whether there is synergy between direct mailing and display advertising and to establish test causality for the effect of direct mailing on consumers' purchase behavior. This section proceeds with a description of our experimental data and the regression models we use to analyze this data. Next, we present our key findings on the effectiveness and synergy of direct mailing.

Field experiment study
The field experiment, which we conducted with the same large German insurance firm, had a 2 (direct mailing: yes/no) by 2 (display advertising: yes/no) between-subjects design. Hence, the zip code regions in our experimental groups differ in receiving direct mails and/or a budget (vs. no budget) being allocated for display advertising (see figure 2). The assignment of zip code regions to the treatment and control groups was randomized. Both direct mailing and display advertising were part of a campaign by the insurance firm to promote liability insurances. For this campaign, the direct mails were sent out on September 19 (i.e., week 38) and the display advertising campaign also started on this day. In order to control for these differences, we include these controls in our models, which will be explained in the model development section. In the following sections, we provide detailed information on the variable operationalizations.

Direct mailing.
The direct mail in our field experiment is highly similar to the direct mail in study 1. It concerns an informational direct mail, mainly featuring information about the product category (i.e., liability insurances) and its relevance in general. Also, the design of the direct mail is the same for all consumers and includes images, a brief description of the insurance promoted by the campaign and the firm's logo (Web appendix D; logo is hidden to maintain confidentiality). For the campaign, 1,800,000 direct mails were sent.

Display advertising.
The display advertising treatment in our campaign concerns the allocation of marketing budget (i.e., 25,000€) for display advertising to our treated zip code regions. During our observation period, the focal firm did not run any other display ad campaigns neither in the treated nor in the non-treated (i.e., control group) zip code regions.
The display ads promote the campaign for liability insurances. The content is comparable to the direct mail featuring information about the product category and including the same images and the firm's logo.

Purchase behavior.
The insurance firm records purchase behaviors, including the number of purchases per zip code region, both online and offline. The data cover purchase behavior from 50 zip codes, with a total number of 173 purchases, ranging from 0 to 7 with an average of 0.12 purchases per zip code region per week over the thirty weeks.

Model development
We propose a set of difference-in-difference (DiD) models to compare our experimental groups on their purchase behavior allowing us to establish causal effects. In order to analyze the DiD models, we evaluated the assumptions of DiD, which indicated no issues. The main assumption of DiD indicates that the treatment variable should be uncorrelated with the error term and thereby, the treatment is not related to another factor that affects the dependent variable (e.g., Wooldridge, 2012). This assumption is termed the parallel trend assumption and can be checked by visual inspection. When plotting the purchase variable over time for our experimental groups, we believe we meet this parallel trend assumption. However, we do also control for potential aspects, which might cause the groups to differ in their behavior. Next, we explore potential synergy between direct mailing and display advertising by estimating a model including both treatments (and their interaction). Before elaborating on the model development of these models, we will discuss the inclusion of our control variables.

Control variables.
Our first analyses including all controls (i.e., socio-demographics on which the experimental groups differ) reveal that multicollinearity might be an issue given the high correlation between some of the controls and the conditional index (CI) indicating values > 30 (Hill & Adkins, 2003). Therefore, we excluded the socio-demographic variables which caused the CI to exceed the critical cut-off value, which left us with the models as explained in the following sections. For these models, multicollinearity does not seem to pose an issue with values for the conditional index between 1 and 30 (for more information, see the correlation tables in Web Appendix H).

Difference-in-difference models.
Our difference-in-difference models represent the different group comparisons and their before and after treatment differences to reveal the effects of direct mailing (i.e., by itself or in combination with display advertising) on consumers' purchase behavior. Moreover, we include control variables in each of the models (i.e., lags of the consumer activities, indexed total industry advertising spending as well as sociodemographics of the zip code regions). With our model set-up taking into account the differences between the before and after treatment period and the differences between our experimental groups, we are able to analyze the causal effects of direct mailing in our observation period for the purchase stage.
We estimate the difference-in-difference models for all zip code regions using a zero-inflated negative binomial model (Wooldridge, 2012;Leeflang et al., 2015). In addition, we also estimate these models with the interaction of treatment and the elapsed number of weeks after the mails have been sent out to test how the treatment effect develops over time. The models are specified as follows: In these models, we define the following additional variables: Direct Mailingi represents whether the zip code region i is in the treatment group (= 1; experimental group 2 which has received direct mails only) or in the control group (= 0; experimental group 4 which has received no direct mails). Media Combinationi represents whether the zip code region i received direct mails and display ads (= 1; experimental group 1) or only display ads (= 0; experimental group 3). Timet represents whether a week t is after (1) or before (0) the campaign period, and Number of householdsi represents the number of households in a zip code region i.

Synergy model.
In order to analyze the potential synergy effects between direct mailing and display advertising, we propose a model which includes an interaction term between direct mailing and display advertising on purchase behavior. We estimate the synergy model for all zip code regions using a zero-inflated negative binomial model. The model is specified as follows: In this model, DMi represents whether the zip code region i received direct mails (1) or not (0) and DAi represents whether the zip code region i received display ads (1) or not (0).

Results
An overview of our main results is provided in Table 4 and 5. According to the differencein-difference models, direct mailing significantly influences the number of purchases, providing causal support of our results of study 1. Furthermore, we find support for a synergy effect between direct mailing and display advertising. Before discussing the results of our models, we first present our initial analyses.

Model-free evidence.
First, we explore the purchase behavior in all our experimental groups (see figure 2) by conducting simple tests (e.g., one-way ANOVA and t-tests). In line with study 1, we find that a higher number of purchases occurs for the experimental groups receiving direct mailing, either by itself (group 2) or in combination with display advertising (group 1) (t = -2.81 and -3.78, p < 0.01 and 0.001, respectively). This is, however, not the case for display advertising. Initial results show that the number of purchases is not higher for the groups with display advertising (groups 1 and 3) compared to the groups with no display advertising (groups 2 and 4). This is true for the comparison for display advertising by itself (group 3) and in combination with direct mailing (group 1) (t = 0.92 and -0.15, p = 0.36 and 0.88, respectively).
Next, we examined whether there might be a first indication of a synergy effect by testing for an interaction between direct mailing and display advertising using a one-way ANOVA and a simple linear regression. The results of the ANOVA indicate that the group receiving direct mailing and display advertising (group 1) have a higher number of purchases compared to the groups, which receive only display advertising (group 3) or none of the treatments (group 4).
However, the number of purchases of the group with both direct mailing and display advertising (group 1) is not higher compared to the group with only direct mailing (group 2). This seems to indicate that there might not be a significant synergy effect. When testing this in a simple linear regression model, the results indicate a significant positive effect for direct mailing, B = 0.21, p < 0.01, but no significant effect for display advertising or the interaction between direct mailing and display advertising. We will discuss our model-based findings in the following 13 .

Difference-in-difference models.
The results on the diff-in-diff estimators indicate that direct mailing by itself 14 leads to an increase in the number of purchases (π3 = 2.77, p = .08).
Similarly, there is a positive effect of direct mailing on the number of purchases when having direct mailing in addition to a display advertising campaign 15 (" 3 = 1.65, p < .05). We provide additional insights by analyzing how these effects develop over time. For this purpose, we also include the elapsed number of weeks after the direct mails have been send out as well as an interaction term with the direct mailing treatment (Direct Mailingi and Media Combinationi, 13 We are also able to analyze the effects of our control variables for all our models. Results indicate weekly industry advertising spending has a significant negative effect in our DiD model for Media Combination. Furthermore, the lagged purchase behavior positively influences the purchase behavior in the week after. 14 For this model, we compare the following experimental groups: the experimental group receiving a direct mailing but no display advertising (2) vs. the experimental group receiving no direct mailing and no display advertising (4). 15 For this model, we compare the following experimental groups: the experimental group receiving a direct mailing and display advertising (1) vs. the experimental group receiving display advertising, but no direct mailing (3). (4a) and (4b), correspondingly (see e.g., Cong & Liu, 2020). To allow for potential non-linear effects, we also include the respective interaction terms with the squared term of this time trend specification. For direct mailing by itself, we find support for an inverted u-shape of our treatment effect over time. More specifically, the treatment effect of direct mailing increases until week 4 after which the effect starts declining, which is in line with previous research and our results in study 1 (Montgomery & Silk, 1972). For direct mailing in addition to a display advertising campaign, we do not find evidence for a time trend (both interaction terms are n.s.).

Synergy model.
The results on the synergy model (see equation (5)) indicate that the interaction between the direct mailing treatment and the display advertising treatment is significant and positive (τ3 = 1.39, p < .05). This result provides evidence for a synergy effect between direct mailing and display advertising. When calculating the actual effect sizes (given our zero-inflated negative binomial model, we have to take the exponential value of the estimates (e.g., Hilbe, 2011)), we find the following: The joint effect of direct mailing and display advertising on the number of purchases (joint effect size = 4.01) is significantly larger compared to the sum of the individual effects of direct mailing and display advertising (sum of individual effect sizes = 0.85). Hence, our results suggest that direct mailing and display advertising complement each other and can be used best together as their joint effect exceeds their individual effects. Generally, direct mails have shown to be relatively more engaging when compared to digital ads (Venkatraman, Dimoka, Pavlou, and Vo, 2016). On the other hand, prior research on display advertising indicates that especially display ads-which generally receive low levels of attention from consumers (only 12% of served display ads are looked at (Inskin, 2018))-serve as strong reminders, if consumers have recently been in touch with the advertising firm, e.g., through direct mails, which is also in line with encoding variability as well as selective attention theory. In this case, consumers are generally more responsive to display ads (Lobschat, Osinga, and Reinartz, 2017). When comparing zip code regions in our data which received direct mails and display advertising (group 1) with regions which only received display advertising (group 3), we find support for this. More specifically, for those zip code regions which received both types of media, the clickthrough rate is significantly higher (Mdisplay = 0.0038, Mdisplay+direct = 0.0043, p < 0.05). The same holds for the number of leads (Mdisplay = 1.17, Mdisplay+direct = 3.17, p < 0.001) as well as the number of online conversions triggered by display advertising (Mdisplay = 0.21, Mdisplay+direct = 0.61, p < 0.01). Combining these results, it seems reasonable to assume that consumers being confronted with a highly engaging direct mail will likely be more responsive to display advertising afterwards, in line with the synergetic effect we find.

Robustness check.
Apart from the estimation of our focal models, we analyze our models (1) without control variables and (2) with our purchase variable with a Poisson distribution. The results for both specifications are highly similar in direction and significance.
This suggests our results are robust and demonstrates we can have confidence in our findings.
In the following conclusion section, we will discuss the results of study 2 as well as the overall conclusions we draw from the two conducted studies.

General Discussion
In today's digital media age, questions remain how direct mailing affects consumers' online and offline activity metrics on the zip code level throughout the purchase funnel and how direct mailing interacts with digital marketing tools. To provide insights into these prevailing research questions, we conducted two studies. In the first study, we assess the impact of a direct mailing campaign on different upper, middle, and lower online and offline performance metrics, using data from an insurance firm. The results indicate that direct mailing exerts effects on all stages of the purchase funnel. We find significant cross-channel effects on consumers' online search and clicking behavior which -in combination with the positive impact on the number of purchases we find -provide evidence for an indirect (sales) effect of direct mailing.
The second study investigates the synergy effect of direct mailing and display advertising, using data from a follow-up field experiment with the same insurance firm. Furthermore, we are able to gain more confidence in the results of study 1 by using field experiment data, which allows us to tackle potential causal issues of study 1, to analyze the effect of direct mailing by itself and in combination with display advertising. The results indicate that direct mailing yields a lift in the number of purchases on its own and together with display advertising. Moreover, we find support for a synergy effect between direct mailing and display advertising. The results of both studies have implications for both research and managerial practice.

Research implications
The results validate prior research investigating the effects of direct mailing on purchase behavior (e.g., Danaher & Dagger, 2013;Verhoef, 2003). We extend these studies by investigating the cross-channel (offline → online) effects as well as the indirect effects of direct mailing on consumer activity metrics which can be used as indicators for the different stages of the purchase funnel. We show a positive impact of direct mailing on the number of generic online searches and purchases, though the impact on the number of branded online searches and clicks on sponsored search ads is negative.
The divergent effects on the number of online searches for generic versus branded keywords suggest that direct mailing mainly influences consumers in the upper funnel stages of the purchase funnel. As suggested by prior research (e.g., Andrews, Li, & Balocco, 2019;Agarwal, Hosanagar, & Smith, 2015), consumer search behavior becomes more focused when moving along the funnel, which implies that at the beginning of their funnel consumers search for more generic keywords whereas more specific keywords are used when further along the funnel.
Thereby, generic keywords imply consumers to be higher in the funnel (i.e., upper level) and specific (branded) keywords indicate that consumers are at a lower level in the funnel. This result also aligns with Valenti et al.'s (2018) finding that direct mailing is effective only for prospective customers, in the earliest stages of their information search.
The negative effect of direct mailing on the number of branded online searches next to the positive effect on the number of generic online searches might be explained by the design and content of the direct mail with a focus on the product category over the focal brand. This negative effect thus might reflect a change in consumers' search behavior. Consumers in the treatment group use more generic keywords for their online searches, at the expense of branded keywords, relative to the control group. Their focus, after receiving the direct mail, seems to turn away from the specific brand and toward the product category in general (Ghose & Todri, 2016). That is, direct mails might trigger consumers to reconsider their (prior) brand choices and potentially switch to another brand. On the other hand, the negative effects of direct mailing on consumers' branded search and clicking behavior might also be explained by the direct mail already fulfilling consumers' needs for brand-specific information.
In addition, by studying how the effects of direct mailing develop over time, we specify that their positive (negative) effects on the number of generic (branded) online searches increase over time, whereas the positive effect on the number of purchases only shows with some delay, i.e., 2 weeks after the direct mails have been send out. In line with previous research on offline marketing communications (i.e., catalogues; Mark et al., 2019), it appears that consumers first put aside the direct mailing, but after some time, they reconsider and start responding (Feld et al., 2013). These results may be partially explained by the end-of-the-year effect present in the German insurance market; insurance becomes increasing relevant for consumers, who must choose before the end of the calendar year whether they will switch or not (Frankfurter Allgemeine Zeitung, 2015). Thus, they face time pressures to start thinking about their car insurance so that they can make an informed decision before the new year starts, or else wait another entire year to be able to switch insurances.
We also explore the indirect sales effects of direct mailing by assessing the relationships between the different stages of the purchase funnel. Overall, we find a positive net effect of direct mailing on the purchase stage (i.e., overall effect, including direct and indirect sales effects). This is also confirmed by a positive ROI. In sum, our results indicate that direct mailing is not only an effective marketing tool to yield direct sales effects, but also helps to positively influence consumers' progression through the purchase funnel, in support of an indirect sales effect.
Furthermore, we extend current research by exploring the synergy effect between direct mailing and display advertising. In practice, digital and traditional (offline) marketing communication tools are utilized frequently (e.g., De Haan, Wiesel, & Pauwels, 2016;AdNews, 2017) and prior research suggests that using multiple marketing communication tools jointly is better than using a single marketing communication tool due to potential synergy effects (Danaher & Dagger, 2013). As one of the first studies, we find evidence for a significant positive synergy effect of direct mailing and display advertising, which implies the combined effect of direct mailing and display advertising is larger compared to the sum of the individual effects.

Managerial implications
Our results have important implications for firms that currently use or are planning to use direct mailing. In our empirical application, we find that direct mailing effectively generates online consumer responses throughout the purchase funnel, confirming that this traditional marketing tool remains effective even in the digital media environment; it is "far from dead" (Forbes, 2017). In the financial services industry, which represent the heaviest users of direct mailing (DMA, 2018), firms can benefit from this tactic. Even one of the largest online companies in the world, Google, uses direct mailing to contact its (potential) B-to-B customers, despite all its online opportunities to do so (CMO, 2017). Our results provide supportive evidence that direct mails are still a viable marketing tool, given their direct and indirect impact on consumers' purchase behavior and synergy with display advertising.
Attribution modeling. Firms also might use our research to assign value to direct mailing beyond just looking at the last action, i.e., conducting a purchase in our case. Our results reveal that direct mailing also impacts the upper and middle funnel stages of the purchase funnel.
Hence, by solely looking at the number of purchases as the only key performance indicator of a direct mailing campaign, marketing managers might underestimate its effectiveness. Also, as our results show, these upper and middle funnel effects eventually lead up to a positive, indirect sales effect of direct mailing. Given the relatively high investments associated with running a direct marketing campaign, marketing managers should consider also using upper and middle funnel (online) performance metrics (and the indirect effect associated with guiding consumers through the purchase funnel) to justify their direct mailing budgets to senior managers and the CEO. By using online metrics (like e.g., the number of clicks on sponsored search ads) also for their offline marketing activities, marketing managers could gain further insights into the relative effectiveness and interplay of their online and offline marketing activities. Overall, these insights can be used as information for a more efficient budget allocation across all marketing instruments.
Managing a direct mailing campaign over time. Our results show that the effects of direct mailing in the upper and lower funnel stages change considerably over time, i.e., in the 6 weeks after the direct mailing is send out. Most prevailing is that the positive direct sales effect of direct mailing only shows 2 weeks after the direct mailing has been send out. Given that our focal product is consultation-intensive, marketing managers should consider allocating sufficient support staff to these promising periods to the online channel, e.g., in the form of online chat personnel, and inform their agencies accordingly. Maybe marketing managers could also consider supporting consumers' decision for the focal brand by running e.g., TV ads (or the alike) during these time periods 16 .
Integration of other (digital) marketing activities. Our finding in study 1 of a negative effect of direct mailing on the number of branded online searches combined with the positive effect on the number of generic online searches suggests that the specific design of the direct mailing managed to put the product category top-of-mind of consumers, but not the focal brand. Hence, managers should reconsider the actual set-up of the mail and integrate the brand more prominently to avoid that consumers start to look for alternative offers. Moreover, they could also integrate a QR code (or the alike) to the focal firm's website to circumvent consumers to go to a search engine and provide easier access to the focal firm's online offering (visit stage).
One of our findings is a positive synergy between direct mailing and display advertising.
Hence, marketing managers could contemplate using direct mails in combination with digital marketing instruments. Notably, only using one of the marketing communication channels results in a smaller effect. When integrating marketing activities, the synergies between different marketing media should carefully be considered to gain the most. Overall, when planning a direct mailing campaign, marketing managers should consider how to effectively integrate other online marketing activities.

Limitations and further research
We acknowledge some limitations of our studies that also provide interesting research opportunities. In particular, though we find that direct mailing significantly affects important consumer activity metrics on the zip code level which are indicative of the different stages of the purchase funnel in our first study, our control group is relatively small. Hutchins et al. (2015) show that effect sizes are equivalent regardless the size of the control group, but our results still might have been influenced by the control group size; Coe (2002) suggests that the control group size can influence estimates of effect sizes. Further research should account for this potential influence by using a larger control group to establish more robust results.
Another avenue for research will be to study possible differences among various types of consumers, according to their demographic information and distinct consumer responses. In both studies, the data do not allow us to distinguish individual consumers by their demographic information. Furthermore, future research could look into long-term effects of direct mails, such as customer retention, cross-buying behavior or CLV (Kumar & Reinartz, 2016;Kannan, Reinartz, & Verhoef, 2016). Also, with access to individual-level data, future research could provide more profound information on the underlying process for the synergetic effect of direct mailing and display advertising and rule out potential alternative explanation for the observed lift in sales.
Additionally, unlike prior research which mainly focusses on factors (such as the rank of a sponsored search ad in Google) that influence the effectiveness of a sponsored search ad (e.g., Ghose & Yang, 2009), we focus on exploring how direct mails impact consumers in different stages of the purchase funnel. We do not have access to any sponsored search ad characteristics and our data does not allow to differentiate the total weekly number of clicks on sponsored search ads by keyword. Thereby, we are not able to explore the effectiveness of specific sponsored search ads. We encourage future research on this aspect.
Finally, both studies refer to a single, financial services firm. Though the financial service industry is studied frequently (e.g., Guitart and Hervet 2017;Verhoef 2003) and is important for Western economies. We acknowledge that other industries also use direct mails (e.g., retailing) and rely on their effects on purchase behaviors (Valenti et al., 2018). One main difference between the industry under study and other industries is the fact that our industry works with annual contracts that consumers potentially renew and change only once a year.
Insurance products are infrequently purchased by consumers and as a consequence changes in their purchase behavior are not observed as frequently (Verhoef and Donkers, 2001), while relative high switching costs make switching behavior also not very common Verhoef (2003).
Together, this can have limited the variance in the investigated consumer responses throughout the purchase funnel. Therefore, we expect that the effects of direct mails might be even stronger in other industries without these binding contracts and with more frequent purchases. Our study could be generalized in other industries.      Notes: AIC = Akaike information criterion; BIC = Bayesian information criterion.

Web Appendix C: Missing data and imputation method
The missing data points in our data set concern (1) the number of generic and branded searches as well as (2) the number of clicks on sponsored search ads for some weeks for some of the zip codes in our (quasi)experimental study. In total, 24,934 observations across all zip code areas, over the 11-week observation period, and for all variables are missing. The amount of missing data per variable also differs with branded search query use being the variable with most missing observations (5,848 missing observations). The reason why there is missing data is simply a technical matter on the side of the data provide. The information we were provided on the missing data issue is as follows: (1) For search queries, we only have entries in our data if a search query is being conducted by a specific zip code in a certain week. However, the "no entries" can be due to (a) no search queries in a given week for a given zip code or (b) missing roll-up of non-attributed query data. The latter also holds true for the number of clicks on sponsored search ads for a given zip code.
We carefully studied the missing data points in our data set and found that they were missing at random; the probability of missing values relates not to the values themselves but instead likely depends on variables available in the data. In our data, the missing values seem to depend on zip codes, which were not all available to link to the responses in some weeks.
Therefore, it is appropriate to treat these missing data points (Schafer & Graham 2002).
Analyses with incomplete data instead risks inaccurate or biased predictions. Moreover, it is better to use a dataset by imputing the missing values and treat these imputed values as real measurements than excluding subjects with incomplete data (e.g., Harrell 2015). Therefore, we imputed the missing values using a multiple imputation method. This method is also the preferred method for handling missing data (when not MCAR) (Jensen, Joy and Jensen 2008).
Multiple imputation has the goal of accounting for the relationship between the unobserved and observed variables, while also taking into account the uncertainty of the imputation (e.g.,