Predictable country-level bias in the reporting of COVID-19 deaths

We examine whether a country's management of the COVID-19 pandemic relate to the downward biasing of the number of reported deaths from COVID-19. Using deviations from historical averages of the total number of monthly deaths within a country, we find that the probability of underreporting of COVID-related deaths for countries with the most stringent policies was 58.6%, compared to a 28.2% for countries with the least stringent policies. Countries with the lowest ex ante healthcare capacity in terms of number of available beds underreport deaths by 52.5% on average, compared to 23.1% for countries with the greatest capacity.


Introduction
As severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) spread across the globe in 2020, researchers and governments devoted significant resources to estimating the fatality ratio and understanding the efficacy of various mitigation efforts. With healthcare resources in short supply in various countries, policy makers also attempted to model pressure on healthcare systems based on the spread of the virus and the ex ante healthcare capacity of the countries ( Verity et al., 2020 ;Verelst et al., 2020 ). These studies largely relied on governments' reporting of the number of deaths resulting from coronavirus disease 2019 , and assumed that the reported information was unbiased, meaning that any errors in reporting were exogenous to the policies being implemented or to the healthcare capacity of a country ( Flaxman et al., 2020 ).
This assumption of unbiased data is unlikely to be true, given that the institutional structures of countries have been shown to impact government reporting of measures of accountability. Prior research suggests that governments could manipulate public statistics for political gain. For example, Milesi-Ferretti (2003) suggests that fiscal rules can encourage "creative accounting " to portray budgets in ways that benefit politicians. Dafflon and Rossi (1999) studies how governments could alter their public accounting data to meet the requirements for European Union membership. In addition, several papers document the propensity of governments to manipulate financial data to disguise the magnitudes of deficits ( von Hagen and Wolff, 2006 ;Buti et al., 2007 ).
While academics and the popular press have reported significant variation in the number of deaths during the pandemic that are un- explained by both COVID-19 and historical trends (i.e., deaths that are likely related to COVID-19 but go unreported as such), we know of no attempt to formally study the role that political incentives or country capacity play in this propensity to underreport (e.g., Viglione, 2020 ;Economist, 2020a ). This lack of understanding is particularly concerning given the abundance of attention paid to the relation among government policies, healthcare capacity, and deaths related to COVID-19 (e.g., Fuller et al., 2020 ).
In this paper, we examine two factors that may encourage governments to bias downward the number of deaths related to COVID-19. First, we examine the relation between the stringency of the policies adopted to curtail the spread of the virus and the propensity to underreport. Policies such as lockdowns have caused protests against politicians, likely increasing pressure on those politicians to document that the policies were effective ( Carothers and Press, 2020 ). Second, we examine whether a country's ex ante healthcare capacity, measured as the number of hospital beds per 1000 residents, is associated with underreporting of COVID-19 deaths. Low healthcare capacity may represent poor planning or an inability of the government to care for its people, suggesting that governments unable to meet the demands placed on their healthcare system by the pandemic could be inclined to underreport COVID-19 deaths. Moreover, a negative relation between healthcare capacity and probability of underreporting could arise if a lack of hospital beds is correlated with a country's inability to keep track of and thereby classify a death as COVID-related. However, this resources explanation could generate a positive association with underreporting as well, given that in countries with low resources, deaths might be classified as COVIDrelated even when they are not. a proxy for those resources, measured as the number of physicians per 1000 people, we find no association between underreporting and this proxy, mitigating concerns that the number of hospital beds is solely a proxy for healthcare resources and is unlikely to be related to bias.
We find statistically and economically significant results consistent with our conjectures. Using deviations from historical averages of the total number of monthly deaths within a country, we find that the probability of underreporting COVID-related deaths in countries that took the most stringent steps to curtail the spread of the was 58.6% on average, compared to 28.2% for countries with the least stringent policies. Countries with the lowest ex ante healthcare capacity in terms of number of available beds underreport deaths by 52.5% on average, compared to 23.1% for countries with the greatest capacity. These results point to the need to account for the misreporting of outcomes when evaluating the efficiency of government policies and preparedness that were adopted across countries.
The remainder of this paper is structured as follows. Section 2 provides a description of the data, variable construction, and research design. Section 3 documents our empirical results. Section 4 concludes.

Measuring unexpected deaths unattributed to COVID-19
To calculate our main variables of interest, the underreporting of COVID-19 deaths at the country level, we begin by measuring the baseline expected monthly deaths for 51 countries where data are available. We first obtain data from the New York Times ( NYT ) that reports for 33 countries the number of deaths from all causes during the pandemic, as well as total deaths for prior years. We augment these data with similar data for 10 additional countries reported by Eurostat. We also hand collect from governmental and statistical agencies these data for an additional eight countries. Table 1 , panel A, reports the list of countries, the unit of measurement, and the source of the data. Data on the number of deaths attributed to COVID-19 is obtained from the Oxford COVID-19 Government Response Tracker (OxCGRT) ( Hale et al. 2020 ). 1 These data allow us to calculate Unexplained excess deaths as follows: where , is total number of people who died from all causes in country i in month t of 2020.
− , is the number of people who died from all causes in country i in month t in a year without a pandemic, calculated as the average total deaths in month i for t-y years. For example, for the month of May, the 2019−2015 , represents the average number of deaths in each May from 2015 to 2019. We calculate − , using the years 2015-2019 when the data are available, which is the case for most countries. For a small subsample of countries, data are only available beginning in 2017, so we use only three years of historical data (those countries are identified in Table 1 ).
19 , represents the number of reported deaths attributed to COVID-19 for country i in month t of 2020.
Unexpected excess deaths would take the value of zero if all of the intertemporal change in deaths can be attributed to COVID-19. Positive values likely reflect COVID-related deaths that are not classified as such, but they could also arise because of random variation across time within a country. We note that negative values could arise because other causes of deaths (e.g., accidents, flu-related deaths) might have decreased in 2020 or because of random variation across time. As we discuss later, we use several different measures of underreporting, including binary variables that require the unexpected excess death measure to be more 1 These data are available from https://covidtracker.bsg.ox.ac.uk . than 1, 2, or 5%, and a continuous measure to show that our results are robust to various specifications.

Measuring government response and healthcare capacity
OxCGRT collects information at the daily level on the actions taken by governments to curtail the spread of COVID-19. The goal of this data collection process is to develop a proxy for the intensity of a government's lockdown policies during the pandemic. To that end, OxCGRT identifies nine relevant containment and closure policies that governments took and creates variables at the daily level for each of these, scaling them from 0 to 100, where 0 means that the government has implemented no policy, and 100 means that the government implemented the most stringent policy possible (e.g., if all schools in a country are closed, then a country would have a school closure policy score of 100). The policies OxCGRT considers are school closures, workplace closures, cancelations of public events, limits on gatherings, public transportation closures, stay-home orders, restrictions in movement domestically and internationally, and presence of public health information campaigns (see Hale et al., 2020 for a detailed description of the data). These variables are then summed to create a daily index of the total stringency of current policies for every country. We then take the average index for each country on the 15 th day of the month to create country-month pairs, 2 calculated as: We refer to this index in our empirical results as the variable Stringency Index. 3 From the "Our World in Data " COVID-19 dataset, we collect our measure of healthcare capacity, Hospital beds , measured as the number of hospital beds per 1000 people in the country in 2018, the most recent year for which the data are available.

Control variables
To improve the predictive power of the model and the precision of our estimates, and to address omitted variable bias, we include several control variables in our regressions. Governments in countries with greater interconnection with the global economy and larger trade flows with partner countries may have different reporting incentives than less economically connected economies. In addition, countries with greater connectedness to the global economy might be at greater risk for exposure to COVID-19. To account for this variation, in our regressions we control for trade as a share of gross domestic product (GDP) for each country as measured in 2019, the latest year with available data ( Krugman, 2008 ;Khan et al., 2020 ).
The United Nations estimates that 90% of all reported COVID-19 cases come from urban areas ( United Nations, 2020 ). Since the virus spreads through person-to-person contact, we control for the percentage of a country's population living in urban areas in 2019 to ensure that our results are not driven by country-level variation in this population. We note that countries with more rural populations might be less able to account accurately for COVID-19 related deaths, leading to underreporting. This table reports information about the data used to create the variables in our main analysis. Panel A lists the countries used in the analysis, the years used to calculate the expected number of deaths based on the historical average, the geographic level at which the data were measured, and the source from which the data were collected. For the countries with asterisk, data was complemented with United Nations Demographics data and country-specific statistical agency. Panel B reports descriptive statistics for the variables used in the analysis. All variables are defined in Appendix A .

Table 2
The relation between excess deaths and the Stringency Index. This table reports the relation between governments' misreporting of deaths related to COVID-19 and the Stringency Index, a country-level measure of the intensity of restrictions imposed to curb the spread of COVID-19. Columns 1-2, 4, and 6-8 of Panel A report the results of logistic regressions, where the dependent variable is an indicator equal to one if underreporting is more than 1%, 2%, or 5%, as noted in the column heads. Columns 3, 5 and 9 report the results of OLS regressions, where the dependent variable is the continuous variable defined in Eq. 1 . Column 2 includes country-level fixed effects. Panel B repeats baseline regression with a continuous dependent variable and includes country fixed effects in column 2. In all specifications, standard errors are clustered at the country level and reported in parentheses. All variables are defined in Appendix A .
In addition, approximately 10.5% of patients with pre-existing cardiovascular disease (CVD) who contract COVID-19 die, compared with only 0.9% of patients who are otherwise healthy ( Ritchie et al., 2021 ). Therefore, we control for the percentage of the population with CVD in 2017, the most recent year for which the data are available, to ensure that our results are not driven by this related factor.
Moreover, we include a control variable measuring the number of days since the reporting of the first COVID-19 death in each country since the pandemic began to spread at different times across countries, and this might affect unexplained excess deaths. Including this variable in our models allows us to make inferences that control for the different stages of the pandemic in each country.
Importantly, the ability to diagnose and record deaths related to COVID-19 is directly related to a country's testing capacity, which varied significantly across time and across countries. 4 We include a country's monthly testing capacity, measured as the number of tests per 1000 residents, as a control to alleviate concerns that the results are driven by how many people are being tested. These data are downloaded from OurWorldinData, supplemented with hand collection of excluded countries. 5 Lastly, because the Stringency Index varies within a country over time, this allows us to also include country fixed effects to accounting for time-invariant country correlated omitted variables in some specifications. Given that the healthcare capacity variable is time invariant within the period of our study, we cannot include country fixed effects in the models that include healthcare capacity. All variables and their data sources are provided in Appendix A .

Research design
When predicting if a country is underreporting deaths and the country's management and exposure to the pandemic are given as features, the response variable has two values, underreporting or not. This type of response can be analyzed using a binomial logistic regression, where the response variable has two values 0 and 1, with 1 representing underreporting. We designate a country-month observation taking the value of 1 if unexplained excess deaths is more than 1%. We also test the robustness of our results using 2% and 5% thresholds to avoid the influence of observations that might have an increase in deaths but which might be small and therefore more likely due to luck or other factors.
In our analysis we are interested in finding whether a country has incentives to underreport fatalities given the explanatory variables, not the magnitude of that underreporting. Hence, we focus on logistic regressions. However, as a robustness test, we perform OLS regressions and report coefficients along with logistic regressions using a continuous measure of unexplained excess deaths. Since logistic regression coefficients are reported in log odds, and in order to have economically meaningful interpretation, we derive marginal effects at means for a select set of values of the Stringency Index and healthcare capacity to show  how the predicted probability of underreporting changes as the levels of the Stringency Index and healthcare capacity change. As described below, Figs. 2 through 4 plot those marginal effects (regression results are reported in Tables 2 through 4 and 6 ). We cluster standard errors at the country level to mitigate serial correlation in the error term within countries.

Unexpected deaths during the COVID-19 pandemic
During the pandemic, which first appeared in China in December 2019 and has since spread to 219 countries, there has been significant country-level variation in the deviation from the historical aver-age in the number of monthly deaths unattributed to the pandemic Economist, 2020a ;Viglione, 2020 ;World Health Organization, 2020 ). Our data sources, at the country level, are reported in Table 1 , Panel A.
For the 51 countries for which monthly historical death rates are available, we find, on average, 3140 unexpected deaths from January through December of 2020. On the extreme end, the United States had 50,876 unexpected deaths during this period (see Table 1 , Panel B, for descriptive statistics). Fig. 1 shows for each country in our sample its highest monthly ratio of total reported deaths to historical expected deaths. For example, Bolivia and Ecuador had 2.5 times as many deaths (unattributed to COVID) than what historical averages would predict for the same months, while Japan and Bulgaria had almost no unexpected deaths. Some behaviors during the pandemic, such as quarantining at home, may have reduced deaths due to factors such as the spread of other illnesses like seasonal flu ( Economist, 2020b ;Oguzuglu, 2020 ). Therefore, this increase in unexpected deaths is likely attributable, at least in part, to COVID-19 but were not reported as such.
This misreporting could be due to random error, as countries unprepared for the outbreak also may have been unable to devote resources for sufficient analysis of causes of death. Alternatively, it could be due to systematic biases related to the management of the outbreak or a country's capacity to handle the outbreak, and thereby its exposure to the outbreak. We further examine the potential for systematic and predictable bias in the sections that follow.

Unexpected non-COVID-19 deaths and country management of the outbreak
Epidemiological responses to contain the outbreak of SARS-CoV-2, such as travel restrictions and geographical lockdowns, have had large negative impacts on countries' economies. As a result, 400 million jobs have been lost worldwide, and global gross domestic product in 2020 was expected to decline by 4.9% ( International Labour Organization, 2020 ; International Monetary Fund, 2020 ) . Still, there is growing Table 3 The relation between excess deaths and healthcare exposure. (1) (2) evidence that the economic sacrifice of lockdowns has played a vital role in curtailing the spread of the virus ( Hale et al., 2020 ). Given the economic consequences of the actions taken to curtail the spread, political leaders may have been incentivized to underreport deaths from COVID-19 to provide evidence that they handled the response to the pandemic properly. We find support for this conjecture. Fig. 2 documents that, for a country that has taken few actions to curtail the spread of SARS-CoV-2 (i.e., a Stringency Index of 10 out of 100), the probability of under-reporting the number of deaths related to COVID-19 by 1% or more is 28.2%.
Alternatively, countries implementing the most stringent policies to curtail the virus's spread are two times more likely to underreport deaths related to COVID-19 (i.e., the likelihood of underreporting is 58.6%). As reported in Table 2 , these findings are robust to various specifications, including using a continuous measure of underreporting, adding a host of country-level control variables, changing the dependent variable from being equal to 1 when the total number of unexpected deaths greater Table 4 The relation among excess deaths, the Stringency Index, and healthcare exposure. (1) (2)  than 1% of total deaths to being equal to 1 when the total number of unexpected deaths is greater than 5% of total deaths, and using a continuous dependent variable of total unexpected deaths.
In Columns 1 through 7 of Table 2 , we also include a calendar control, measured as the number of days since the first reported death related to COVID-19. We include our control for testing capacity in columns 8 and 9. Given that testing increased over time, these two variables have a correlation of 0.53, so in this table, as well as Tables 3 and  4 , we do not include both in the same specification to mitigate concerns about multicollinearity. 6 Importantly, the marginal probability estimates for underreporting, reported both in Fig. 2 and in Table 6 , increase monotonically across deciles as the Stringency Index increases.
This finding is surprising given that one component of the effort to reduce the fatality ratio of the virus is implementing policies to curtail its spread. Still, if the actions taken to curtail the SARS-CoV-2 outbreak did not achieve their intended effect of reducing the spread, or if a government took precautionary stringent steps because it knew it would be unable to effectively identify all cases of COVID-19, then, as we find, there should be systematic bias in the reporting of deaths attributable to COVID-19 for countries that took the most stringent steps to curtail the virus ( von Hagen and Wolff, 2006 ).

Unexpected non-COVID-19 deaths and country exposure based on healthcare capacity
We next examine whether the exposure risk of a country's population to SARS-CoV-2, based on a country's healthcare capacity, is related to underreporting of total deaths from COVID-19. The number of hospital beds is frequently used as an indicator of a country's ability to cope with COVID-19 and provide care for its citizens ( Cavallo et al., 2020 ). Therefore, governments in countries with fewer hospital beds per capita are incentivized to underreport the number of deaths related to COVID-19 since political leaders risk being blamed for deaths that could be perceived as avoidable were it not for an inadequate healthcare system. In addition, countries with the lowest healthcare capacity are also less likely to be able to appropriately diagnose all causes of death when the healthcare system is taxed, thereby giving rise to a predictable negative relation between number of beds and probability of underreporting. This table reports the relation between lockdown stringency and underreporting, in a quasi-event study. We estimate following event-study with standard twoway fixed effects: = + + + Γ + , where D it is an indicator equal to 1 if a country's Stringency Index is above 20 (the level at which policies start going into effect) in a given month and 0 otherwise, are country fixed effects, are time fixed effects (month), is indicator variable as mentioned above (Treat t * Post t ), is main coefficient of interest identifying the average treatment effect on the treated countries and Γ is a time-varying control (monthly deaths). In column 1, remains equal to one for the entire duration after the Stringency Index takes a value of 20 or greater. In the second column, we limit the sample to four months for each country, from two months before to two months after the Stringency Index first becomes 20 or greater. In both specifications, standard errors are clustered at the country level and reported in parentheses. All variables are defined in Appendix A .

Table 6
Marginal probabilities derived from logistic regression. (1) (2) This table reports marginal probabilities derived from logistic regressions and shows the predicted probabilities of underreporting COVID-19-related deaths by at least 5% for each decile of the Stringency Index and healthcare capacity (Hospital beds). Ranks in Columns 1 and 3 are in ascending order, with 10 representing the most stringent policies to curb the spread of the pandemic. Ranks in columns in 2 and 4 are in descending order, with 10 representing the lower number of hospital beds per 1000 residents. Column 1 reports coefficients derived from running the logistic regression as in Column 4 of Table 2 . Column 2 reports coefficients derived from running logistics regression in Column 3 of Table 3 . Columns 3 repeats the analysis as in Column 1, but controls for hospital beds, and column 4 repeats the analysis in Column 2 with the Stringency Index as an additional control. In all specifications, standard errors are clustered at the country level and reported in parentheses. All variables are defined in Appendix A . As reported in Fig. 3 and Tables 3 and 6 , we find a negative relation between the likelihood of underreporting deaths related to COVID-19 and the number of hospital beds. The probability of underreporting COVID-related deaths in countries with fewer than 100 beds per 100,000 people is 52.5%, compared to 23.1% in countries with the highest ex ante healthcare capacity. As in our prior analysis, these results remain unchanged when changing the specification or adding country-level controls. These additional results are reported in Table 6 . Similar to the results for the Stringency Index, the marginal probability estimates for underreporting, reported in Fig. 3 and in Table 6 , decrease monotonically as healthcare capacity increases.

Incremental explanations for misreporting
It is plausible that there exists an endogenous relation between a country's management of the virus outbreak and its exposure to the virus given that ex ante healthcare capacity is likely to influence the actions a country takes to address the spread of the virus. For example, it might be the case that countries with more hospital beds would take less stringent measures as they can better cope with infected people. To mitigate this concern, Fig. 4 , panel A reports the relation between unexpected deaths, country management of the virus, and country exposure to the virus. The Stringency Index has a mediating effect on the relation between exposure and underreporting, but the intuition remains the same. Specifically, as reported in Fig. 4 and Table 4 , healthcare capacity remains directionally consistent and predictive of underreporting in most specification, but the statistical significance decreases (in column 2 of Table 4 , the coefficient on Hospital beds is significant at the 5% level, while it is significant at the 10% level in columns 1 and 3). Importantly, the Stringency Index remains statistically significant across all specifications. As shown in Fig. 4 , panel B, for countries with the lowest healthcare capacity, the probability of underreporting remains above 40%. Table 4 provides evidence that these results are mostly insensitive to research design choices and adding the following control variables: Population density, as countries with more dense populations might be less likely to have underreporting due to geographical dispersion; the population Age 70 and older as the attribution of cause of death might be less costly in countries with more aged populations; the number of Physicians per 1,000 people, as in countries with more physicians, COVID-related classification of cause of death might be more likely due to lower resource constraints; and testing capacity, which is directly related to a country's ability to diagnose residents. The inferences from the prior analysis remain unchanged in this test, suggesting that management and exposure are both predictors of government underreporting of deaths related to COVID-19. In addition, the coefficient on Physicians is statistically insignificant in all specifications, mitigating concerns that Hospital beds is solely a proxy for healthcare resources and unlikely to be related to bias.

Cross-sectional versus within-country variation
In our main analysis, we use a panel of country-months to examine the relation between government policies and the underreporting of COVID-19-related deaths. To better show how these policies relate to underreporting over time we conducted detailed within-country tests. Panel A of Table 2 reports logistic regression with time fixed effects (and country fixed effects in column 2) and suggests that, within country, increasingly stringent policies are associated with underreporting. 7 In Panel B of Table 2 , we also report this fixed effects specification for the continuous measure of misreporting and again find that our inferences do not change.

Quasi-event study examining misreporting before and after lockdowns
To examine the relation between lockdown stringency and underreporting, we conduct a quasi-event study. We create an indicator equal to 1 if a country's Stringency Index is above 20 (the level at which policies start going into effect) in a given month and 0 otherwise. We estimate the following event-study with standard two-way fixed effects: where is a continuous measure of misreporting, are country fixed effects, are time fixed effects (month), is indicator variable as mentioned above (Treat t * Post t ), is main coefficient of interest identifying the average treatment effect on the treated, and Γ is time varying control (i.e., total monthly deaths, including those that are COVIDrelated and those that are unattributed to COVID-19). Table 5 reports the results. In the column 1, remains equal to one for the entire duration after the Stringency Index takes a value of 20 or greater. In the second column, we limit the sample to four months for each country, from two months before and to two months after the Stringency Index first becomes 20 or greater. In both columns, the coefficient on is positive and significant, providing evidence that within countries, as COVID-related lockdown policies become more stringent, government underreporting of COVID-related deaths increases.

Conclusion
Our results are subject to two important caveats. First, our measure of excess deaths assumes that total deaths are reported accurately throughout time. Second, our measure of excess deaths assumes that changes in behavior during the pandemic do not influence the frequency with which other types of death are expected to occur. While the first caveat is less of a concern, given that it is easier to misclassify the cause of death than it is to not report that death at all, the second is unlikely to hold as deaths from behaviors like driving have been influenced by the pandemic and other infectious diseases have spread less rapidly as more people have stayed home due to lockdowns ( Economist, 2020b ;Oguzuglu, 2020 ). Still, these changes in behavior are likely to bias against our results, given that total deaths unrelated to COVID-19 are likely to decrease during this period.
Overall, our results suggest that incentives and response capacity to the pandemic shape reported outcomes. This finding has implications for researchers who seek to understand the efficacy of actions taken by or characteristics of countries that might mitigate or exacerbate deaths related to COVID-19. Our results point to the need to account for misreporting of reported outcomes to assess the efficacy of these actions and characteristics more accurately.

Declaration of Competing Interest
Not applicable.