Testing the Perturbation Sensitivity of Abortion-Crime Regressions

The hypothesis that the legalisation of abortion contributed significantly to the reduction of crime in the United States in 1990s is one of the most prominent ideas from the recent “economics-made-fun�? movement sparked by the book Freakonomics. This paper expands on the existing literature about the computational stability of abortion-crime regressions by testing the sensitivity of coefficients’ estimates to small amounts of data perturbation. In contrast to previous studies, we use a new data set on crime correlates for each of the US states, the original model specifica-tion and estimation methodology, and an improved data perturbation algorithm. We find that the coefficients’ estimates in abortion-crime regressions are not computationally stable and, therefore, are unreliable.


Introduction
In a famous and controversial paper, Donohue and Levitt (2001), hereafter DL, argued that the legalisation of abortion in the United States (US) in the 1970s may account for as much as one-half of the overall crime reduction in the US in the 1990s. According to the theory behind this result, increased availability of abortion led to fewer unwanted children, who are more likely to become criminals when they reach adulthood. This hypothesis has become one of the most widely discussed ideas from Levitt and Dubner's (2005) Freakonomics, which was enormously popular among the general public.
DL's empirical analysis was criticised for various reasons by Joyce (2004;2009), Lott andWhitley (2007), Foote and Goetz (2008), Moody and Marvell (2010) and others. Donohue and Levitt (2004; responded to some of these critiques; see also Joyce (2010) for a general overview of the debate about the impact of abortion on crime.
One recent criticism of DL's abortion-crime regressions involves testing the computational stability of their results using numerical analysis and computational economics tools. In particular, Anderson and Wells (2008) have argued that the computational problem posed in DL is ill-conditioned because it is very sensitive to small amounts of perturbation in the data, and therefore, their regression results are not computationally stable. Anderson and Wells (2008) showed that the condition number, κ, which is an upper bound for the sensitivity of the least squares solution to data perturbations, takes a very large value (κ = 1,329,930) for the basic regressions calculated by Donohue and Levitt (2001). Moreover, they calculated the bound on the relative error of the coefficients estimated by DL and found that it is too high to have any confidence in the estimated results. They concluded that there is not enough information in the data used by DL to mean-

Testing the Perturbation Sensitivity of Abortion-Crime Regressions
Testing the Perturbation Sensitivity of Abortion-Crime Regressions ingfully estimate regression coefficients. Anderson and Wells (2008) also showed that DL's models suffer from collinearity and that the linear specification used in these models is problematic. Finally, they show that similar problems also affect the results in Foote and Goetz (2008).
McCullough (2010) complements the theoretical insights in Anderson and Wells (2008) using a visual diagnostic tool for computational stability Beaton et al. (1976). His results, obtained using simplified versions of DL's models, suggest that DL's regressions were too demanding for the data, and therefore, the estimated results are not numerically stable.
In this paper, we provide another test of perturbation sensitivity for DL's original abortion-crime mod- Finally, we use a formal algorithm, proposed by Vinod (2009), for producing perturbed data sets.
The remainder of the paper is organised as follows.
Section II presents an introduction to the methods of testing for computational stability using data perturbation, Section III introduces the data, Section IV offers empirical results, and Section V concludes.

Testing computational stability using data perturbation methods
Testing the computational stability of regression coefficients using data perturbation was first proposed by Beaton et al. (1976). Their procedure consists of simulating a large number of perturbed data sets by adding uniformly distributed numbers from the range [-0.5, A recent study by Vinod (2009)  To produce perturbed data sets, Vinod (2009) proposed a simple algorithm to retain only the reliable digits of every perturbed variable and replace the trailing digits with suitably chosen random numbers; see Vinod (2009, pp. 207-208) for details. We follow his algorithm to produce perturbed data sets for our analysis.

Data
We use data from a new comprehensive panel data set with crime statistics from each US state with several  Table 1. In all of our calculations, we also include the state and year indicator variables, which control for state-year effects. Foote and Goetz (2008)

Results
Similar to DL's original study, we use fixed-effects mod- Another simple test of computational stability using perturbed coefficients is to determine whether nearly all of the perturbed solutions agree with the unperturbed solution to at least one significant digit (Beaton et al., 1976). The definition of agreement to one significant digit is as follows. If U is the unperturbed solution and P is a perturbed solution, then P is said to agree with U if P falls within the interval U ± five units in the second significant digit of U.
In cases where regression results are computationally stable, all or almost all the perturbed coefficients should agree with their unperturbed counterparts to at least the first significant digit. Table 2 shows results of the test based on this idea. The perturbed coefficients for the abortion and prison variables always agree to a single significant digit with the unperturbed solution. However, this is not the case for other variables. In particular, for the beerpc, prate and rincpc variables, on average, only 15%, 21% and 69%, respectively, of perturbed coefficients agree with the unperturbed coefficients. In the case of the property crime equation, less than 1.5% of the simulated coefficients agree with the original coefficients for the policepc and rwelpc15 variables, which is clearly a sign that, for our data set, the original (unperturbed) solution for the DL's abortion-crime regressions is not computationally stable. Testing the Perturbation Sensitivity of Abortion-Crime Regressions