Abstract
This paper concerns recent attempts to use the epistemological notions of sensitivity and safety to shed light on legal debates about so-called “bare” statistical evidence. These notions might be thought to explain either the outright inadmissibility of such evidence or its inadequacy for a finding of fact—two different phenomena that are often discussed in tandem, but that, I insist, we do better to keep separate. I argue that neither sensitivity nor safety can hope to explain statistical evidence’s inadmissibility, since neither offers a plausible criterion of admissibility that would exclude such evidence; both are subject to copious counterexamples, especially given their factivity, and it is difficult even to state a coherent criterion of admissibility in terms of either sensitivity or safety. The possibility remains, though, that either notion might explain statistical evidence’s inadequacy for a finding of fact; I express some doubts about this possibility but do not rule it out.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Not everyone accepts this starting assumption: Ross (2021) and Allen (2021), for instance, reject it as misguided. And there is scope for denying that the case history widely taken to support the assumption genuinely does support it. Ross and Allen both argue that, in the most famous cases in the literature, careful scrutiny suggests that the problem with the evidence was never supposed to be that it was statistical in nature but merely that it was insufficiently strong.
But it is this assumption on which the literature on statistical evidence is founded.
The practice of using the terms “bare” or “naked” to refer to cases involving statistical evidence but no other evidence originates with Kaye (1980).
Nesson’s original case involves a far-away eyewitness rather than grainy CCTV footage. But this creates some noise, given that we might wonder whether the eyewitness is reliable as to the number of prisoners in the yard and what each of them did. Modifying the example as in the main text eliminates this noise.
I do not actually accept this, for reasons given in Johnson King and Babic (2020). But it is another starting assumption of the literature, and one that I leave unquestioned for the purposes of this paper.
For instance, Allen (1990) argues that the prison yard case shows that the criminal standard of proof requires a state of “subjective certainty” that cannot be expressed probabilistically.
Here, too, some think that our intuitions about statistical evidence tell us about the nature of legal proof. For example, Cohen (1977) takes a case that is structurally similar to the prison yard case (but involves a civil rather than a criminal violation) to show that legal verdicts cannot be calculated using standard probability theory, offering an alternative that he calls “Baconian” probability. And Nesson (1985, p. 22) suggests that a case like Smith shows that the acceptability of a verdict “is not a simple function of mathematical probability, but rather is a complex matter of communication that depends on the nature of the issue, the process of decision, and the purposes and audiences the conclusion serves”.
One of Koehler’s observations is that even courts’ rulings about the admissibility of base rates intended to establish presumptive ownership have been mixed. There is Smith, of course, as we all know. And, in Guenther v. Armstrong Rubber Co., 406 F.2d 1315, 1318 (3rd Cir. 1969), the court ruled against a plaintiff who was injured at work by a faulty tire and sued the manufacturer that makes 75% of the tires used by his company, on the grounds that, based on this evidence alone, a verdict for the plaintiff “would at best be a guess” and “could not be reasonably supported”. This is very similar to the ruling in Smith. But what appears to be the exact opposite opinion was returned in Kaminsky v. Hertz Corp, 288 N.W.2d 426 (Mich. Ct. App. 1979), in which the plaintiffs were taken to have established presumptive ownership of a truck (from which a chunk of ice had fallen and injured them) on the grounds that the truck bore a Hertz logo and Hertz owns 90% of the trucks that bear its logo. And in Kramer v. Weedhopper of Utah, Inc., 490 N.E.2d 104 (Ill. App. Ct. 1986), the court ruled that the fact that a certain company supplied 90% of the bolts used by Weedhopper’s aircraft kits was sufficient to permit the inference that they had supplied the bolt that sheared off the defendant’s aircraft and injured him. It is very difficult to square Smith with Kaminsky and Guenther with Kramer.
Safety is sometimes introduced as the contrapositive of sensitivity: it is said that while sensitivity requires that the agent does not hold the belief if it is false, safety requires that it is true if she does hold it (Sosa 1999, p. 142). (Note: Sosa offers multiple formulations of safety in this initial article, including one introducing the notion of “easiness”.) Testing the truth of the conditional “If the agent believes the proposition, then it is true” requires us to look at the closest worlds in which the agent still believes the proposition and make sure that none of them are worlds in which it is false.
N.B. Enoch, Fisher and Spectre think that this epistemological story explains our aversion to statistical evidence, but not what is wrong with statistical evidence. They do not think that their epistemological story is vindicatory, as they think the law should not care about knowledge, and they offer an alternative vindicatory story (op. cit. pp. 210–223).
Spelling this out in further detail in another footnote, Pardo says that “the key point is that neither the statistical-individual distinction nor the admissibility-inadmissibility distinction can be explained in terms of the sensitivity-insensitivity distinction. None of these three distinctions (sensitivity-insensitivity, statistical-individual, admissible-inadmissible) can be explained in terms of another” (p. 16, fn. 77).
They also leave open that statistical evidence is indeed sometimes—or even often—inadmissible, there being some other explanation of this fact. What I argue is that sensitivity and safety cannot explain inadmissibility, not that nothing can explain inadmissibility or that sensitivity and safety cannot explain anything.
Externalists have standard responses to the charge that their criteria are not action-guiding: sometimes they point out that the idea of an epistemic “action” seems out of place since we lack direct control over our beliefs (the classic statement of this view is in Alston 1988), and sometimes they point out that no criterion is perfectly action-guiding, since there is no condition such that we are always able to tell whether we are in it (for discussion see especially Srinivasan 2015, Hughes forthcoming). But these responses are not much help in the legal context. For we are not discussing beliefs but rulings of admissibility, which are actions subject to direct control. And, even if no criterion of admissibility can ever be perfectly action-guiding, it still seems a bad idea to propose criteria that it is in principle impossible for judges to apply at the time when they must issue their rulings.
We do not have infallible epistemic access to our own beliefs and credences. But we do at least have better access to them than judges have to the disputed facts of a case at the time when they must issue rulings of admissibility.
I am grateful to an anonymous referee for suggesting that I consider Günther’s position here and for helping me to understand the implications of his model. It should be borne in mind that Günther does not himself propose epistemic sensitivity as a criterion of admissibility; Günther is interested in sufficiency rather than admissibility. Still, someone could propose that we use Günther’s notion as a criterion of admissibility in order to avoid problems with factivity.
Here is some more detail. Günther models belief states using nested sets of possible worlds and probability functions: there is a set S of worlds that the agent “cannot exclude to be actual”, to which her probability function assigns some non-zero probability, and there is a larger set S’ that includes all worlds in S plus some other worlds to which the agent assigns zero probability but that “may become relevant under a supposition”. “Supposing” a proposition p amounts to altering one’s probability function by “shifting the probability shares from the worlds in S to the worlds in Sp”, where Sp is the set of all worlds in S’ in which p is true. Günther then proposes that a conditional p > e holds true in an agent’s belief state just in case e is true in all the most likely worlds in Sp.
One might think that, since the assassin is usually scrupulous, it is likely—or at least possible—that she was framed. One might then think that the fingerprint evidence does not suffice for conviction. Nonetheless, this evidence surely should be admissible. The mere fact that the assassin does not usually leave fingerprints should not render her actual fingerprint inadmissible on the rare occasion when (it appears that) she has finally slipped up.
Thanks to an anonymous reviewer for suggesting that I consider this idea.
Thanks to Beth Valentine for suggesting this example to me.
Alternatively, we could generate the same results by sticking with the first-pass sensitivity and safety criteria but stipulating that the similarity relation should be specified in such a way as to minimally alter the basic facts of the case. Thanks to an anonymous reviewer for this observation.
One might think that appealing to causal relations poses no threat to criteria of admissibility that look for patterns of facts across worlds, because one might think that causal relations just are patterns of facts across worlds. And one might appeal to Lewis’s (1973b) account of causation in support of this view. However, Lewis’s view does not support the conclusion that just any old pattern of facts across worlds—such as those in the two examples above—constitutes a causal relation. On the contrary, Lewis was well-aware of the fact that these patterns do not distinguish E1’s causing E2 from E2’s causing E1 and from E1 and E2 being effects of a common cause. It is in light of these problems that Lewis revised his account by adding a no-backtracking condition and introducing the notion of chains of causal dependence; i.e., precisely the sort of chain that does not obtain between the premises and conclusions of the above inferences. Lewis (2000) further revised the account in recognition of deeper problems for counterfactual theories of causation, chiefly involving cases of preemption. Thanks to an anonymous referee for suggesting that I mention Lewis’s account here.
For some classic treatments of the topic that I have not already mentioned, see Tribe (1971)’s scathing attack on the use of mathematical reasoning in the legal process, Brilmayer and Kornhauser (1978) and Colyvan, Regan and Ferson (2002)’s individuality-based arguments, Posner (1972) and Brook (1982)’s cumulative-impact-based arguments, Nesson (1985)’s public-trust-based argument, Wasserman (1991) and Pundik (2008)’s autonomy-based arguments, and Stein (2005)’s “case specificity” argument, and compare Koehler and Shaviro (1990)’s defense of the use of naked statistical evidence by courts.
I presented previous versions of this paper at the Barcelona Workshop in Law and Philosophy in April 2021, the Truth, Power, and the Foundations of Democracy series at Dartmouth in August 2019, and the New Directions in Philosophy of Law conference at the University of Surrey in July 2019. I am grateful to all participants in these sessions for their formative feedback. I am also grateful to Maggie O’Brien, David Enoch, and Sarah Buss for reading and/or discussing the ideas in earlier drafts, and especially to Sarah Moss for reading the very first write-up of the initial idea four years ago and encouraging me to pursue it.
References
Allen, R. J. (1990). On the significance of batting averages and strikeout totals: A clarification of the naked statistical evidence debate, the meaning of evidence, and the requirement of proof beyond a reasonable doubt. Tulsa Law Review, 65, 1093–1110.
Allen, R. J. (2021). Naturalized epistemology and the law of evidence revisited. Quaestio Facti, 2, 253–284.
Alston. (1988). The deontological conception of epistemic justification. Philosophical Perspectives, 2, 257–299.
Blome-Tillmann, M. (2015). Sensitivity, causality, and statistical evidence in courts of law. Thought A Journal of Philosophy, 4(2), 102–112.
Brilmayer, L., & Kornhauser, L. (1978). Review: Quantitative methods and legal decisions. University of Chicago Law Review, 46, 116–149.
Brook, J. (1982). Inevitable errors: The preponderance of the evidence standard in civil litigation. Tulsa Law Review, 18, 79–109.
Cohen, L. J. (1977). The probable and the provable. Clarendon Press.
Colyvan, M., Regan, H. M., & Ferson, S. (2002). Is it a crime to belong to a reference class? The Journal of Political Philosophy, 9(2), 168–181.
Enoch, D., & Fisher, T. (2015). Sense and ‘sensitivity’: Epistemic approaches and instrumental approaches to statistical evidence. Stanford Law Review, 67, 557–611.
Enoch, D., & Spectre, L. (2019). Sensitivity, safety, and the law: A reply to Pardo. Legal Theory, 25(3), 178–199.
Enoch, D., Spectre, L., & Fisher, T. (2012). Statistical evidence, sensitivity, and the legal value of knowledge. Philosophy and Public Affairs, 40(3), 197–224.
Gärdenfors, P. (1982). Imaging and conditionalization. The Journal of Philosophy, 79(12), 747–760.
Guenther v. Armstrong Rubber Co., 406 F.2d 1315, 1318 (3rd Cir. 1969).
Günther, M. (2021). Epistemic sensitivity and evidence. Inquiry. https://doi.org/10.1080/0020174X.2021.1936158
Hall v. State, 692 S.W.2d 759 (Ark. Ct. App. 1985).
Hughes, Nicholas H. (forthcoming). Epistemology without guidance.
Kaminsky v. Hertz Corp., 288 N.W.2d 426 (Mich. Ct. App. 1979).
Kaye, D. (1980). Naked statistical evidence. Yale Law Journal, 89, 601.
King, J. Z., & Babic, B. (2020). Moral obligation and epistemic risk. Oxford Studies in Normative Ethics, 10, 81–105.
Koehler, J. J. (2002). When do courts think base rate statistics are relevant? Jurimetrics, 42, 380–385.
Koehler, J. J., & Shaviro, D. N. (1990). Veridical verdicts: Increasing verdict accuracy through the use of overtly probabilistic evidence and methods. Cornell Law Review, 75, 247–279.
Lewis, D. (1973a). Counterfactuals. Harvard University Press.
Lewis, D. (1973b). Causation. Journal of Philosophy, 70, 556–567.
Lewis, D. (2000). Causation as influence. Journal of Philosophy, 97, 182–197.
Littlejohn, C. (2020). Truth, knowledge, and the standard of proof in criminal law. Synthese, 192(12), 5253–5286.
Nesson, C. (1979). Reasonable doubt and permissive inferences: The value of complexity. Harvard Law Review, 92(6), 1187–1225.
Nesson, C. (1985). The evidence or the event? On judicial proof and the acceptability of verdicts. Harvard Law Review, 98(7), 1357–1392.
Nozick, R. (1981). Philosophical explanations. Harvard University Press.
Pardo, Michael (2018). Safety vs sensitivity: Possible worlds and the law of evidence.d Legal Theory 24(1):50–75.
Pollock, J. (1976). Subjunctive reasoning. D. Reidel Publishing Co.
Posner, R. (1972). Economic analysis of the law. Little, Brown.
Pritchard, Duncan (2008). Safety, sensitivity, and anti-luck epistemology. In John Greco (Ed.) The oxford handboook of skepticism. Oxford University Press.
Pundik, A. (2008). Statistical evidence and individual litigants: A reconsideration of Wasserman’s argument from autonomy. International Journal of Evidence and Proof, 12, 303–331.
Ross, L. (2021). Rehabilitating statistical evidence. Philosophy and Phenomenological Research, 102(1), 2–23.
Smith v. Rapid Transit Inc., 317 Mass. 58 N.E.2d 754 (Mass. 1945).
Smith, M. (2018). When does evidence suffice for conviction? Mind, 127(508), 1193–1218.
Sosa, E. (1999). How to defeat opposition to Moore. Philosophical Perspectives, 33, 141–153.
Srinivasan, A. (2015). Normativity without Cartesian privilege. Philosophical Issues, 25(1), 273–299.
Stalnaker, R. (1968). A theory of conditionals. In N. Rescher (Ed.), Studies in logical theory (pp. 98–112). Basil Blackwell.
State v. Maule, 667 P.2d 96 (Wash. Ct. App. 1983).
Stein, A. (2005). Foundations of evidence law. Oxford: Oxford University Press.
Stephens v. State, 774 P.2d 60 (Wyo. 1989).
Thomson, J. J. (1986). Liability and individualized evidence. Law and Contemporary Problems, 49(3), 199–219.
Tribe, L. (1971). Trial by mathematics: Precision and ritual in the legal process. Harvard Law Review, 84(6), 1329–1393.
Wasserman, D. T. (1991). The morality of statistical proof and the risk of mistaken liability. Cardozo Law Review, 13(935), 942–943.
Wells, G. L. (1992). Naked statistical evidence of liability: Is subjective probability enough? Journal of Personality and Social Psychology, 62(5), 739–752.
Wright, R. W. (1985). Causation in Tort law. California Law Review, 73, 1735–1828.
Wright, R. W. (1988). Causation, responsibility, risk, probability, naked statistics, and proof: Pruning the bramble bush by clarifying the concepts. Iowa Law Review, 73(5), 1001–1078.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
There are no conflicts of interest to report for this publication.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Johnson King, Z.A. Sensitivity, safety, and admissibility. Synthese 200, 511 (2022). https://doi.org/10.1007/s11229-022-03972-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11229-022-03972-9