Bayes factor functions for reporting outcomes of hypothesis tests

Significance Bayes factors represent an informative alternative to P-values for reporting outcomes of hypothesis tests. They provide direct measures of the relative support that data provide to competing hypotheses and are able to quantify support for true null hypotheses. However, their use has been limited by several factors, including the requirement to specify alternative hypotheses and difficulties encountered in their calculation. Bayes factor functions (BFFs) overcome these difficulties by defining Bayes factors from classical test statistics and using standardized effect sizes to define alternative hypotheses. BFFs provide clear summaries of the outcome from a single experiment, eliminate arbitrary significance thresholds, and are ideal for combining evidence from replicated studies.

[7] 25 Substituting for a and dividing m1(z | τ 2 ) by m0(z) produces the Bayes factor appearing in Theorem 1 of the article.
31 m1(t) can be expressed by integrating the integral form of the non-central t density from (2) with respect to a J(0, τ 2 ) prior 32 density to obtain Again letting a = τ 2 /(τ 2 + 1) and noting that application of Fubini's theorem implies that the integral with respect to λ is proportional to the second moment of a normal 37 density with variance a and mean (ayt/d). Thus, . [13]
and dividing in equation (26) yields the Bayes factor stated in the theorem.
As an aside, the m1(h) can be expressed as . [28]

74
Proof of Theorem 4. Under H0, the marginal density of the f statistic is The marginal density of f under H1 is obtained by 77 integrating the non-central f density (2) with respect to the gamma prior on its non-centrality parameter λ: and expanding Γ(r + k 2 + m 2 ) as an integral allows the marginal density to be expressed as The series in equation (35) can be summed and is equal to where b(x) is the bracketed term raised to power r. Letting c = b(x)/x, the integration with respect to x represents the sum of 90 two scaled gamma functions, leading to [37]

104
As stated in the main article, our criteria for selecting τ 2 in the specification of the prior distribution on the non-centrality 105 parameters of the test statistics is to select τ 2 so that the prior modes on the non-centrality parameters correspond to a  Table 1 follow. hypothesis is H0 : µ1 = µ2 and either Theorem 1 (σ 2 known) or Theorem 2 (σ 2 unknown) may be applied, with test statistics The standardized effect size for both tests is ω = (µ1 − µ2)/( √ 2σ), and the non-centrality parameter is 126 defined for counts n k in K cells, an s dimensional parameter θ ∈ Θ with 1 ≤ s < K, and a k × 1 vector-valued function [47] 134 We define the standardized effect size vector as [48]

136
The mode of the prior density on the non-centrality parameter in Theorem 3 occurs at kτ 2 , and the non-centrality parameter 137 can be written as nω ′ ω. Matching the mode of the prior density to the non-centrality parameter leads to 138 τ 2 = nω ′ ω k . [49]

139
In some applications, it is convenient to replace ω ′ ω by kω 2 , where ω 2 = 1 k k i=1 ω 2 i , the average squared standardized effect.
142 for a p × 1 vector β and n × p matrix X of rank p < n. The null hypothesis can be expressed as H0 : Aβ = a where the rank 143 of A is k ≤ p. The F statistic against the alternative hypothesis H1 : Aβ ̸ = a can be expressed as where RSS0 is the constrained residual sum-of-squares under the null model and RSS1 is the residual sum-of-squares under 146 the unconstrained model.

147
As demonstrated in, for example (5), the non-centrality parameter for the F statistic can be expressed as Letting L denote the Cholesky decomposition of V, i.e., LL ′ = V, we define the standardized effect size vector ω as [53]

151
It follows that the non-centrality parameter can be written [54]

155
As for count models, it is sometimes convenient to replace ω ′ ω by kω 2 , where Assuming regularity conditions in (6), h converges to a χ 2 distribution on k degrees of freedom and non-centrality parameter [57] 164 Let L denote the Cholesky decomposition of Vr and define 165 ω = L −1 (θr − θr0). [58] 166 Then λ = nω ′ ω, and setting λ = kτ 2 implies 167 τ 2 = nω ′ ω k . [59] 168 For independent and identically distributed observations, V −1 r has elements [60] The examples in the main article depicted BFFs in which there was positive support for alternative hypotheses centered on 175 some subset of standardized effect sizes. In this section, we briefly examine the shape of BFF curves when data provide no or 176 little support for any alternative hypotheses. For simplicity, we restrict attention to z tests of a normal mean with a sample 177 size n = 10, 000. The figure below depicts BFFs for z statistics ranging from 0 to 2.5.
178 Figure S1 shows that for large values of n (e.g., n = 10, 000), BFFs decrease from 1 as the standardized effect increases for