Targeted Estimation of Nuisance Parameters to Obtain Valid Statistical Inference

1.1 The role of nuisance parameter estimation

Suppose we observe n independent and identically distributed copies of a random variable O with probability distribution P0. In addition, assume that it is known that P0 is an element of a statistical model M and that we want to estimate ψ0=Ψ(P0) for a given target parameter mapping Ψ:M↦IR. In order to guarantee that P0∈M one is forced to only incorporate real knowledge, and, as a consequence, such models M are always very large and, in particular, are infinite dimensional. We assume that the target parameter mapping is path-wise differentiable and let D∗(P) denote the canonical gradient of the path-wise derivative of Ψ at P∈M [1]. An estimator ψn=Ψˆ(Pn) is a functional Ψˆ applied to the empirical distribution Pn of O1,…,On and can thus be represented as a mapping Ψˆ:MNP↦IR from the non-parametric statistical model MNP into the real line. An estimator Ψˆ is efficient if and only if it is asymptotically linear with influence curve D∗(P0):

The empirical mean of the influence curve D∗(P0) represents the first-order linear approximation of the estimator as a functional of the empirical distribution, and the derivation of the influence curve is a by-product of the application of the so-called functional delta-method for statistical inference based on functionals (i.e. Ψˆ) of the empirical distribution [2–4].

Suppose that Ψ(P) only depends on P through a parameter Q(P) and that the canonical gradient depends on P only through Q(P) and a nuisance parameter g(P). The construction of an efficient estimator requires the construction of estimators Qn and gn of these nuisance parameters Q0 and g0, respectively. Targeted minimum loss-based estimation (TMLE) represents a method for construction of (e.g. efficient) asymptotically linear substitution estimators Ψ(Qn∗), where Qn∗ is a targeted update of Qn that relies on the estimator gn [5–7]. The targeting of Qn is achieved by specifying a parametric submodel {Qn(∈):∈}⊂{Q(P):P∈M} through the initial estimator Qn and a loss function O↦L(Q)(O) for Q0=argminQP0L(Q)≡∫L(Q)(o)dP0(o), so that the generalized score dd∈L(Qn(∈))|∈=0 spans a desired user-supplied estimating function D(Qn,gn). In addition, one may decide to target gn by specifying a parametric submodel {gn(∈1):∈1}⊂{g(P):P∈M} and loss function O↦L(g)(O) for g0=argmingP0L(g), so that the generalized score dd∈1L(gn(∈1))|∈1=0 spans another desired estimating function D1(gn,ηn) for some estimator ηn of nuisance parameter η. The parameter ∈ is fitted with MLE ∈n=argmin∈PnL(Qn(∈)), providing the first-step update Qn1=Qn(∈n), and similarly ∈1,n=argmin∈1PnL(gn(∈1)). This updating process that mapped a current fit (Qn,gn) into an update (Qn1,gn1) is iterated till convergence at which point the TMLE (Qn∗,gn∗) solves PnD(Qn∗,gn∗)=0, i.e. the empirical mean of the estimating function equals zero at the final TMLE (Qn∗,gn∗). If one also targeted gn, then it also solves PnD1(gn∗,ηn)=0. The submodel through Qn will depend on gn, while the submodel through gn will depend on another nuisance parameter ηn. By setting D(Q,g) equal to the efficient influence curve D∗(Q,g), the resulting TMLE solves the efficient influence curve estimating equation PnD∗(Qn∗,gn)=0 and thereby will be asymptotically efficient when (Qn,gn) is consistent for (Q0,g0) under appropriate regularity conditions, where the targeting of gn is not needed.

The latter is shown as follows. By the property of the canonical gradient (in fact, any gradient) we have Ψ(Qn∗)−Ψ(Q0)=−P0D∗(Qn∗,gn)+Rn(Qn∗,Q0,gn,g0), where Rn involves integrals of second-order products of the differences (Qn∗−Q0) and (gn−g0). Combined with PnD∗(Qn∗,gn)=0, this implies the following identity:

The first term is an empirical process term that, under empirical process conditions (mentioned below), equals (Pn−P0)D∗(Q,g), where (Q,g) denotes the limit of (Qn∗,gn), plus an oP(1/n)-term. This then yields

To obtain the desired asymptotic linearity of Ψ(Qn∗) one needs Rn=oP(1/n), which in general requires at minimal that both nuisance parameters are consistently estimated: Q=Q0 and g=g0. However, in many problems of interest, Rn only involves a cross-product of the differences Qn∗−Q0 and gn−g0, so that Rn converges to zero if either Qn∗ is consistent or gn is consistent: i.e. Q=Q0 or g=g0. In this latter case, the TMLE is so-called double robust. Either way, the consistency of the TMLE relies now on one of the nuisance parameter estimators being consistent, thereby requiring the use of non-parametric adaptive estimation such as super-learning [8–10] for at least one of the nuisance parameters. If only one of the nuisance parameter estimators is consistent, and we are in the double robust scenario, then it follows that the bias is of the same order as the bias of the consistent nuisance parameter estimator. However, if the nuisance parameter estimator is not based on a correctly specified parametric model, but instead is a data-adaptive estimator, then this bias will be converging to zero at a rate slower than 1/n: i.e. nRn converges to infinity as n ↦ ∞. Thus, in that case, the estimator of the target parameter may thus be overly biased and thereby will not be asymptotically linear.

1.2 Targeting the fit of the nuisance parameter: general approach

In this article, we demonstrate that if Q≠Q0, then it is essential that the consistent nuisance parameter estimator gn be targeted toward the estimand so that the bias for the estimand becomes second order: that is, in our new TMLEs relying on consistent estimation of g0 presented in this article one simultaneously updates gn into a gn∗ so that certain smooth functionals of gn∗, derived from the study of Rn, are asymptotically linear under appropriate conditions. Even if both estimators Qn∗ and gn∗ are consistent, but Qn∗ might be converging at a slower rate than gn∗, this targeting of the nuisance parameter estimator may still remove finite sample bias for the estimand. In addition, we also present such TMLE when only relying on one of the nuisance parameters to be consistently estimated, but not knowing which one: i.e. either Q=Q0 or g=g0. The same arguments applies to other double robust estimators, such as estimating equation based estimators and inverse probability of treatment weighted (IPTW) estimators [11–16]. In fact, we demonstrate such a targeted IPTW-estimator in our next section.

The current article concerns the construction of such targeted IPTW and TMLE that are asymptotically linear under regularity conditions, even when only one of the nuisance parameters is consistent and the estimators of the nuisance parameters are highly data adaptive. In order to be concrete in this article, we will focus on a particular example. In such an example we can concretely present the second-order term Rn mentioned above and thereby develop the concrete form of the TMLE.

The same approach for construction of such TMLE can be carried out in much greater generality, but that is beyond the scope of this article. Nonetheless, it is helpful for the reader to know that the general approach is the following (considering the case that g=g0, but Q can be misspecified): (1) approximate Rn(Qn∗,Q0,gn∗,g0)=Φ0,n(gn∗)−Φ0,n(g0)+R1,n for some mapping Φ0,n that depends on P0 (e.g. through Q0) and the data (e.g. Qn∗,gn∗), and where R1,n is a second-order term so that it is reasonable to assume R1,n=oP(1/n); (2) approximate Φ0,n(gn∗)−Φ0,n(g0)=Φn(gn∗)−Φn(g0)+R2,n, where R2,n is a second-order term and Φn is now a known (only based on data) mapping approximating Φ0; (3) construct gn∗ so that it is a TMLE of the target parameter Φn(g0) thereby allowing an expansion Φn(gn∗)−Φn(g0)=(Pn−P0)D1,n(P0)+R3,n with D1,n(P0) being the efficient influence curve of Φn(g0). That is, in step 3, gn∗ is iteratively updated to solve PnD1,n(gn∗,ηn)=0 with D1,n(P0) depending on P0 through g0 and a nuisance parameter η0, so that Φn(gn∗) is an asymptotically linear estimator of Φn(g0) under regularity conditions. After these three steps, we have that Rn(Qn∗,Q0,gn∗,g0)=(Pn−P0)D1,n(P0)+R1,n+R2,n+R3,n, where R1,n+R2,n+R3,n=oP(1/n), and these steps provide us with the parameter Φn(g0) that needs to be targeted by gn∗, thereby telling us how to target gn∗ in the TMLE of ψ0. In addition, we can then conclude that this TMLE is asymptotically linear with known influence curve D∗(Q,g0)+D1(P0), where D1(P0) represents the limit of the efficient influence curve D1,n(P0) of Φn(g0): Ψ(Qn∗)−Ψ(Q0)=(Pn−P0){D∗(Q,g0)+D1(P0)}+oP(1/n).

1.3 Concrete example covered in this article

Let us now formulate our concrete example we will cover in this article. Let O=(W,A,Y)∼P0, W baseline covariates, A a binary treatment, and Y a final outcome. Let M be a model that makes at most some assumptions about the conditional distribution of A, given W, but leaves the marginal distribution of W and the conditional distribution of Y, given A,W, unspecified. Let Ψ:M↦IR be defined as Ψ(P)=EPEP(Y|A=1,W), the so-called treatment specific mean controlling for the baseline covariates. The canonical gradient, also called the efficient influence curve, of Ψ at P is given by D∗(P)(O)=A/g(1|W)(Y−Qˉ(1,W))+Qˉ(1,W)−Ψ(P), where g(1|W)=P(A=1|W) is the propensity score and Qˉ(a,W)=EP(Y|A=a,W) is the outcome regression [13]. Let Q=(QW,Qˉ), where QW is the marginal distribution of W, and note that Ψ(P) only depends on P through Q=Q(P). For convenience, we will denote the target parameter with Ψ(Q) in order to not have to introduce additional notation. A targeted minimum loss-based estimator (TMLE) is a plug-in estimator Ψ(Qn∗), where Qn∗ is an update of an initial estimator Qn that relies on an estimator gn of g0, and it has the property that it solves PnD∗(Qn∗,gn)=0, where we used the notation Pf=∫f(o)dP(o).

For this particular example, such TMLE are presented in Scharfstein et al. [17]; van der Laan and Rubin [7]; Bembom et al. [18–21]; Rosenblum and van der Laan [22]; Sekhon et al. [23]; van der Laan and Rose [6, 24]. Since P0D∗(Q,g)=ψ0−Ψ(Q)+P0(Qˉ0−Qˉ)(gˉ0−gˉ)/gˉ [25, 26], where we use the notation gˉ(W)=g(1|W) and Qˉ(W)=Qˉ(1,W), and PnD∗(Qn∗,gn)=0, we obtain the identity:

The first term equals (Pn−P0)D∗(Q,g)+oP(1/n) if D∗(Qn∗,gn) falls in a P0-Donsker class with probability tending to 1, and P0{D∗(Qn∗,gn)−D∗(Q,g)}2↦0 in probability as n↦∞ [4, 27]. If Qˉn∗ and gˉn are consistent for the true Qˉ0 and gˉ0, respectively, then the second term is a second-order term. If one now assumes that this second-order term is oP(1/n), it has been proven that the TMLE is asymptotically efficient. This provides the general basis for proving asymptotic efficiency of TMLE when both Q0 and g0 are consistently estimated.

However, if only one of these nuisance parameter estimators is consistent, then the second term is still a first-order term, and it remains to establish that it is also asymptotically linear with a second-order remainder. For sake of discussion, suppose that Qˉn∗ converges to a wrong Qˉ while gˉn is consistent. In that case, this remainder behaves in first order as P0(Qˉ0−Qˉ)(gˉn−gˉ0)/gˉ0. To establish that such a term is asymptotically linear requires that gˉn solves a particular estimating equation: that is, gˉn needs to be a TMLE itself targeting the required smooth functional of g0. This is naturally achieved within the TMLE framework by specifying a submodel through gn and loss function with the appropriate generalized score, so that a TMLE update step involves both updating Qn and gn, and the iterative TMLE algorithm now results in a final TMLE (Qn∗,gn∗), not only solving PnD∗(Qn∗,gn∗)=0 but also these additional equations that allow us to establish asymptotic linearity of the desired smooth functional of gn∗: see general description of TMLE above.

In this article, we present TMLE that targets gn in a manner that allows us to prove the desired asymptotic linearity of the second term in the right-hand side of eq. (1) when either gˉn or Qˉn is consistent, under conditions that require specified second-order terms to be oP(1/n). The latter type of regularity conditions are typical for the construction of asymptotically linear estimators and are therefore considered appropriate for the sake of this article. Though it is of interest to study cases in which these second-order terms cannot be assumed to be oP(1/n), this is beyond the scope of this article.

1.4 Relation to current literature on targeted nuisance parameter estimators

The construction of TMLE that utilizes targeting of the nuisance parameter gn has been carried out in earlier papers. For example, in van der Laan and Rubin [7], we target gn to obtain a TMLE that, beyond being double robust locally efficient, also equals the IPTW-estimator. In Gruber and van der Laan [29] we target gn to guarantee that, beyond being double robust locally efficient, also outperforms a user-supplied given estimator, based on the original idea of Rotnitzky et al. [28]. In that sense, the distinction of the current article with these previous articles is that gn∗ is now targeted to guarantee that the TMLE remains asymptotically linear when Qn∗ is misspecified. This task of targeting gn∗ appears to be one step more complicated than in these previous articles, since the smooth functionals of gn∗ that need to be targeted are themselves indexed by parameters of the true data distribution P0, and thus unknown. As mentioned above, our strategy is to approximate these unknown smooth functionals by an estimated smooth functional and develop the targeted estimator gn∗ that targets this estimated parameter of g0.

The TMLEs presented in this article are always iterative and thereby rely on convergence of the iterative updating algorithm. Since the empirical risk increases at each updating step, such convergence is typically guaranteed by the existence of the MLE at each updating step (e.g. an MLE of coefficient in a logistic regression). Either way, in this article, we assume this convergence to hold. Since our assumptions of our theorems require gn∗(1|W) to be bounded away from zero, we demonstrate how this property can be achieved by using submodels for updating gn that guarantee this property. Detailed simulations will appear in a future article.

1.5 Organization

The organization of this paper is as follows. In Section 2, we introduce a targeted IPTW-estimator that relies on an adaptive consistent estimator of g0, and we establish its asymptotic linearity with known influence curve, allowing for the construction of asymptotically valid confidence intervals based on this adaptive IPTW-estimator. In the remainder of the article, we focus on construction of TMLE involving the targeting of gn to establish the asymptotic linearity of the resulting TMLE under appropriate conditions. In Section 3, we introduce a novel TMLE that assumes that the targeted adaptive estimator gn∗ is consistent for g0, and we establish its asymptotic linearity. In Section 4, we introduce a novel TMLE that only assumes that either the targeted Qˉn∗ or the targeted gˉn∗ is consistent, and we establish its asymptotic linearity with known influence curve. This TMLE needs to protect the asymptotic linearity under misspecification of either gn∗ or Qˉn∗, and, as a consequence, relies on targeting of gn (in order to preserve asymptotic linearity when Qˉn∗ is inconsistent), but also extra targeting of Qˉn (in order to preserve asymptotic linearity when Qˉn∗ is consistent, but gn is inconsistent). The explicit form of the influence curve of this TMLE allows us to construct asymptotic confidence intervals. Since this result allows statistical inference in the statistical model that only assumes that one of the estimators is consistent, and we refer to this as “double robust statistical inference”. Even though double robust estimators have been extensively presented in the current literature, double robust statistical inference in these large semi-parametric models has been a difficult topic: typically, one has suggested to use the non-parametric bootstrap, but there is no theory supporting that the non-parametric bootstrap is a valid method when the estimators rely on data-adaptive estimation.

In Section 5, we extend the TMLE of Section 3 (that relies on gn∗ being consistent for g0) to the case that gn∗ converges to a possibly misspecified g but one that suffices for consistent estimation of ψ0 in the sense that Ψ(Qˉn∗) will be consistent. We present a corresponding asymptotic linearity theorem for this TMLE that is able to utilize the so-called collaborative double robustness of the efficient influence curve which states that Ψ(Q)=ψ0 if P0D∗(Q,g)=0 and g∈G(Q,P0) for a set G(Q,P0) (including g0). In order to construct a collaborative estimator gn∗ that aims to converge to an element in G(Qn∗,P0) in collaboration with Qn∗, we use the framework of collaborative targeted minimum loss-based estimator (C-TMLE) [20, 29–35]. Our asymptotic linearity theorem can now be applied to this C-TMLE. Again, even though C-TMLEs have been presented in the current literature, statistical inference based on the C-TMLEs has been another challenging topic, and Section 5 provides us with a C-TMLE with known influence curve. We conclude this article with a discussion. The proofs of the theorems are presented in the Appendix.

1.6 Notation

In the following sections, we will use the following notation. We have O=(W,A,Y)∼P0∈M, where M is a statistical model that makes only assumptions on the conditional distribution of A, given W. Let g0(a|W)=P0(A=a|W), and gˉ0(W)=P0(A=1|W). The target parameter is Ψ:M→IR defined by Ψ(P0)=EQW,0Qˉ0(1,W), where Qˉ0(1,W)=EP0(Y|A=1,W), which will also be denoted with Qˉ0(W), and QW,0 is the distribution of W under P0. We also use the notation Ψ(Q), where Q=(QW,Qˉ). In addition, D∗(Q,g) denotes the efficient influence curve of Ψ at (Q,g). We also use the following notation:

2.1 An IPTW-estimator using super-learning to fit the treatment mechanism

We consider a simple IPTW-estimator Ψˆ(Pn)=PnD(gˆ(Pn)), where D(g)(O)=YA/gˉ(W), and gˆ:MNP↦G is an adaptive estimator of g0 based on the log-likelihood loss function L(g)(O)≡−logg(A|W). For a general presentation of an IPTW-estimator, we refer to Robins and Rotnitzky [11], van der Laan and Robins [13], and Hernan et al. [36]. We wish to establish conditions under which reliable statistical inference based on this estimator of ψ0 can be obtained. One might wish to estimate g0 with ensemble learning, and, in particular, super-learning in which cross-validation [37] is used to determine the best weighted combination of a library of candidate estimators: van der Laan and Dudoit [8]; van der Laan et al. [9, 38, 39]; van der Vaart et al. [10]; Dudoit and van der Laan [40]; Polley et al. [41]; Polley and van der Laan [42]; van der Laan and Petersen [43]. The super-learner is a general template for construction of an adaptive estimator based on a library of candidate estimators, a loss function whose expectation is minimized over the parameter space by the true parameter value, a parametric family that defines “weighted” combinations of the estimators in the library. We will start with presenting a succinct description of a particular super-learner. Consider a library of estimators gˆj:MNP↦G, j=1,…,J and a family of weighted (on logistic scale) combinations of these estimators Logit gˆα(1|W)=∑j=1JαjLogit gˆj(1|W), indexed by vectors α for which αj∈[0,1] and ∑jαj=1. Consider a random sample split Bn∈{0,1}n into a training sample {i:Bn(i)=0} of size n(1−p) and validation sample {i:Bn(i)=1} of size np, and let Pn,Bn1 and Pn,Bn0 denote the empirical distribution of the validation sample and training sample, respectively. Define

as the choice of estimator that minimizes cross-validated risk. The super-learner of g0 is defined as the estimator gˆ(Pn)=gˆαn(Pn).

2.2 Asymptotic linearity of a targeted data-adaptive IPTW-estimator

The next theorem presents an IPTW-estimator that uses a targeted fit gn∗ of g0, involving the updating of an initial estimator gn, and conditions under which this IPTW-estimator of ψ0 is asymptotically linear. For example, gn could be defined as a super-learner of the type presented above. In spite of the fact that such an IPTW-estimator uses a very data adaptive and hard to understand estimator gn, this theorem shows that its influence curve is known and can be well estimated.

Theorem 1We consider a targeted IPTW-estimatorΨˆ(Pn)=PnD(gn∗), whereD(g)(O)=YA/g(A|W), andgn∗is an update of an initial estimatorgnofg0∈Gdefined below.

Definition of targeted estimatorgn∗: LetQˉnr∗be obtained by non-parametric estimation of the regression functionEP0(Y|A=1,gˉn(W))treatinggˉnas a fixed covariate (i.e. function of W). This yields an estimatorHnr∗≡Qˉnr∗/gˉnofH0r=Qˉ0r/gˉ0, whereQˉ0r=EP0(Y|A=1,gˉ0). Consider the submodelLogit gˉn(∈)=Logit gˉn+∈Hnr∗, and fit∈with the MLE

We definegn∗=gn(∈n)as the corresponding targeted update ofgn. This TMLEgn∗satisfies

Empirical process condition: Assume thatD(gn∗),DA(Qˉnr∗,gˉn∗)fall in aP0-Donsker class with probability tending to 1.

Negligibility of second-order terms: DefineQˉ0,nr≡EP0(Y|A=1,gˉ0(W),gˉn∗(W)). Assumegˉn∗>δ>0with probability tending to 1 and assume

Then,

where

So under the conditions of this theorem, we can construct an asymptotic 0.95-confidence interval ψn±1.96σn/n based on this targeted IPTW-estimator ψn=Ψˆ(Pn), where

and ICn(O)=YA/gˉn∗(W)−ψn−Hnr∗(W)(A−gˉn∗(W)) is the plug-in estimator of the influence curve IC(P0) obtained by plugging in gn or gn∗ for g0 and Qˉnr∗ for Qˉ0r.

Regarding the displayed second-order term conditions, we note that these are satisfied if gˉn∗−gˉ0 converges to zero w.r.t. L2(P0)-norm at rate oP(n−1/4), gˉn∗>δ>0 for some δ>0 with probability tending to 1 as n ↦ ∞, and the product of the rates at which gˉn∗ converges to gˉ0 and (Qˉnr∗,Qˉ0,nr) converges to Qˉ0r is oP(1/n).

Regarding the empirical process condition, we note that an example of a Donsker class is the class of multivariate real-valued functions with uniform sectional variation norm bounded by a universal constant [44]. It is important to note that if each estimator in the library falls in such a class, then also the convex combinations fall in that same class [4]. So this Donsker condition will hold if it holds for each of the candidate estimators in the library of the super-learner.

2.3 Comparison of targeted data-adaptive IPTW and an IPTW using parametric model

Consider an IPTW-estimator using a MLE gn,1 according to a parametric model for g0, and let us contrast this IPTW-estimator with an IPTW-estimator defined in the above theorem based on an initial super-learner gn that includes gn,1 as an element of the library of estimators. Let us first consider the case that the parametric model is correctly specified. In that case gn,1 converges to g0 at a parametric rate 1/n. From the oracle inequality for cross-validation [8, 10, 38], it follows that gn also converges at the rate 1/n to g0 possibly up to a log n-factor in case the number of algorithms in the library is of the order nq for some fixed q. As a consequence, all the consistency and second-order term conditions for the IPTW-estimator using a targeted gn∗ based on gn hold. If one uses estimators in the library of algorithms that have a uniform sectional variation norm smaller than a M<∞ with probability tending to 1, then also a weighted average of these estimators will have uniform sectional variation norm smaller than M<∞ with probability tending to 1. Thus, in that case we will also have that D(gn∗),DA(Qˉnr∗,gˉn∗) fall in a P0-Donsker class. Examples of estimators that control the uniform sectional variation norm are any parametric model with fewer than K main terms that themselves have a uniform sectional variation norm, but also penalized least-squares estimators (e.g. Lasso) using basis functions with bounded uniform sectional variation norm, and one could map any estimator into this space of functions with universally bounded uniform sectional variation norm through a smoothing operation. Thus, under this restriction on the library, the IPTW-estimator using the super-learner is asymptotically linear with influence curve IC(P0)(O) as stated in the theorem. We note that IC(P0) is the efficient influence curve for the target parameter EP0EP0(Y|A=1,gˉ0(W)) if the observed data were (gˉ0(W),A,Y) instead of O=(W,A,Y).

The parametric IPTW-estimator is asymptotically linear with influence curve O↦YA/g0(A|W)−ψ0−Π(YA/gˉ0(W)|Tg), where Tg is the tangent space of the parametric model for g0, and Π(f|Tg) denotes the projection of f onto Tg in the Hilbert space L02(P0) [13]. This IPTW-estimator could be less or more efficient than the IPTW-estimator using the targeted super-learner depending on the actual tangent space of the parametric model.

For example, if the parametric model happens to have a score equal to O ↦ Qˉ0(W)(A/gˉ0(W)−1), then the parametric IPTW-estimator would be asymptotically efficient. Of course, a standard parametric model is not tailored to correspond with such optimal scores, but this shows that we cannot claim superiority of one versus the other in the case that the parametric model for g0 is correctly specified.

If, on the other hand, the parametric model is misspecified, then the IPTW-estimator using gn,1 is inconsistent. However, the super-learner gn will be consistent if the library contains a non-parametric adaptive estimator, and will perform asymptotically as well as the oracle selector among all the weighted combinations of the algorithms in the library. To conclude, the IPTW-estimator using super-learning to estimate g0 will be as good as the IPTW-estimator using a correctly specified parametric model (included in the library of the super-learner), but will remain consistent and asymptotically linear in a much larger model than the parametric IPTW-estimator relying on the true g0 being an element of the parametric model.

3.1 Asymptotic linearity of a TMLE using a targeted estimator of the treatment mechanism

The following theorem presents a novel TMLE and corresponding asymptotic linearity with specified influence curve, where we rely on consistent estimation of g0. The TMLE still uses the same updating step for the estimator of Qˉ0 as the regular TMLE [7], but uses a novel updating step for the estimator of g0, analogue to the updating step of the IPTW-estimator in the previous section. We remind the reader of the importance of using the logistic fluctuations as working-submodels for Qˉ0 in the definition of the TMLE, guaranteeing that the TMLE update stays within the bounded parameter space (see, e.g. Gruber and van der Laan [19]).

Theorem 2

Iterative targeted MLE ofψ0:

Definitions: GivenQˉ,gˉ, letQˉnr(Qˉ,gˉ)be a consistent estimator of the regressionQˉ0r(Qˉ,gˉ)=EP0(Y−Qˉ|A=1,gˉ)of(Y−Qˉ)ongˉ(W)andA=1. Let(gn,Qˉn)be an initial estimator of(g0,Qˉ0).

Initialization: Letgn0=gn,Qˉn0=Qˉn, andQˉnr0=Qˉnr(Qˉn0,gˉn0). Letk=0.

Updating step forgnk: Consider the submodelLogit gˉnk(∈)=Logit gˉnk+∈HA(Qˉnrk,gˉnk), and fit∈with the MLE

We definegnk+1=gnk(∈n)as the corresponding update ofgnk. Thisgnk+1satisfies

Updating step forQˉnk: Let−L(Qˉ)(O)≡YlogQˉ(A,W)+(1−Y)log(1−Qˉ(A,W))be the quasi-log-likelihood loss function forQˉ0=E0(Y|A=1,W)(allowing that Y is continuous in[0,1]). Consider the submodelLogit Qˉnk(∈)=Log itQˉnk+∈HY(gnk), and let∈n=argmin∈PnL(Qˉnk(∈)). DefineQˉnk+1=Qˉnk(∈n)as the resulting update. DefineQˉnrk+1=Qˉnr(Qˉnk+1,gˉnk+1).

Iterating till convergence: Now, setk←k+1, and iterate this updating process mapping a(gnk,Qˉnk,Qˉnrk)into(gnk+1,Qˉnk+1,Qˉnrk+1)till convergence or till large enough K so that the estimating equations (2) below are solved up till anoP(1/n)-term. Denote the limit of this iterative procedure with(gn∗,Qˉn∗,Qˉnr∗).

Plug-in estimator: LetQn∗=(QW,n,Qˉn∗), whereQW,nis the empirical distribution estimator ofQW,0. The TMLE ofψ0is defined asΨ(Qn∗).

Estimating equations solved by TMLE: This TMLE(Qn∗,gn∗,Qˉnr∗)solves

Empirical process condition: Assume thatD∗(Qn∗,gn∗), DA(Qˉnr∗,gˉn∗)falls in aP0-Donsker class with probability tending to 1 asn ↦∞.

Negligibility of second-order terms: Define

wheregˉn∗(W)is treated as a fixed covariate (i.e. function of W) in the conditional expectationQˉ0,nr. Assume that there exists aδ>0, so thatgˉn∗>δ>0with probability tending to 1, and

Then,

whereIC(P0)=D∗(Q,g0)−DA(Qˉ0r,gˉ0).

Thus, under the assumptions of this theorem, an asymptotic 0.95-confidence interval is given by ψn∗±1.96σn/n, where σn2=PnICn2, and ICn=D∗(Qn∗,gn∗)−DA(Qˉnr∗,gˉn∗).

3.2 Using a δ-specific submodel for targeting g that guarantees the positivity condition

The following is an application of the constrained logistic regression approach of the type presented in Gruber and van der Lann [19] for the purpose of estimation of gˉ0 respecting the constraint that gˉ0>δ>0 for a known δ>0. Recall that A∈{0,1}. Suppose that it is known that gˉ0(W)∈(δ,1] for some δ>0, a condition the asymptotic linearity of our proposed estimators relies upon. Define Aδ≡A−δ1−δ. We have gˉ0(W)=δ+(1−δ)gˉ0,δ, where gˉ0,δ=E0(Aδ|W) is a regression that is known to be between [0,1]. Let gδ,n0 be an initial estimator of the true conditional distribution gδ,0 of Aδ, given W, which implies an estimator gˉn0=δ+(1−δ)gˉδ,n0 of gˉ0. Let k=0. Consider the following submodel for the conditional distribution of Aδ, given W, through a given estimator gδ,nk:

The MLE is simply obtained with logistic regression of Aδ on W (see, e.g. Gruber and van der Lann [19]) based on the quasi-log-likelihood loss function:

where

is the quasi-log-likelihood loss. The update gˉδ,nk+1=gˉδ,nk(∈n) implies an update gˉnk+1=δ+(1−δ)gˉδ,nk+1 of gˉnk=δ+(1−δ)gˉδ,nk, and, by construction gˉnk+1>δ>0. The above submodel gˉnk(∈)=δ+(1−δ)gˉδ,nk(∈) and corresponding loss function L(gˉ)=L(gˉδ) generates the same score equation as the submodel and loss function used in Theorem 2. Therefore, the TMLE algorithm presented in Theorem 2 but now using this δ-specific logistic regression model solves the same estimating equations, so that the same Theorem 2 immediately applies. However, using this submodel we have now guaranteed that gˉnk>δ>0 for all k in the iterative TMLE algorithm, and thereby that gˉn∗>δ>0.