Tension in the Recent Type Ia Supernovae Datasets

In the present work, we investigate the tension in the recent Type Ia supernovae (SNIa) datasets Constitution and Union. We show that they are in tension not only with the observations of the cosmic microwave background (CMB) anisotropy and the baryon acoustic oscillations (BAO), but also with other SNIa datasets such as Davis and SNLS. Then, we find the main sources responsible for the tension. Further, we make this more robust by employing the method of random truncation. Based on the results of this work, we suggest two truncated versions of the Union and Constitution datasets, namely the UnionT and ConstitutionT SNIa samples, whose behaviors are more regular.


I. INTRODUCTION
The current accelerated expansion of our universe (see e.g.[1]) has been one of the most active fields in modern cosmology since its discovery from the observations of Type Ia supernovae (SNIa) [2].Later on, the observations of the cosmic microwave background (CMB) anisotropy [3,4] and the large-scale structure (LSS) [5] confirmed this discovery.Although there are already many observational methods to date, SNIa have been proven to be one of the most powerful tools to probe this mysterious phenomenon.In the passed decade, many SNIa datasets have been released, while the number and quality of SNIa have continually increased.The most familiar SNIa datasets include, for instance, Gold04 [6], Gold06 [7], SNLS [8], ESSENCE [9], SDSS [34], Davis [10], and, most recently, Union [11], Constitution [12].
In [11], 414 SNIa from some heterogeneous compilations have been analyzed with the same analysis procedure.After selection cuts, this compilation was reduced to 307 SNIa, which have been named the Union dataset [11].There are 250 high redshift SNIa (z > 0.2) and 57 low redshift SNIa (z ≤ 0.2) in the Union dataset [11].Very recently, the CfA3 sample [13] has been added to the 307 SNIa Union dataset to form the Constitution dataset [12].Originally, the CfA3 sample consisted of 185 SNIa, which are all at the fairly low redshift, i.e., z < 0.08.After applying the same Union cuts, 90 SNIa from CfA3 survive.This increases the low redshift sample (z ≤ 0.2) to 147 SNIa in the resulting Constitution dataset [12].The 397 SNIa Constitution dataset is the largest published, spectroscopically confirmed sample to date.
Soon after the release of the 397 SNIa Constitution dataset, some authors have used it to study dark energy.For example, the authors of [14] smoothly reconstructed the deceleration parameter q(z) and the Om(z) diagnostic by using the Constitution dataset, and found that the cosmic acceleration might be slowing down.This is an unexpected result in some sense.The authors of [15] compared the holographic dark energy model [16], the Ricci dark energy model [17] and the new agegraphic dark energy model [18], by using the Constitution dataset and other observational data; they found that the holographic dark energy model is more favored.This result is also somewhat different from the previous studies.Further, the authors of [19] found that dark energy seemingly did not exist in the past and suddenly emerged at redshift z ∼ 0.331, by fitting to the Constitution dataset alone.Needless to say, this is a striking claim.In addition, the authors of [20] found that the dark energy equation of state parameter (EoS) deviates from the cosmological constant at z ∼ > 0.5 significantly, by using the SNIa dataset (Union or Constitution) together with the gamma-ray bursts (GRBs) data; they noted that the result from Constitution dataset is somewhat different from others.In [20], they also suggested that the deviation might arise from some biasing systematic errors in the SNIa and/or GRBs datasets.
These unusual results bring our attention from dark energy to the SNIa dataset itself, especially the Constitution SNIa dataset.In fact, this situation is reminiscent of the one of Gold04 and Gold06 SNIa datasets, which also bring some interesting results.In [21,22], the Gold04 dataset has been shown to be in 2σ tension with the SNLS dataset and the WMAP observations.Although the Gold04 sample updated to Gold06 several years later, the tension still persisted.In [23], the tension and systematics in the Gold06 SNIa dataset has been investigated in great detail.In the present work, we will follow the method used in [23] to study the tension in the recent SNIa datasets.

II. OBSERVATIONAL DATA
Before plunging into the issue of the tension in the recent SNIa datasets, here we briefly present the cosmological model and the observational data.As is well known, the SNIa data points are given in terms of the distance modulus µ obs (z i ).The theoretical distance modulus is defined as where µ 0 ≡ 42.38 − 5 log 10 h and h is the Hubble constant H 0 in units of 100 km/s/Mpc, whereas in which E ≡ H/H 0 and H is the Hubble parameter; p denotes the model parameters.In the present work, we consider the familiar Chevallier-Polarski-Linder (CPL) model [24], in which the EoS of dark energy is parameterized as where w 0 and w a are constants.As is well known, the corresponding E(z) is given by [25][26][27] where Ω m0 is the present fractional energy density of pressureless matter.The χ 2 from the SNIa data is given by where σ is the corresponding 1σ error.The parameter µ 0 is a nuisance parameter but it is independent of the data points.One can perform an uniform marginalization over µ 0 .However, there is an alternative way.Following [22,28,29], the minimization with respect to µ 0 can be made by expanding the χ 2 µ of Eq. ( 5) with respect to µ 0 as where .
Eq. ( 6) has a minimum for Since χ 2 µ, min = χ2 µ, min obviously, we can instead minimize χ2 µ which is independent of µ 0 .Note that the above summations are over the whole SNIa dataset.
There are some other observational data relevant to this work, such as the observations of the cosmic microwave background (CMB) anisotropy [3,4] and the large-scale structure (LSS) [5].However, using the full data of the CMB and the LSS to perform a global fitting consumes a large amount of time.As an alternative, one can instead use the shift parameter R from the CMB, and the distance parameter A of the measurement of the baryon acoustic oscillation (BAO) peak in the distribution of SDSS luminous red galaxies.In the literature, the shift parameter R and the distance parameter A have been used extensively.It is argued that they are model-independent [30], whereas R and A contain the main information of the observations of the CMB and the BAO, respectively.The shift parameter R is defined by [30,31] where the redshift of recombination is z * = 1090, which has been updated in WMAP5 [4].The shift parameter R relates the angular diameter distance to the last scattering surface, the comoving size of the sound horizon at z * and the angular scale of the first acoustic peak in the CMB power spectrum of the temperature fluctuations [30,31].The value of R has been updated to 1.710 ± 0.019 from WMAP5 [4].
The χ 2 from the shift parameter R is On the other hand, the distance parameter A is given by where z b = 0.35.In [32], the value of A has been determined to be 0.469 (n s /0.98) −0.35 ± 0.017.Here the scalar spectral index n s is taken to be 0.960 which has been updated from WMAP5 [4].The χ 2 from the distance parameter A is The best-fit model parameters are determined by minimizing the corresponding χ 2 .As in [25,33], the 68% confidence level (C.L.) is determined by ∆χ 2 ≡ χ 2 − χ 2 min ≤ 1.0, 2.3 and 3.53 for n p = 1, 2 and 3, respectively, where n p is the number of free model parameters.Similarly, the 95% C.L. is determined by ∆χ 2 ≡ χ 2 − χ 2 min ≤ 4.0, 6.17 and 8.02 for n p = 1, 2 and 3, respectively.These results are for the case of Constitution dataset.

III. TENSION IN THE RECENT SNIA DATASETS
At first, we consider the 397 SNIa Constitution dataset.We fit the CPL model to the observations of SNIa only, SNIa+A, SNIa+R, and SNIa+A+R, respectively.The best-fit values are presented in Table I.In Fig. 1, we also present the 68% and 95% C.L. contours in the w 0 − w a plane and the w 0 − Ω m0 plane.From Fig. 1, we find that the Constitution SNIa dataset is in tension (significantly beyond 2σ) with the observations of the CMB and the BAO.
Next, we turn to the 307 SNIa Union dataset.Similarly, we present the results in Table II and Fig. 2. From Fig. 2, we can see that the Union dataset is also in tension (beyond 2σ) with the observations of the CMB and the BAO, although the corresponding tension is weaker than the one in the case of Constitution dataset.Comparing the cases of the Constitution and Union datasets, we find that they have two common features: (i) the best-fit value of Ω m0 from SNIa data only is fairly larger than 0.3; (ii) the best-fit value of w a from SNIa data only is strongly negative, namely w a is much smaller than −1.On the other hand, they have also two differences: (i) the χ 2 min ∼ 460 is fairly larger than the corresponding degrees of freedom dof ∼ 397 for the Constitution dataset, whereas the χ 2 min ∼ 310 is approximately equal to the corresponding dof ∼ 307 for the Union dataset; (ii) for the case of Union dataset, the best-fit value of w 0 remains about −1 for all observations, whereas it is not for the case of Constitution dataset.Now, one might ask whether or not all SNIa datasets are in tension with the observations of CMB and BAO.A third recent SNIa dataset is the Davis sample [10], which consists of 192 SNIa.We present the corresponding results in Table III and Fig. 3. From Fig. 3, we see that the Davis SNIa dataset is fully consistent with the observations of CMB and BAO.There is no tension in the case of Davis dataset.For the case of 192 SNIa Davis dataset, the corresponding χ 2 min /dof ∼ 1, and its w 0 remains about −1 for all observations, similar to the case of Union dataset.However, its best-fit value of Ω m0 from SNIa data only is much closer to 0.3 than the cases of Union and Constitution datasets.So, the situation is clear now: not all SNIa datasets are in tension with the observations of CMB and BAO.Besides the Davis SNIa dataset shown above, it is well known that the SNLS SNIa dataset is also fully consistent with the observations of CMB and BAO [21,22].Therefore, the recent SNIa datasets Constitution and Union are in tension not only with the observations of CMB and BAO, but also with the other SNIa datasets such as Davis and SNLS.In fact, they are in the similar situation of the Gold04 and Gold06 SNIa datasets.

IV. OUTLIERS WITH RESPONSIBILITY FOR THE TENSION
In this section, we elucidate sources of the tension in the Constitution and Union SNIa datasets.Since the Constitution dataset comes from the Union dataset by adding 90 low redshift SNIa from the CfA3 sample, and the tension exists in both Constitution and Union datasets, we speculate that the main sources might be in the Union dataset, and the tension in the Constitution dataset is mainly inherited from the Union dataset.To verify this speculation, we divide the Constitution sample into the high and low redshift groups with the same dividing line z = 0.2 in [11,12], and repeat the procedure in the previous section.We find that the results for the high and low redshift groups have only little difference, and this supports our speculation mentioned above.So, we firstly pay our attention to the Union dataset.
In the case of Gold06 SNIa dataset, there are only five subsets (see Table I of [23]).Using the method of subset truncation (see Sec. V below), the authors of [23] found that the HZSST subset is the main source of the tension in the Gold06 dataset.Then, they isolated six SNIa in the HZSST subset which are mostly responsible for the tension.On the side of Union dataset, there are 13 subsets in it (see Table 3 and Table 11 of [11]).Even throwing away the first five subsets at low redshift, there are still eight subsets to be considered.By careful observation, we find that three subsets (Barris, Perlmutter, and Riess1998+HZT) are notable, because their RMS listed in Table 3 of [11] are significantly higher than other subsets.Then we try to subtract one of these three subsets in turn, and subtract these three subsets together from the full Union dataset, and see whether or not the tension can be removed.However, we find that it does not work in fact.This leads us to speculate that the main sources of the tension in the Union dataset might be homogeneously distributed in the whole Union dataset, namely, they do not concentrate in a single subset listed in Table 3 of [11], unlike the case of Gold06 dataset.
Here, we follow the simple method used in [23] to find the outliers responsible for the tension.In [23], the distance moduli of the six SNIa which are mostly responsible for the tension in Gold06 dataset differ by more than 1.8σ from the ΛCDM (with Ω m0 = 0.28) prediction.Similarly, we firstly fit the flat ΛCDM model to the whole 307 SNIa in the Union dataset, and find that the best-fit parameter is Ω m0 = 0.287 (the corresponding µ 0 = B/ C = 43.16).Then, we calculate the relative deviation to the best-fit ΛCDM prediction, |µ obs − µ ΛCDM |/σ obs , for all the 307 points.There are 16 SNIa which differ from the bestfit ΛCDM prediction beyond 2σ.However, we find that the tension cannot be completely removed by subtracting these 16 SNIa from the 307 SNIa Union dataset.We should adopt a severer cut.There are 21 SNIa differing from the best-fit ΛCDM prediction beyond 1.9σ, and we call them the "UnionOut" subset.The names of these 21 SNIa are listed in Table IV.In Fig. 4, we also plot their distance modulus deviations relative to the best-fit ΛCDM prediction.These 21 SNIa are indeed homogeneously distributed in the whole Union dataset.On the other hand, their minimum and maximum redshift are z min = 0.0488 (1995ac) and z max = 1.1900 (05Red), respectively.It is worth noting that if one chooses to use instead the XCDM model in which w de = const.to find the outliers, the results are almost the same of the case of ΛCDM model used here.
By subtracting the 21 SNIa UnionOut subset from the whole 307 SNIa Union dataset, we obtain the so-called "UnionT" sample ("T" stands for "truncated").Obviously, the UnionT  SNIa.Then, we repeat the same fitting as in Sec.III for this 286 SNIa UnionT sample, and present the corresponding results in Table V and Fig. 5. From Fig. 5, we clearly see that the UnionT SNIa sample is fully consistent with the other observations, and the tension has been completely removed.On the other hand, by dropping the 21 outliers, the χ 2 min significantly reduces from χ 2 min ∼ 310 for the 307 SNIa Union dataset (see Table II) to χ 2 min ∼ 204 for the 286 SNIa UnionT sample (see Table V); the corresponding χ 2 min /dof has been improved.Notice that since dropping these 21 SNIa from the Union dataset is enough, we need not adopt other severer cuts, such as 1.8σ or 1.7σ.To preserve the number of usable SNIa as much as possible, the cut 1.9σ adopted here is the best choice.Now, let us turn to the Constitution dataset.Similarly, we firstly fit the flat ΛCDM model to the whole 397 SNIa in the Constitution dataset, and find that the best-fit parameter is Ω m0 = 0.290 (the corresponding µ 0 = B/ C = 43.3158).Then, we calculate the relative deviation to the best-fit ΛCDM prediction, |µ obs − µ ΛCDM |/σ obs , for all the 397 points.We adopt the same cut 1.9σ used in the Union dataset.There are 34 SNIa differing from the best-fit ΛCDM prediction beyond 1.9σ, and we call them the "ConstitutionOut" subset.The names of these 34 SNIa are listed in Table IV.In Fig. 4, we also plot their distance modulus deviations relative to the best-fit ΛCDM prediction.These 34 SNIa are indeed homogeneously distributed in the whole Constitution dataset.On the other hand, their minimum and maximum redshift are z min = 0.015 (sn07ca) and z max = 1.190 (05Red), respectively.Obviously, the UnionOut subset and ConstitutionOut subset do heavily overlap.In fact, most SNIa of the ConstitutionOut subset come from the Union sample.Only the last five SNIa in the ConstitutionOut subset come from the CfA3 sample.This confirms our speculation in the beginning of the present section that the tension in the Constitution dataset is mainly inherited from the Union dataset.In other words, the tension is not caused by the low redshift CfA3 sample.
By subtracting the 34 SNIa ConstitutionOut subset from the whole 397 SNIa Constitution dataset, we obtain the so-called "ConstitutionT" sample.Obviously, the ConstitutionT sample consists of 363 SNIa.Then, we repeat the same fitting as in Sec.III for this 363 SNIa ConstitutionT sample, and present the corresponding results in Table VI and Fig. 6.From Fig. 6, we see that the ConstitutionT SNIa sample is consistent with the other observations at the 2σ level, and the tension has been significantly alleviated, especially comparing Fig. 6 with Fig. 1.On the other hand, by dropping the 34 outliers, the χ 2 min significantly reduces from χ 2 min ∼ 461 for the 397 SNIa Constitution dataset (see Table I) to χ 2 min ∼ 269 for the 363 SNIa ConstitutionT sample (see Table VI); the corresponding χ 2 min /dof has been significantly improved.
Notice that if one instead adopt a severer cut 1.8σ for the case of Constitution dataset, the results are close to the one of 1.9σ in fact.So, to preserve the number of usable SNIa as much as possible, the cut 1.9σ adopted here is appropriate.

V. VERIFYING THE OUTLIERS WITH THE METHOD OF RANDOM TRUNCATION
In the previous section, we have identified the outliers which are mostly responsible for the tension in the Union and Constitution datasets.Here, we will make them more robust by employing the method of random truncation used in [23].
First, we consider the Union SNIa dataset.Following [23], we compare the best-fit values of Ω m0 , w 0 and w a for the 286 SNIa UnionT data with a large number N of corresponding random truncations of the Union SNIa data (here we adopt the same number N = 500 as in [23]).The random truncations involve random subtractions of the same number of SNIa and in the same redshift range as the UnionOut subset from the full Union dataset (notice that UnionT = Union − UnionOut).We can easily obtain the mean best-fit value qr and the 1σ range σ q r of the quantity q = Ω m0 , w 0 and w a for these random truncations.According to [23], if the best-fit quantity q of the UnionT sample is within the 1σ range of the mean best-fit value qr of the random truncations, the UnionOut subset is a typical truncation representative of the Union dataset and statistically consistent with it.If on the other hand the best-fit quantity q of the UnionT sample differs from the mean best-fit value qr of the random truncations beyond 2σ, we can conclude that the UnionOut subset is not a typical truncation and is systematically different from the full q q r q− qr σ q r Ωm0 = 0.360 Ω r m0 = 0.448 ± 0.025 −3.6σThe first column is the best-fit values of the quantity q for the 286 UnionT SNIa sample, which can be read from Table V.The second column is the mean best-fit value and corresponding 1σ range of the quantity q for the N = 500 random truncations.The third column is the relative deviation.See the text for details.Union dataset [23].We present the results in Table VII and Fig. 7. Obviously, the UnionOut subset are systematically different from the full Union dataset, and its 21 SNIa are indeed the outliers.Therefore, it is reasonable to cut these 21 SNIa outliers in the UnionOut subset from the full Union dataset.
Next, we turn to the case of the Constitution dataset.Similarly, we follow the procedure described above (notice that ConstitutionT = Constitution − ConstitutionOut), and present the results in Table VIII and Fig. 8. Significantly, the ConstitutionOut subset are systematically different from the full Constitution dataset, and its 34 SNIa are indeed the outliers.Therefore, it is reasonable to cut these 34 SNIa outliers in the ConstitutionOut subset from the full Constitution dataset.

VI. CONCLUDING REMARKS
In the present work, we investigated the tension in the recent SNIa datasets Constitution and Union.We have shown that they are in tension not only with the observations of CMB and BAO, but also with other SNIa datasets such as Davis and SNLS.Then, we found the main sources which are mostly responsible for the tension.Further, we made this more robust by employing the method of random truncation.
Based on the results of this work, we suggest to perform the severer selection cut in the Union and Constitution SNIa datasets to reject the outliers further.While one uses the full 307 SNIa Union dataset and the full 397 SNIa Constitution dataset to constrain the cosmological models, we strongly recommend use also of the 286 SNIa UnionT sample and the 363 SNIa ConstitutionT sample given in this work for comparison.In fact, it is anticipated that the results might be fairly different, and the unusual features which arise from the full 307 SNIa Union dataset and the full 397 SNIa Constitution dataset might disappear in the cases of the 286 SNIa UnionT sample and the 363 SNIa ConstitutionT sample.
As communicated by [35], the era of precision cosmology has arrived (e.g.[34] and [1]) and there is a deemed need for an increased understanding of fundamental supernova (SN) physics.In addition, future SN datasets, e.g., those from DES and LSST, will rely significantly on photometric redshift determinations and SN type classifications (particularly in the case of the LSST).The details of SN colors will have to be   unraveled if purely photometric SN are to be successfully used for precision cosmology [34].Enlightened by the results of this work, one might further consider the possible impact of the ConstitutionOut and UnionOut subsets on the understanding of fundamental SN physics and SN colors [35].We leave this issue as an open question.
Research Fund of Beijing Institute of Technology, and the Project of Knowledge Innovation Program (PKIP) of Chinese Academy of Sciences under Grant No. KJCX2.YW.W10, which financially supported the KITPC Programs.
Note added: After the submission of the present paper, the 1st year SDSS-II supernova sample has been released [34].It consists of 103 SNIa with redshifts 0.04 < z < 0.42.Here, we would like to have a primary analysis.Note that the 103 SDSS SNIa dataset was given in both the frameworks of using SALT2 and MLCS2k2 light curve fitters.Similarly, we fit the CPL model to the observations of SNIa only, SNIa+A, SNIa+R, and SNIa+A+R, respectively.The best-fit values are presented in Table IX.We can see that the best-fit parameters Ω m0 , w 0 and w a for the case of SNIa only (both SALT2 and MLCS2k2) are fairly unusual.On the other hand, we find that in the case of SNIa only, the constraints on the model parameters Ω m0 , w 0 and w a are very loose.These results might be mainly due to the relatively narrow redshift range (0.04 < z < 0.42) and the relatively small number of the SDSS SNIa sample.Considering these issues, unlike the cases of the Constitution, Union and Davis datasets, the tension analysis might be not so robust for the case of the SDSS SNIa dataset.However, since the SDSS SNIa dataset fills in the redshift "desert" between low-and high-redshift SN surveys, it deserves further attentions in the SN studies.

FIG. 1 :
FIG.1:The 68% and 95% C.L. contours in the w0 − wa plane and the w0 − Ωm0 plane for the observations of SNIa only (blue dashed lines) and SNIa+A+R (black solid lines).We also show the best-fit values for the observations of SNIa only (blue box), SNIa+A (red star), SNIa+R (green triangle) and SNIa+A+R (black point).

FIG. 2 :
FIG. 2:The same as in Fig.1, except for the case of Union dataset.

FIG. 3 :
FIG.3:The same as in Fig.1, except for the case of Davis dataset.

FIG. 5 :
FIG.5:The same as in Fig.1, except for the case of the 286 SNIa UnionT sample.

FIG. 6 :
FIG.6:The same as in Fig.1, except for the case of the 363 SNIa ConstitutionT sample.

FIG. 7 :
FIG. 7: Comparing the best-fit parameters of the 286 UnionT SNIa sample (red stars) with the ones of the 500 random truncations (orange points) in the w0 − wa plane and the w0 − Ωm0 plane.The mean values of these 500 points are also indicated by the black solid points.See the text for details.

FIG. 8 :
FIG. 8: Comparing the best-fit parameters of the 363 ConstitutionT SNIa sample (red stars) with the ones of the 500 random truncations (orange points) in the w0 − wa plane and the w0 − Ωm0 plane.The mean values of these 500 points are also indicated by the black solid points.See the text for details.

TABLE I :
The χ 2 min and the best-fit values of Ωm0, w0 and wa for the various observations.These results are for the case of Constitution dataset.

TABLE II :
The same as in TableI, except for the case of Union dataset.

TABLE IV :
The names of SNIa in the UnionOut and ConstitutionOut subsets.

TABLE V :
FIG.4: ∆µ = µ obs (zi) − µΛCDM(zi) for the SNIa in the UnionOut and ConstitutionOut subsets, where the best-fit ΛCDM are for the whole Union and Constitution datasets, respectively.See the text for details.The same as in TableI, except for the case of the 286 SNIa UnionT sample.

TABLE VI :
The same as in TableI, except for the case of the 363 SNIa ConstitutionT sample.

TABLE VIII :
The first column is the best-fit values of the quantity q for the 363 ConstitutionT SNIa sample, which can be read from TableVI.The second column is the mean best-fit value and corresponding 1σ range of the quantity q for the N = 500 random truncations.The third column is the relative deviation.See the text for details.

TABLE IX :
The same as in TableI, except for the case of SDSS SNIa dataset.