An individualized predictor of health and disease using paired reference and target samples

Background Consider the problem of designing a panel of complex biomarkers to predict a patient’s health or disease state when one can pair his or her current test sample, called a target sample, with the patient’s previously acquired healthy sample, called a reference sample. As contrasted to a population averaged reference this reference sample is individualized. Automated predictor algorithms that compare and contrast the paired samples to each other could result in a new generation of test panels that compare to a person’s healthy reference to enhance predictive accuracy. This paper develops such an individualized predictor and illustrates the added value of including the healthy reference for design of predictive gene expression panels. Results The objective is to predict each subject’s state of infection, e.g., neither exposed nor infected, exposed but not infected, pre-acute phase of infection, acute phase of infection, post-acute phase of infection. Using gene microarray data collected in a large scale serially sampled respiratory virus challenge study we quantify the diagnostic advantage of pairing a person’s baseline reference with his or her target sample. The full study consists of 2886 microarray chips assaying 12,023 genes of 151 human volunteer subjects under 4 different inoculation regimes (HRV, RSV, H1N1, H3N2). We train (with cross-validation) reference-aided sparse multi-class classifier algorithms on this data to show that inclusion of a subject’s reference sample can improve prediction accuracy by as much as 14 %, for the H3N2 cohort, and by at least 6 %, for the H1N1 cohort. Remarkably, these gains in accuracy are achieved by using smaller panels of genes, e.g., 39 % fewer for H3N2 and 31 % fewer for H1N1. The biomarkers selected by the predictors fall into two categories: 1) contrasting genes that tend to differentially express between target and reference samples over the population; 2) reinforcement genes that remain constant over the two samples, which function as housekeeping normalization genes. Many of these genes are common to all 4 viruses and their roles in the predictor elucidate the function that they play in differentiating the different states of host immune response. Conclusions If one uses a suitable mathematical prediction algorithm, inclusion of a healthy reference in biomarker diagnostic testing can potentially improve accuracy of disease prediction with fewer biomarkers. Electronic supplementary material The online version of this article (doi:10.1186/s12859-016-0889-9) contains supplementary material, which is available to authorized users.

(b) Subtract baseline maximum symptom score (t = 0 or pre-inoculation daily max for each symptom) from all other time points, symptom by symptom (e.g., baseline Sneezing is 2, then subtract 2 from all other non-zero Sneezing symptoms from all other time points. If more than 1 pre-inoculation baseline time points available, then adjust for the max pre-inoculation symptom score for each symptom. (c) If baseline adjusted and unadjusted symptom labels differ, flag for clinical review.
7. Symptom onset is first day of 2 or more consecutive with DailyM axSum of ≥ 2.
* For the purpose of calculating daily symptoms, "calendar days" (e.g midnight to midnight on Wed, 9/3/2014) are used rather than 24 hr periods post inoculation. For calculation of symptom onset, symptom resolution, etc, time relative to inoculation (e.g. +12hrs) is used. (a) Infected if there existed greater or equal to 2 positive titer measurements that were larger than 1.25, observed at more than 24 hr post inoculation;

Infection/Shedding Status
(b) Infected if there existed more than 1 strong positive titer measurement that was larger than 3.0, observed at more than 24 hr post inoculation; (c) 2 measurable titers need not be on same or consecutive days; (d) Do not include Day 0 measures (0-24hrs post inoculation) since inoculum may be detected; do not include Day 28 measures where available.
3. For virus PCR data, the same thresholds as virus quantitative culture (see 2 above): (a) Infected if there existed more than 2 measurements that were larger than 1.25, observed at more than 24 hr post inoculation; (b) Infected if there existed more than 1 strong positive measurement that was greater or equal to 3.0, observed at more than 24 hr post inoculation; (c) PCR data should be calculated based upon standard curves, and expressed in EID50/ml or pfu/ml or pfu-e/ml; (d) 2 measurable titers need not be on same or consecutive days; (e) Do not include measures in first 24hrs post inoculation (0-24hrs) since inoculum may be detected; do not include Day 28 measures where available.
4. If both viral culture and PCR data are available, positive by one method is considered positive.

Five time-specific infection states
Consider the subjects whose titer scores and symptom scores agree, i.e., those who are either infected and symptomatic or uninfected and asymptomatic. We set the infection onset time and offset time for infected subjects as the time point of the first and last occurrence of measurable positive titer > 1.25 for any virus assays defined in section 2.2 respectively.

Prediction of ambiguous subject's state of infection and symptom
In the main text we have we excluded clinically ambiguous subjects due to inconsistencies between their declared symptomatic status and measured shedding status. Here we apply the predictors trained using the unambiguously healthy and unambiguously ill subjects to predict the infected/uninfected states of the ambiguous subjects. These are different predictors trained for each viral challenge. Not surprisingly, the states of the clinically ambiguous subjects are difficult to predict even when using the reference aided classifer. Table 1 shows that, as compared to the standard classifier, the reference aided classifier attains a lower error rate than the standard classifier for H1N1 and HRV but not for the other viral species. However, the reference-aided classifier does achieve a reduction in the average classification error. When averaged over all the different viral species (rows of Table 1), the mean prediction accuracies on the uninfected but symptomatic subjects are 0.57 by the standard predictors and 0.49 by the reference-aided predictors. The corresponding mean accuracies on the infected but asymptomatic subjects are 0.66 by the standard predictors and 0.57 by the reference-aided predictors. Next, we apply the predictors trained using the unambiguously healthy and unambiguously ill subjects to predict the symptomatic/asymptomatic states of the ambiguous subjects. Table 2 shows that, in opposition to Table 1, the Sx/Asx reference aided predictor reduces the error for H3N2 and RSV but not for H1N1 and HRV. This dichotomy might be partially explained by the fact that symptoms were milder in the H1N1 and HRV cohorts than in the H3N2 and RSV cohorts. Therefore, a larger number of H1N1 and HRV subjects who were clearly infected may not have accurately reported their symptoms.
Unlike for infected state prediction, shown in Table 1, the referenced based symptom predictor does not reduce the average error when averaged over all viral challenge cohorts. The overall prediction error on all ambiguous subjects is 0.43 using the standard predictors, and 0.49 using the reference-aided predictors. The accuracies on the uninfected but symptomatic subjects are 0.53 by the standard predictors and 0.58 by the reference-aided predictors. The accuracies on the infected but asymptomatic subjects are 0.34 by the standard predictors and 0.43 by the reference-aided predictors. Table 2: Average accuracy (error rate) for prediction of symptomatic vs asymptomatic state for different viral challenges (data from DEE2/DEE5, DEE3/DEE4 and HRV-UVA/HRV-Duke were pooled and designated as H3N2, H1N1, and HRV in table). Shown are the standard predictor (w/o baseline reference), the reference-aided predictor (w/ baseline reference) trained using the unambiguously healthy and unambiguously ill subjects and applied to the ambiguous subjects to classify the state of symptoms, i.e., the predictors classify the ambiguous subjects as either symptomatic or asymptomatic subjects. The ground truth symptom states of the subjects were determined from self-reported symptoms as described in Sec 2.