Crowdsourced assessment of common genetic contribution to predicting anti-TNF treatment response in rheumatoid arthritis

Rheumatoid arthritis (RA) affects millions world-wide. While anti-TNF treatment is widely used to reduce disease progression, treatment fails in ∼one-third of patients. No biomarker currently exists that identifies non-responders before treatment. A rigorous community-based assessment of the utility of SNP data for predicting anti-TNF treatment efficacy in RA patients was performed in the context of a DREAM Challenge (http://www.synapse.org/RA_Challenge). An open challenge framework enabled the comparative evaluation of predictions developed by 73 research groups using the most comprehensive available data and covering a wide range of state-of-the-art modelling methodologies. Despite a significant genetic heritability estimate of treatment non-response trait (h2=0.18, P value=0.02), no significant genetic contribution to prediction accuracy is observed. Results formally confirm the expectations of the rheumatology community that SNP information does not significantly improve predictive performance relative to standard clinical traits, thereby justifying a refocusing of future efforts on collection of other data.


Supplementary Figures
for the discovery cohort at significance thresholds 5e-8 (genomewide significance), 5e-6 (correction for testing 10,000 independent loci), 5e-5 (correction for testing 1,000 independent loci) and 5e-4 (correction for testing 100 independent loci), and including expected AUROC corresponding to the percent heritability explained (y-axis, right), and (B) for various sample sizes assuming a genomewide significance cutoff of 5e-8 for inclusion in the model. Expected percent heritability explained is computed assuming population prevalence of 0.217 and heritability (h 2 ) of 0.18 as estimated from the discovery cohort. Collaborative phase leaderboard score versus final submission score for AUPR, AUROC for the classification subchallenge, and correlation for the quantitative subchallenge with linear regression fit and 95% confidence region (shaded). While the two metrics for the classification subchallenge showed positive correlation (r= 0.71 and 0.60 for AUPR and AUROC, respectively) between the scores on the leaderboard data, which is a held-out portion of the training dataset, and the scores on the test data, the quantitative prediction subchallenge showed a negative correlation (r= -0.052) suggesting a tendency toward overfitting in that subchallenge.

Supplementary Tables
Supplementary

Team SBI_Lab
The aim of team SBI_Lab's study was to identify candidate SNPs playing a role in the response to therapy in RA patients by compiling several sources of information. To account for the imbalance between the number of SNPs and the number of patients, they performed a feature selection procedure on the genetic data. First, SNPs were mapped to genes using the Ensembl Variation database 2 and BIOMART service 3

Team Lucia
Team Lucia selected SNPs in two independent steps, based on prior biological knowledge and on statistical criteria. 242 genes were selected from literature that were known to play a role in the development of RA, or to be targets for the drugs used in the considered treatments 9,10 . All SNPs within 10kb upstream and 1 kb downstream of the selected genes, as well as those located in their distant enhancer sequences 11,12 , were included. Among these SNPs, those located in introns were discarded. This led to a list of 3,840 SNPs selected on available biological data. SNPs were also ranked by mutual information and the top 3000 were first projected to make these predictors linearly independent of the clinical variables.
In BEMKL prediction, Kronecker delta kernels were used for the categorical features and Gaussian kernels for the other predictors.

Team wtwt5237
According to existing literature, 18 MTX response related SNPs, 124 anti-TNF drug response related SNPs, 78 RA-related SNPs, and 20,385 immune-related SNPs were selected.
Principal components (PCs) of RA-related SNPs were calculated by PCA. PCs of all SNPs and immune-related SNPs were calculated by PLINK 21 . For both sub challenges, 5 different algorithms were trained, which include randomForest (alg1), LASSO 22 (alg2), SVM (alg3), Adaboost (alg4) and PLS (alg5). For the quantitative subchallenge, we predicted final DAS28 levels for alg1, alg3 and alg5, and calculated ΔDAS28 by subtracting baseline DAS28 from the predicted final DAS28 levels. In each of the 5 algorithms, we included baseline DAS28, age, gender and MTX usage. Additionally, anti-TNF treatment is used as a covariate for alg1, alg3, and alg4, but for alg2 and alg5, patients were stratified according to the anti-TNF drugs used and made drug-specific training and prediction. Two novel drugs, certolizumab and golimumab, were modeled as adalimumab and infliximab, respectively, according to similarities of these compounds. In addition, team wtwt5237 included different combinations of SNP sets or the first 3-70 PCs of immune-related, RA-related or all SNPs in the machine learning algorithms. Finally, predictions from the 5 algorithms were combined using different weights.