Abstract
Recent work on predicting patient outcomes in the Intensive Care Unit (ICU) has focused heavily on the physiological time series data, largely ignoring sparse data such as diagnoses and medications. When they are included, they are usually concatenated in the late stages of a model, which may struggle to learn from rarer disease patterns. Instead, we propose a strategy to exploit diagnoses as relational information by connecting similar patients in a graph. To this end, we propose LSTM-GNN for patient outcome prediction tasks: a hybrid model combining Long Short-Term Memory networks (LSTMs) for extracting temporal features and Graph Neural Networks (GNNs) for extracting the patient neighbourhood information. We demonstrate that LSTM-GNNs outperform the LSTM-only baseline on length of stay prediction tasks on the eICU database. More generally, our results indicate that exploiting information from neighbouring patient cases using graph neural networks is a promising research direction, yielding tangible returns in supervised learning performance on Electronic Health Records.
Keywords
Catherine Tong and Emma Rocheteau—Equal first authorship.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
The average patient in eICU has 9 recorded diagnoses, but there are 4172 distinct diagnoses in our cohort.
- 2.
We use \(*\) to denote all models which exclude diagnoses from input \(x_S\).
- 3.
Note that the LSTM-MPNN model is the most expressive of the GNNs evaluated, as it has the capacity to model edge features (similarity scores from Eq. 1) while other GNNs do not.
References
Albert, K., Sherman, B., Backus, B.: How length of stay for congestive heart failure patients was reduced through six sigma methodology and physician leadership. Am. J. Med. Qual. 25(5), 392–397 (2010)
Choi, E., Xu, Z., Li, Y., Dusenberry, M.W., Flores, G., Xue, Y., Dai, A.M.: Graph convolutional transformer: learning the graphical structure of electronic health records (2019). arXiv:190604716
Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Measur. 20(1), 37–46 (1960)
Couturier, B., Carrat, F., Hejblum, G.: A systematic review on the effect of the organisation of hospital discharge on patient health outcomes. BMJ Open 6(12) (2016). https://doi.org/10.1136/bmjopen-2016-012287
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT (2019)
Duran, A.G., Niepert, M.: Learning graph representations with embedding propagation. In: Advances in Neural Information Processing Systems, pp. 5119–5130 (2017)
Falcon, W.: Pytorch lightning (2019). GitHub https://github.com/PyTorchLightning/pytorch-lightning
Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: ICML’17, pp. 1263–1272 (2017)
Gori, M., Monfardini, G., Scarselli, F.: A new model for learning in graph domains. IEEE Int. Joint Conf. Neural Netw. IEEE 2, 729–734 (2005)
Goyal, P., Chhetri, S.R., Canedo, A.: dyngraph2vec: Capturing network dynamics using dynamic graph representation learning. Knowledge-Based Syst. 187, 104816 (2020). https://doi.org/10.1016/j.knosys.2019.06.024
Gruenberg, D.A., Shelton, W., Rose, S.L., Rutter, A.E., Socaris, S., McGee, G.: Factors influencing length of stay in the intensive care unit. Am. J. Critical Care 15(5), 502–509 (2006). https://doi.org/10.4037/ajcc2006.15.5.502
Hamilton, W.L., Ying, R., Leskovec, J.: Inductive representation learning on large graphs. In: NIPS (2017)
Harutyunyan, H., Khachatrian, H., Kale, D.C., Ver Steeg, G., Galstyan, A.: Multitask learning and benchmarking with clinical time series data. Sci. Data 6(96) (2019)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014). arXiv:14126980
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks (2016). arXiv:160902907
Li, Y., Yu, R., Shahabi, C., Liu, Y.: Diffusion convolutional recurrent neural network: data-driven traffic forecasting (2018). arXiv:170701926
Malone, B., Garcia-Duran, A., Niepert, M.: Learning representations of missing data for predicting patient outcomes (2018). arXiv:181104752
Moon, A., Cosgrove, J., Lea, D., Fairs, A., Cressey, D.: An eight year audit before and after the introduction of modified early warning score (MEWS) charts, of patients admitted to a tertiary referral intensive care unit after CPR. Resuscitation 82(2), 150–154 (2011)
Pareja, A., Domeniconi, G., Chen, J., et al.: EvolveGCN: evolving graph convolutional networks for dynamic graphs (2019). arXiv:190210191
Pencina, M.J., Goldstein, B.A., Navar, A.M., Ioannidis, J.P.A.: Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J. Am. Med. Inform. Assoc. 24(1), 198–208 (2016)
Pollard, T.J., Johnson, A.E.W., Raffa, J.D., Celi, L.A., Mark, R.G., Badawi, O.: The eICU collaborative research database, a freely available multi-center database for critical care research. Sci. Data 5(1), 180178 (2018). https://doi.org/10.1038/sdata.2018.178
Rajkomar, A., Oren, E., Chen, K., et al.: Scalable and accurate deep learning with electronic health records. Nature 1(1), 18 (2018)
Rocheteau, E., Lió, P., Hyland, S.L.: Temporal pointwise convolutional networks for length of stay prediction in the intensive care unit (2020). arXiv:2007.09483
Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Netw. 20(1), 61–80 (2009)
Schrodt, J., Dudchenko, A., Knaup-Gregori, P., Ganzinger, M.: Graph-Representation of patient data: a systematic literature review. J. Med. Syst. 44(4), 86 (2020). https://doi.org/10.1007/s10916-020-1538-4
Sheikhalishahi, S., Balaraman, V., Osmani, V.: Benchmarking machine learning models on eICU critical care dataset (2019). arXiv:191000964
Smith, M.E.B., Chiovaro, J.C., O’Neil, M., et al.: Early warning system scores for clinical deterioration in hospitalized patients: a systematic review. Ann. Am. Thoracic Soc. 11(9), 1454–1465 (2014)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks (2017). arXiv:171010903
Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph cnn for learning on point clouds. ACM Trans. Graph. (TOG) 38(5), 1–12 (2019)
Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks? In: International Conference on Learning Representations (2019)
Yu, B., Yin, H., Zhu, Z.: Spatio-temporal graph convolutional neural network: a deep learning framework for traffic forecasting (2017). arXiv:170904875
Acknowledgements
This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) under Grant No.: DTP (EP/R513295/1), MOA (EP/S001530/), Samsung AI, the Armstrong Fund, the Frank Edward Elmore Fund, and the School of Clinical Medicine at the University of Cambridge. Additionally we thank Cristian Bodnar, Cătălina Cangea, Louis-Pascal Xhonneux, Stephanie Hyland, and anonymous reviewers at W3PHIAI-21 for their feedback.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
1.1 Dynamic LSTM-GNNs
In our paper we propose a fixed patient graph constructed using diagnoses. However, here we investigate whether a useful graph can be learnt dynamically from the time series alone (in the absence of diagnoses). Inspired by Dynamic Graph CNNs [30], we explore a dynamic variant of LSTM-GNN. Here we train an LSTM on the time series \(x_{1:T}\) with mini-batching, each time computing the pairwise Euclidean distance of the hidden vectors \(h_{T}\) in the batch. Again, we apply k-NN to obtain the graph.
Table 5 shows that LSTM* and dynamic LSTM-GNNs generally perform similarly on IHM, but the dynamic LSTM-GNNs have an advantage on the LOS task. We also observe that the dynamic LSTM-GCN* model, despite not having access to diagnoses, performs similarly to LSTM (second row of Table 3). This suggests that relating patients via a graph structure has value for modelling patient outcomes independently of diagnoses. This is possibly because where the data is poor quality or missing, the model can rely more on the neighbouring patients. However, the most visible gains (Table 3a) still come from using diagnoses for the graph construction.
1.2 Implementation and Hyperparameter Search Methodology
For each model, we conducted 10 random hyperparameter trials. The hyperparameter search ranges and selected values can be found in our repository: https://github.com/EmmaRocheteau/eICU-GNN-LSTM.
All deep learning methods were implemented in PyTorch and optimised using Adam [15]. We used PyTorch Lightning [7] and Tune to structure our experiments and easily compare different hyperparameter choices. The maximum number of epochs was 25, although many models finished before this due to early stopping.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Tong, C., Rocheteau, E., Veličković, P., Lane, N., Liò, P. (2022). Predicting Patient Outcomes with Graph Representation Learning. In: Shaban-Nejad, A., Michalowski, M., Bianco, S. (eds) AI for Disease Surveillance and Pandemic Intelligence. W3PHAI 2021. Studies in Computational Intelligence, vol 1013. Springer, Cham. https://doi.org/10.1007/978-3-030-93080-6_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-93080-6_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-93079-0
Online ISBN: 978-3-030-93080-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)