Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Clinical Research Article
  • Published:

Development and validation of thromboembolism diagnostic algorithms in children with cancer from real-world data

Pediatric Research (2024)Cite this article

Abstract

Objective

To evaluate the accuracy of diagnostic algorithms developed using the International Classification of Diseases (ICD-9-CM and ICD-10-CA) diagnostic codes and physician billing codes for thromboembolism (TE) from health administrative data compared to chart review diagnoses of TE in children with cancer.

Methods

Using data linkage between the Pediatric Oncology Group of Ontario Network Information System (Ontario pediatric cancer registry) and various administrative data housed at ICES, eight algorithms were developed including a single reference to one of the billing codes, multiple references with varying time intervals, and combinations of various billing codes during primary cancer therapy for the whole cohort and, for early (<04/2002) and later (≥04/2002, solely ICD-10 codes) periods. Reference standard was chart review data from prior studies (from 1990 to 2016) among children (≤19 years) with cancer and radiologically confirmed TE.

Results

Records of 2056 patients diagnosed with cancer at two participating sites during study period were reviewed; 112 had radiologically confirmed TE. The algorithm with addition of anticoagulation utilization codes was the best performing algorithm (sensitivity = 0.76;specificity = 0.85). With use of ICD-10 only codes, sensitivity of the same algorithm improved to 0.84 with specificity of 0.80.

Conclusion

This study provides a valid approach for ascertaining pediatric TE using real-world data.

Impact

  • Research in pediatric thrombosis, especially cancer-related thrombosis, is limited mainly due to small-sized studies.

  • Real-world data provide ready access to large and diverse populations. However, there are no validated algorithms for identifying thrombosis in real-world data for children.

  • An algorithm based on combination of thrombosis and anticoagulation utilization codes had 76% sensitivity and 85% specificity to identify diagnosis of thrombosis in children in administrative data.

  • This study provides a valid approach for ascertaining pediatric thrombosis using real-world data and offers a good avenue to advance pediatric thrombosis research.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Thromboembolism (TE) diagnostic ICD-9 and ICD-10 codes were used to identify cases of TE in administrative databases.

Similar content being viewed by others

Data availability

The dataset from this study is held securely in coded form at ICES. While legal data sharing agreements between ICES and data providers (e.g., healthcare organizations and government) prohibit ICES from making the dataset publicly available, access may be granted to those who meet pre-specified criteria for confidential access, available at www.ices.on.ca/DAS (email: das@ices.on.ca). The full dataset creation plan and underlying analytic code are available from the authors upon request, understanding that the computer programs may rely upon coding templates or macros that are unique to ICES and are therefore either inaccessible or may require modification.

References

  1. Raffini, L., Huang, Y.-S., Witmer, C. & Feudtner, C. Dramatic increase in venous thromboembolism in U.S. Children’s Hospitals from 2001–2007. Pediatrics 124, 1001–1008 (2009).

    Article  PubMed  Google Scholar 

  2. O’Brien, S. H., Klima, J., Termuhlen, A. M. & Kelleher, K. J. Venous thromboembolism and adolescent and young adult oncology inpatients in US Children’s Hospitals, 2001 to 2008. J. Pediatr. 159, 133–137 (2011).

    Article  PubMed  Google Scholar 

  3. Pelland-Marcotte, M.-C. et al. Thromboembolism incidence and risk factors in children with cancer: a population-based cohort study. Thromb. Haemost. 118, 1646–1655 (2018).

    Article  PubMed  Google Scholar 

  4. Athale, U. Thrombosis in pediatric cancer: identifying the risk factors to improve care. Expert Rev. Hematol. 6, 599–609 (2013).

    Article  CAS  PubMed  Google Scholar 

  5. Ranta, S. et al. Cerebro-sinovenous thrombosis in children with acute lymphoblastic leukemia- a multicenter study from nordic pediatric hematology oncology. Br. J. Haematol. 168, 547–552 (2015).

    Article  PubMed  Google Scholar 

  6. Tuckuviene, R. et al. Prospective study of thromboembolism in 1038 children with acute lymphoblastic leukemia: a Nordic Society of Pediatric Hematology and Oncology (NOPHO) study. J. Thromb. Haemost. 14, 485–494 (2016).

    Article  CAS  PubMed  Google Scholar 

  7. Caruso, V. et al. Thrombotic complications in childhood acute lymphoblastic leukemia: a meta-analysis of 17 prospective studies comprising 1752 patients. Blood 108, 2216–2222 (2006).

    Article  CAS  PubMed  Google Scholar 

  8. Mitchell, L. G. et al. A prospective cohort study determining the prevalence of thrombotic events in children with acute lymphoblastic leukemia and a central venous line who are treated with L-asparaginase. Results of the Prophylactic Antithrombin Replacement in Kids with Acute Lymphoblastic Leukemia Treated with Asparaginase (PARKAA) study. Cancer 97, 508–516 (2003).

    Article  CAS  PubMed  Google Scholar 

  9. Paz-Priel, I., Long, L., Helman, L. J., Mackall, C. L. & Wayne, A. S. Thromboembolic events in children and young adults with pediatric sarcoma. J. Clin. Oncol. 25, 1519–1524 (2007).

    Article  PubMed  Google Scholar 

  10. Grace, R. F. et al. The frequency and management of asparaginase-related thrombosis in paediatric and adult patients with acute lymphoblastic leukaemia treated on Dana-Farber Cancer Institute consortium protocols. Br. J. Haematol. 152, 452–459 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  11. Athale, U. et al. Predictors of thrombosis in children receiving therapy for acute lymphoblastic leukemia: results from Dana-Farber Cancer Institute ALL Consortium trial 05-001. Pediatr. Blood Cancer 69, e29581 (2022).

    Article  CAS  PubMed  Google Scholar 

  12. Setty, B. A., O’Brien, S. H. & Kerlin, B. A. Pediatric venous thromboembolism in the United States: a tertiary care complication of chronic diseases. Pediatr. Blood Cancer 59, 258–264 (2012).

    Article  PubMed  Google Scholar 

  13. Forbrigger, Z. et al. The association of venous thromboembolism with survival in pediatric cancer patients: a population-based cohort study. Ann. Hematol. 97, 1903–1908 (2018).

    Article  PubMed  Google Scholar 

  14. Kulkarni, K. et al. Increased requirement for central venous catheter replacement in paediatric oncology patients with deep venous thrombosis: a multicentre study. Thromb. Haemost. 113, 434–435 (2015).

    Article  PubMed  Google Scholar 

  15. Journeycake, J. M. & Buchanan, G. R. Catheter-related deep venous thrombosis and other catheter complications in children with cancer. J. Clin. Oncol. 28, 4575–4580 (2006).

    Article  Google Scholar 

  16. Pelland-Marcotte, M. C. et al. Thrombosis is associated with worse survival in children with acute lymphoblastic leukemia: a report from CYP-C. Am. J. Hematol. 96, 796–804 (2021).

    Article  PubMed  Google Scholar 

  17. Doiron, D., Raina, P. & Fortier, I. Linking Canadian population health data: maximizing the potential of cohort and administrative data. Can. J. Public Health 104, e258–e261 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  18. Ulrich, E. H., So, G., Zappitelli, M. & Chanchlani, R. A review on the application and limitations of administrative health care data for the study of acute kidney injury epidemiology and outcomes in children. Front. Pediatr. 9, 742888 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  19. Burles, K., Innes, G., Senior, K., Lang, E. & McRae, A. Limitations of pulmonary embolism ICD-10 codes in emergency department administrative data: let the buyer beware. BMC Med. Res. Methodol. 17, 89 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Alotaibi, G. S., Wu, C., Senthilselvan, A. & McMurtry, M. S. The validity of ICD codes coupled with imaging procedure codes for identifying acute venous thromboembolism using administrative data. Vasc. Med. 20, 364–368 (2015).

    Article  PubMed  Google Scholar 

  21. Tamariz, L., Harkins, T. & Nair, V. A systematic review of validated methods for identifying venous thromboembolism using administrative and claims data. Pharmacoepidemiol. Drug Saf. 21, 154–162 (2021).

    Article  Google Scholar 

  22. Branchford, B. R., Gibson, E., Manco-Johnson, M. J. & Goldenberg, N. A. Sensitivity of discharge diagnosis ICD-9 codes for pediatric venous thromboembolism is greater than specificity, but still suboptimal for surveillance and clinical research. Thromb. Res. 129, 662–663 (2012).

    Article  CAS  PubMed  Google Scholar 

  23. https://www.pogo.ca/wp-content/uploads/2015/02/POGO_CC-Atlas-2-POGONIS_Feb-2015.pdf, accessed 7 April 2023.

  24. https://www.ices.on.ca/Data-and-Privacy/ICES-data, accessed 7 April 2023.

  25. Greenberg, M. L. et al. Childhood cancer registries in Ontario, Canada: lessons learned from a comparison of two registries. Int. J. Cancer 105, 88–91 (2003).

    Article  CAS  PubMed  Google Scholar 

  26. Pole, J. D. et al. Subsequent malignant neoplasms in pediatric cancer patients treated with and without hematopoietic SCT. Bone Marrow Transpl. 50, 721–726 (2015).

    Article  CAS  Google Scholar 

  27. Pole, J. D., Gu, L. Y., Kirsh, V., Greenberg, M. L. & Nathan, P. C. Subsequent malignant neoplasms in a population-based cohort of pediatric cancer patients: a focus on the first 5 years. Cancer Epidemiol. Biomark. Prev. 24, 1585–1592 (2015).

    Article  Google Scholar 

  28. Gupta, S. et al. Validity of administrative data in identifying cancer-related events in adolescents and young adults. Med. Care 56, e32–e38 (2018).

    Article  PubMed  Google Scholar 

  29. Athale, U. H. et al. Epidemiology and clinical risk factors predisposing to thromboembolism in children with cancer. Pediatr. Blood Cancer 51, 792–797 (2008).

    Article  PubMed  Google Scholar 

  30. Spavor, M. et al. Age at cancer diagnosis, non-O blood group and asparaginase therapy are independently associated with deep venous thrombosis in pediatric oncology patients: a risk model. Thromb. Res. 144, 27–31 (2016).

    Article  CAS  PubMed  Google Scholar 

  31. Halton, J. et al. Do children with central venous line dysfunction have increased risk of symptomatic thromboembolism compared to those without CVL-dysfunction, while on cancer therapy? Study protocol. BMC Cancer 12, 314 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Steliarova-Foucher, E., Stiller, C., Lacour, B. & Kaatsch, P. International classification of childhood cancer, third edition. Cancer 103, 1457–1467 (2005).

    Article  PubMed  Google Scholar 

  33. Walker, A. J. et al. Venous thromboembolism in children with cancer- a population-based cohort study. Thromb. Res. 133, 340–344 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Delate, T. et al. Outpatient use of low molecular weight heparin monotherapy for first-line treatment of venous thromboembolism in advanced cancer. Oncologist 17, 419–427 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Molnar, A. O. et al. Risk and complications of venous thromboembolism in dialysis patients. Nephrol. Dial. Transpl. 33, 874–880 (2018).

    Google Scholar 

  36. Bujang, M. A. & Adnan, T. H. Requirements for minimum sample size for sensitivity and specificity analysis. J. Clin. Diagn. Res. 10, YE01–YE06 (2016).

    PubMed  PubMed Central  Google Scholar 

  37. Fang, M. C. et al. Validity of using inpatient and outpatient administrative codes to identify acute venous thromboembolism: the CVRN VTE study. Med. Care 55, e137–e143 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  38. Quan, H. et al. IMECCHI investigators. Assessing validity of ICD-9-CM and ICD-10 administrative data in recording clinical conditions in a unique dually coded database. Health Serv. Res. 43, 1424–1441 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  39. Monaghan, T. F. et al. Foundational statistical principles in medical research: sensitivity, specificity, positive predictive value, and negative predictive value. Medicine 57, 503 (2021).

    Google Scholar 

  40. Lucyk, K. et al. Administrative health data in Canada: lessons from history. BMC Med. Inf. Decis. Mak. 15, 69–74 (2015).

    Article  Google Scholar 

Download references

Acknowledgements

This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health (MOH) and the Ministry of Long-Term Care (MLTC). This research was facilitated by the Pediatric Oncology Group of Ontario’s Networked Information System, financially supported by Ontario’s Ministry of Health and Long-Term Care. This study also received funding from Hamilton Health Sciences. We thank IQVIA Solutions Canada Inc. for use of their Drug Information File. This research was facilitated by the Pediatric Oncology Group of Ontario’s Networked Information System, financially supported by Ontario’s Ministry of Health and Long-Term Care. Parts of this material are based on data and/or information compiled and provided by CIHI and the Ontario Ministry of Health. The analyses, conclusions, opinions, and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred.

Funding

This study was supported in part by Hamilton Health Sciences Foundation grant. The funder/sponsor did not participate in the work.

Author information

Authors and Affiliations

  1. Division of Hematology/Oncology, McMaster Children’s Hospital, Hamilton Health Sciences, Hamilton, Ontario, Canada

    Uma Athale & Anthony K. C. Chan

  2. Pediatrics, McMaster University, Hamilton, Ontario, Canada

    Uma Athale & Anthony K. C. Chan

  3. Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada

    Uma Athale

  4. Division of Hematology/Oncology, Children’s Hospital of Eastern Ontario, Ottawa, Ontario, Canada

    Jacqueline Halton

  5. Pediatrics, University of Ottawa, Ottawa, Ontario, Canada

    Jacqueline Halton

  6. ICES, McMaster, University, Hamilton, Ontario, Canada

    Anastasia Gayowsky

  7. Centre for Health Services Research, The University of Queensland, Queensland, Australia

    Jason D. Pole

  8. ICES, Toronto, Ontario, Canada

    Jason D. Pole

Authors
  1. Uma Athale
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacqueline Halton
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anastasia Gayowsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anthony K. C. Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jason D. Pole
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Dr Uma Athale conceptualized and designed the study, collected data, drafted the initial manuscript, and critically reviewed and revised the manuscript. Dr Jacqueline Halton collected data, and critically reviewed and revised the manuscript. Anastasia Gayowsky carried out the analyses, and critically reviewed and revised the manuscript. Dr. Anthony Chan conceptualized the study, and critically reviewed and revised the manuscript. Dr Jason Pole conceptualized and designed the study, guided the analyses, and critically reviewed and revised the manuscript. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

Corresponding author

Correspondence to Uma Athale.

Ethics declarations

Competing interest

The authors declare no competing interests.

Consent statement

ICES is an independent, non-profit research institute whose legal status under Ontario’s health information privacy law allows it to collect and analyze health care and demographic data, with a consent waiver for health system evaluation and improvement. The study was approved by each of the institution’s research and ethics board.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Athale, U., Halton, J., Gayowsky, A. et al. Development and validation of thromboembolism diagnostic algorithms in children with cancer from real-world data. Pediatr Res (2024). https://doi.org/10.1038/s41390-024-03082-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41390-024-03082-x

Search

Advanced search

Quick links