ABSTRACT
The aim of the Social Media Mining for Health Applications (#SMM4H) shared tasks is to take a community-driven approach to address the natural language processing and machine learning challenges inherent to utilizing social media data for health informatics. The eighth iteration of the #SMM4H shared tasks was hosted at the AMIA 2023 Annual Symposium and consisted of five tasks that represented various social media platforms (Twitter and Reddit), languages (English and Spanish), methods (binary classification, multi-class classification, extraction, and normalization), and topics (COVID-19, therapies, social anxiety disorder, and adverse drug events). In total, 29 teams registered, representing 18 countries. In this paper, we present the annotated corpora, a technical summary of the systems, and the performance results. In general, the top-performing systems used deep neural network architectures based on pre-trained transformer models. In particular, the top-performing systems for the classification tasks were based on single models that were pre-trained on social media corpora. To facilitate future work, the datasets—a total of 61,353 posts—will remain available by request, and the CodaLab sites will remain active for a post-evaluation phase.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
AZK, JIFA, DX, and GGH were supported in part by the National Library of Medicine (R01LM011176). YG and AS were supported in part by the National Institute on Drug Abuse (R01DA057599). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. JMB was supported in part by a Google Award for Inclusion Research (AIR).
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
This study used publicly available Twitter data. The institutional review boards of the University of Pennsylvania and Cedars-Sinai Medical Center reviewed this study and deemed it exempt human subjects research.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
According to the Twitter Terms of Service, the content (e.g., text) of Tweet Objects cannot be made publicly available; however, a limited number of Tweet Objects are permitted to be shared directly. Requests for data can be sent to Ari Z. Klein (ariklein{at}pennmedicine.upenn.edu) or Graciela Gonzalez-Hernandez (Graciela.GonzalezHernandez{at}csmc.edu).