A Comprehensive COVID-19 Daily News and Medical Literature Briefing to Inform Health Care and Policy in New Mexico: Implementation Study

Background On March 11, 2020, the New Mexico Governor declared a public health emergency in response to the COVID-19 pandemic. The New Mexico medical advisory team contacted University of New Mexico (UNM) faculty to form a team to consolidate growing information on severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and its disease to facilitate New Mexico’s pandemic management. Thus, faculty, physicians, staff, graduate students, and medical students created the “UNM Global Health COVID-19 Intelligence Briefing.” Objective In this paper, we sought to (1) share how to create an informative briefing to guide public policy and medical practice and manage information overload with rapidly evolving scientific evidence; (2) determine the qualitative usefulness of the briefing to its readers; and (3) determine the qualitative effect this project has had on virtual medical education. Methods Microsoft Teams was used for manual and automated capture of COVID-19 articles and composition of briefings. Multilevel triaging saved impactful articles to be reviewed, and priority was placed on randomized controlled studies, meta-analyses, systematic reviews, practice guidelines, and information on health care and policy response to COVID-19. The finalized briefing was disseminated by email, a listserv, and posted on the UNM digital repository. A survey was sent to readers to determine briefing usefulness and whether it led to policy or medical practice changes. Medical students, unable to partake in direct patient care, proposed to the School of Medicine that involvement in the briefing should count as course credit, which was approved. The maintenance of medical student involvement in the briefings as well as this publication was led by medical students. Results An average of 456 articles were assessed daily. The briefings reached approximately 1000 people by email and listserv directly, with an unknown amount of forwarding. Digital repository tracking showed 5047 downloads across 116 countries as of July 5, 2020. The survey found 108 (95%) of 114 participants gained relevant knowledge, 90 (79%) believed it decreased misinformation, 27 (24%) used the briefing as their primary source of information, and 90 (79%) forwarded it to colleagues. Specific and impactful public policy decisions were informed based on the briefing. Medical students reported that the project allowed them to improve on their scientific literature assessment, stay current on the pandemic, and serve their community. Conclusions The COVID-19 briefings succeeded in informing and guiding New Mexico policy and clinical practice. The project received positive feedback from the community and was shown to decrease information burden and misinformation. The virtual platforms allowed for the continuation of medical education. Variability in subject matter expertise was addressed with training, standardized article selection criteria, and collaborative editing led by faculty.


Introduction
On March 11, 2020, New Mexico Governor, Michelle Lujan Grisham, and the New Mexico Department of Health declared a public health emergency in response to the COVID-19 pandemic, after announcing 3 New Mexico residents tested presumptive positive for COVID-19 [1]. New Mexico Department of Health responded by creating the Medical Advisory Team, which brought together state officials, health care providers, and community members to compile and disseminate scientific findings, and create guidelines and recommendations to navigate the challenges of the pandemic. With the growing number of scientific publications and news reports and the potential for misinformation dissemination into the community, the Medical Advisory Team reached out to faculty at the University of New Mexico (UNM) to form a team to analyze and distribute reliable information to inform health care and public policy decisions for the state.
There was a high volume of both vital and inaccurate information available regarding COVID-19 [2,3]. To reduce information overload and misinformation, quality content had to be filtered and consolidated for state and health care leaders [4][5][6]. Researchers recommended the use of official public health organization websites as the most reliable source of information on COVID-19 preventative measures [7].
Prior to the COVID-19 pandemic, there was little guidance on methodology to generate comprehensive daily briefings. We aimed to provide guidance on how to create a daily briefing to address the high volume of information and misinformation. We surveyed the readers to determine if the briefing influenced their professional practice, if it was a main source of information, and if it helped combat misinformation. Readers also had the opportunity to share their thoughts on the briefings in free text.
During the first few months of the global pandemic, medical schools and various medical education governing bodies agreed that clinical medical education needed to be suspended due to high infectivity risk, limited COVID-19 testing supplies, and limited personal protective equipment (PPE). Studies have since assessed how psychologically and educationally detrimental it can be to study medicine in isolation. Participants in 1 study found an increase in depression, detachment from family and friends, and hopelessness, with a decrease in work performance and study time [8]. While virtual platforms are less ideal than in-person learning, learners found team-based projects to be more engaging [9,10]. Medical students were given the opportunity to take part in this project with the goal of increasing engagement during a difficult time of learning. We aimed to qualitatively determine if participation in the briefings showed net benefit to virtual medical education.
With a robust team of professionals, students, and volunteers, a comprehensive daily briefing was first disseminated on April 5, 2020, as the "UNM Global Health COVID-19 Intelligence Briefing." Here, we describe the process of creating such briefings, the usefulness of these briefings to the community and its leaders, as well as their benefit to virtual medical education.

Briefing Process
A team of medical doctors, PhDs, journalists, graduate students, medical students, and researchers from both the United States and Kenya volunteered to participate in the composition of the daily briefings. Microsoft Teams served as the platform for automatic and manual article collection and triaging, composition of the briefing, as well as the administration of a qualitative survey. Microsoft Teams Flows were developed to automatically gather COVID-19-related papers and reports from various sources including Twitter (eg, BBC, JAMA, New Mexico Department of Health, New Mexico Office of the Governor, President of the United States), Google, LitCovid (National Institutes of Health), World Health Organization, and the Centers for Disease Control and Prevention while excluding duplicate URLs (Figure 1; Multimedia Appendix 1). Articles were categorized into "New Mexico Mainstream Media," "Manual Requests," "US Mainstream Media," "International Media," "Health Organization," "Science and Medicine," and "Literature" and were assigned a default "medium" priority tag through Microsoft Planner. Articles manually submitted by readers through a submission link from each briefing generated a Microsoft Teams Planner task and were reviewed and triaged accordingly. Triaging was completed by 3 administrators to ensure quality and consistency. Articles were manually triaged by administrators into "urgent," "important," or "low" priority groups. Scientific reports were triaged based on the veracity and potential impact on COVID-19-related health care or public policy response. Information on epidemiology, testing, public guidelines, medical practice guidelines, new therapies, vaccines, and pathogenesis were of primary focus. Evidentiary priority was given to systematic reviews of randomized controlled trials with meta-analyses and individual randomized controlled trials, followed by quasiexperimental, case control, or cohort studies. These were often labeled "urgent." Studies with small sample sizes or problematic study designs were included or excluded at the triaging administrator's judgment. Opinions pieces were largely excluded. News articles covering New Mexico, United States, and international pandemic responses and impacts were included based on relevance and content validity.
Articles labeled "important" were analyzed and summarized by a team member to include the type of study, pertinent results, other relevant information, the Digital Object Identifier, or hyperlink, and marked as "complete." Headlines were carefully constructed to be as informative as possible rather than as "teasers." Completed articles were autopopulated first into a Microsoft Excel spreadsheet, then relocated to a Microsoft Word document. Items typically triaged as "low" priority were commentaries, editorials, political messages, and non-evidence-based studies, and excluded from the briefing. The briefing was divided into the following sections: "Executive Summary;" "New Mexico Highlights;" "US Highlights;" "Economics, Workforce, Supply Chain, PPE Highlights;" "Epidemiology Highlights;" "Healthcare Policy Recommendations;" "Practice Guidelines;" "Testing;" "Drugs, Vaccines, Therapeutics, Clinical Trials;" and "Other Science." Volunteers collaboratively edited the document with videoconference coordination each evening to ensure quality, clarity, accuracy, grammar, and proper citation. Reports with insufficient sample size, unclear conclusions, or poorly executed study designs were excluded. The final document included an "Executive Summary" with the general content, a disclaimer noting the inclusion of non-peer-reviewed content, and a list of all the participants of the day. The finalized briefing was emailed to the New Mexico Medical Advisory Team, sent to a listserv of subscribers, and posted on the publicly accessible UNM digital repository [11]. The UNM Department of Internal Medicine also incorporated the briefing into its daily department email. Listserv subscribers consisted of health care providers, researchers, and government employees, as well as UNM faculty, students, and readers not involved in health care or state policy. The UNM Digital Repository runs on BePress, which tracks global downloads of the briefings. The briefings have also been indexed by Google Scholar. There was no monetary gain associated with the creation of these briefings, with full access provided free of charge.

COVID-19 Global Health Briefing Survey
To assess the usefulness of the briefings to New Mexico policy makers, researchers, educators, physicians, and other health care professionals, an 18-question survey was created through Microsoft Forms, approved by the Institutional Review Board , and distributed to all briefing recipients. Survey participant demographics included academic degree(s) and place of occupation. We determined if participants provided direct clinical care to COVID-19-positive patients, if the briefing was their primary source of information regarding the pandemic, and how likely they were to share the briefing with colleagues.
To determine the briefing's usefulness to the respondent, we assessed the following on a scale from strongly disagree to strongly agree: 1. The daily briefing has informed or changed my response to the COVID-19 crisis. 2. I have gained relevant knowledge I wouldn't have otherwise because of this briefing. 3. This briefing has enabled me to combat misinformation. 4. This briefing has enabled me to clarify information I have seen elsewhere.
Lastly, the respondent could respond in free text how information from the briefing was applied professionally or personally and if they had comments or suggestions to improve the briefings.

Medical Education
Medical student education in the hospital setting was suspended, and students were not involved in direct patient care during the initial surge of COVID-19 around the globe. The UNM School of Medicine created a virtual COVID-19 course to allow for virtual didactics as well as asynchronous, self-directed learning. This course allowed students to be involved in various projects and, with curriculum committee approval, receive a 4th year elective credit for those efforts. A few students who were involved in the briefing proposed to the UNM School of Medicine curriculum committee that contribution to the briefings should count as course credit. The UNM School of Medicine approved the proposal.
Medical students were involved in manual submission of information, summarizing articles, editing the briefings every evening, and recruiting other students to join the project. The students also assisted in the assembly of the COVID-19 global health briefing survey. Lastly, the compilation and analysis of the data in this publication was entirely led by medical students who took part in various stages of the COVID-19 briefing development and dissemination.

Briefing Process
Microsoft Beginning April 24, 2020, all briefings were uploaded to the UNM Digital Repository for public access. The number of briefing downloads is plotted both daily and cumulatively in    (88/111) reported the briefings helped them combat public misinformation, and 89% (99/111) said it helped to clarify the information from other data sources (Figure 4). Moreover, 24% (27/111) of respondents cited the briefing as their primary source of information on the pandemic, and 73% (81/111) reported having shared the briefing with their colleagues. On a scale of 0 ("would never share") to 10 ("will definitely share"), the respondents reported being very likely to continue to share the briefing with colleagues (mean score 8.8, 95% CI 8.5-9.1).  . Survey responses from "strongly disagree" to "strongly agree".

Medical Education
The UNM Global Health COVID-19 Intelligence Report project enhanced medical student education. A total of 31 medical students contributed to the creation of the daily briefings.
Participation in the project was approved for course credit by the UNM School of Medicine curriculum committee. The students were able to practice the analysis and quality assessment of scientific data, and they found participation in the briefings to be informative and rewarding (Textbox 1). Textbox 1. Direct quotes from medical students regarding the impact of the briefings on medical education.

Medical students' statements
• "[The briefing] provided a meaningful purpose as we were able to contribute directly to our community's wellness …. I appreciated participating, seeing daily updates, and knowing hospital teams were making real time policy changes with the help of the information provided in the reports." • "Once the COVID-19 block started it was refreshing to use my skills to analyze articles because I was already doing so on my own… It was great to be able to create deliverables as part of the project and feel like I was making an impact to the state of New Mexico, which eased my anxiety significantly." • "This style of project is immensely helpful for students who want to be more involved with research and have a limited history of working in research. I hope that every project I work on subsequently will be so well organized." • "Participating in the daily briefing helped me stay connected to the ever-changing world of COVID research. In a world of information overload, helping with this project grounded me to the primary information coming out of the scientific world. I felt confident giving friends and family advice and educating them on this pandemic because of the briefing." • "Being involved in this project has helped me learn more about this novel virus and feel more confident about the information I can disseminate to my friends and family. It also taught me about how difficult it can be to find trustworthy news, and how much of a problem can misinformation be. Even my parents (who are college educated with scientific degrees) presented wrong information to me many times. This reminded me of the importance of reliable news sources these days, and how we struggle to find them."

Strengths and Principal Findings
The development of the briefing succeeded in informing the practice of physicians and other health care personnel, facilitating COVID-19 research efforts by faculty and staff in academia, influencing policy making by the government of New Mexico, and alerting the public about the most relevant developments as is evident in our survey results. Additionally, the briefings were accessed by approximately 3 times the number of people compared with those who were directly emailed and catered to an audience beyond the scientific and medical community as the categorization of articles with headers and summaries reported in everyday language.
This project also succeeded in allowing for the continuation of medical education during the initial surge of COVID-19 when medical students and research students were prohibited from their normal duties. The learners improved on their ability to analyze a scientific paper and helped them empathize with the public regarding misinformation. The students reported being part of a community project to be meaningful and that influencing the health of New Mexico reduced student anxiety of being away from direct patient care.
The utilization of a listserv and online archive helped expand the audience beyond medical advisory team and the state of New Mexico to the briefing's eventual global reach. Organizing interest through a listserv would help disseminate the material quicker and broader should another similar endeavor be attempted.
Due to both limited funding and volunteers, the effort was brought to a close as the final briefing was published on June 30, 2020. We were unable to continue the production of thorough, high-quality publications free of charge due to the inevitable loss of many of our volunteers. We were unable to replace team administrators and leadership to triage daily and ensure quality control. Medical students returned to clinical duties, and faculty were unable to simultaneously sustain this unfunded effort along with their teaching, research, and administrative responsibilities. Between April 9, 2020, to July 2, 2021, a similar COVID-19 briefing effort was created ("COVID-19 Literature Surveillance Team") of medical students and faculty, who were able to continue their reports by changing their daily briefing to weekly [12]; however, this team ultimately discontinued publication [13]. Our COVID-19 briefing efforts were closed with a presentation that expressed gratitude to all who participated [14].

Limitations
Contributors and authors of the briefings had variable expertise in assessing scientific literature. Initially, all participants were allowed to triage content. Due to quality concerns, triaging was reduced to 3 administrators. It was not obvious how to standardize the judgment process that went into rapid decision-making regarding article inclusion. Those triaging needed to be aware of the prior body of work addressed, be able to analyze an article quickly, and be well versed in various research methodologies and their assessment. Standardized training for editors and authors was implemented to improve the consistency and quality of article summaries.
Initial article selection was less stringent and included more speculative reports. More selective criteria were implemented over time. The inclusion of articles of varying evidence, such as preprints, received criticism. Recent studies have shown that the standard in preprints for the life sciences is similar to that of peer-reviewed articles; therefore, they can be considered valid scientific contributions [15]. As a clarification, our later briefings included the source and type of study (preprint, meta-analysis, Reuters, etc) before each summary.
A major concern was the disagreement between global governing entities on treatment and the quickly evolving understanding of the virus. For example, the US Food and Drug Administration granted emergency use authorization for both chloroquine and hydroxychloroquine for certain hospitalized COVID-19 patients on March 28, 2020, whereas the European Union did not allow for its use. The emergency use authorization was revoked on June 15, 2020, due to side effects of the medications.
While our survey was disseminated to all subscribers of the briefing, persons who may have derived less value from the briefings or had more substantial criticism of the publication were not well represented among the respondents. While favorable impact ratings were provided by survey respondents, our free-text section (on how the respondent-applied information from the briefing influenced their active response to the pandemic) could have benefitted from more specific wording to solicit specific uses. The most substantive feedback came from a nonsurvey source, namely Secretary Scrase via email.

Comparison to Prior Work and Future Directions
Other programs have released daily briefings [16,17,18,19,20] in similar efforts to combat information overload and implement quality control during the COVID-19 pandemic. At the time of this publication, there has been no research that has compared briefing dissemination to other forms of information dissemination during a rapidly changing health crisis. The pandemic has shifted the scientific community toward a more robust and rapid data-sharing culture [21]. Future work could compare the efficacy of briefings compared to other measures to alter and improve policy and information dissemination.
LJ: administrative support (student lead for the COVID-19 curriculum; organizing, recruiting, and training students), briefing content, briefing editing, briefing triage, survey development, drafting substantial portions of the manuscript, and critical revisions to the manuscript. JS: briefing content, briefing editing, and manuscript draft and revision. RDK: administrative (student leader for COVID-19 block 1), founding member, briefing content, briefing editing, briefing triage, and manuscript revisions. EM: student coordinator for continuing education, founding member, student leader, briefing content, briefing editing, briefing triage, and manuscript revisions. HG: training material development, briefing triage, briefing content, briefing editing, and manuscript revisions. RO: briefing content, briefing editing, survey development, data analysis, drafting substantial portions of the manuscript, and critical revisions to the manuscript. MC: briefing content, briefing editing, survey development, data analysis, drafting substantial portions of the manuscript, and revisions to the manuscript. AR: briefing content, briefing editing, and manuscript drafting. AM: briefing content, briefing editing, and manuscript draft and revision. JT: briefing content, briefing editing, and manuscript draft and revision. RFK: briefing content, briefing editing, briefing triage, and manuscript drafting and revisions. SP: briefing content, briefing triage, and manuscript drafting. AP: briefing content, briefing triage, and briefing editing. AH: administrative, briefing content, and manuscript drafting. AY: briefing content, briefing editing, and briefing triage. TC: briefing content, briefing editing, and survey development. LTE: briefing content, briefing editing, and manuscript revisions. FN: briefing content, briefing editing, and manuscript revision. EW: briefing content and briefing editing. ME: briefing content, briefing editing, and manuscript revision. JL: briefing content and briefing editing. DR: briefing content, briefing triage, and manuscript revisions. JL: organization and obtaining institutional approval for a medical student COVID-19 elective workforce; contribution to the conceptual design of the manuscript; briefing editing; and manuscript editing. LS: administrative support, briefing content, briefing editing, briefing triage, dissemination activities, survey development, data analysis, and substantial manuscript draft and revision. IH: briefing content from medRxiv, bioRxiv, PsyArXiv, and arXiv. COO: briefing content. PKO: briefing content. SBA: briefing triage, briefing editing, manuscript revisions, and administrative tasks (Kenyan group). EOM: briefing content. PK: software infrastructure and maintenance, technical maintenance, briefing triage, briefing content, and briefing editing. NL: software infrastructure and maintenance, technical maintenance, briefing triage, briefing content, and briefing editing. SS: administrative tasks and briefing editing. AN: briefing triage, content, editing, and manuscript editing. CGB: briefing content, briefing editing, briefing triage, dissemination activities, survey development, and critical revisions to the manuscript. TIO: briefing editing, briefing content, dissemination activities, and manuscript editing. KT: briefing content, briefing editing, and manuscript editing. OM: briefing content and briefing editing. DJP: briefing content and briefing editing. CGL: project and editorial leadership, software infrastructure and maintenance, briefing content, briefing editing, briefing triage, technical maintenance, dissemination activities, survey development, and critical revisions to the manuscript.

Conflicts of Interest
None declared.

Multimedia Appendix 1
Data accumulation.