Characterizing Myasthenia Gravis Symptoms, Exacerbations, and Crises From Neurologist's Clinical Notes Using Natural Language Processing

Background Myasthenia gravis (MG) is a rare, autoantibody neuromuscular disorder characterized by fatigable weakness. Real-world evidence based on administrative and structured datasets regarding MG may miss important details related to the clinical encounter. Examination of free-text clinical progress notes has the potential to illuminate aspects of MG care. Objective The primary objective was to examine and characterize neurologist progress notes in the care of individuals with MG regarding the prevalence of documentation of clinical subtypes, antibody status, symptomatology, and MG deteriorations, including exacerbations and crises. The secondary objectives were to categorize MG deteriorations into practical, objective states as well as examine potential sources of clinical inertia in MG care. Methods We performed a retrospective, cross-sectional analysis of de-identified neurologist clinical notes from 2017 to 2022. A qualitative analysis of physician descriptions of MG deteriorations and a discussion of risks in MG care (risk for adverse effects, risk for clinical decompensation, etc.) was performed. Results Of the 3,085 individuals with MG, clinical subtypes and antibody status identified included gMG (n = 400; 13.0%), ocular MG (n = 253; 8.2%), MG unspecified (2,432; 78.8%), seropositivity for acetylcholine receptor antibody (n = 441; 14.3%), and MuSK antibody (n = 29; 0.9%). The most common gMG manifestations were dysphagia (n = 712; 23.0%), dyspnea (n = 626; 20.3%), and dysarthria (n = 514; 16.7%). In MG crisis patients, documentation of difficulties with MG standard therapies was common (n = 62; 45.2%). The qualitative analysis of MG deterioration types includes symptom fluctuation, symptom worsening with treatment intensification, MG deterioration with rescue therapy, and MG crisis. Qualitative analysis of MG-related risks included the toxicity of new therapies and concern for worsening MG because of changing therapies. Conclusions This study of neurologist progress notes demonstrates the potential for real-world evidence generation in the care of individuals with MG. MG patients suffer fluctuating symptomatology and a spectrum of clinical deteriorations. Adverse effects of MG therapies are common, highlighting the need for effective, less toxic treatments.

The physician-patient encounter remains the primary setting for MG care, yet little is known about what occurs during these encounters.The application of natural language processing (NLP) methods to clinical notes is an active area of research and has been applied to identify symptoms, conditions, and laboratory results, as well as generate digital patient phenotypes [32][33][34][35].NLP has been used to study neurologic disorders, including stroke, transient ischemic attack, epilepsy, headaches, multiple sclerosis, and cerebral aneurysms [36,37].To our knowledge, NLP has not been previously used to study myasthenia gravis.
To better understand care delivery during the clinical encounter and the language used by physicians to describe MG-related phenomena, we performed a mixed-methods study of neurologist clinical notes using NLP and qualitative analysis of physician documentation.
To better understand care delivery during the clinical encounter and the language used by physicians to describe MG-related phenomena, we performed a mixed-methods study of neurologist clinical notes using NLP and qualitative analysis of physician documentation.

Study objectives
The objectives of this study were to analyze neurologist clinical notes and examine physician language, clinical events, and challenges in managing myasthenia gravis.Specifically, the primary objective was to use NLP to assess provider documentation of details of the patient's MG condition (i.e., clinical subtypes, antibody status), symptoms (e.g., diplopia, dysphagia), and disease states (worsening, exacerbation, crisis).Additionally, this study sought to examine the experiences of individuals with a history of MG crises, specifically their negative experiences with MG therapies.Lastly, this study sought to examine neurologist documentation of MG acute exacerbations to identify meaningful classifications that could be used to track clinical outcomes.

Materials And Methods
We performed a retrospective, cross-sectional analysis of de-identified neurologist clinical notes from 2017 to 2022.The analysis was conducted on a medical transcription dataset, Amplity Insights, which includes full-text transcripts of dictated clinical notes covering 150,000 physicians from private practice, integrated delivery networks, and academic institutions from over 40,000 clinics and hospitals, with representation across all 50 U.S. states.The dataset includes initial office consultations, follow-up visits, urgent care visits, emergency department and hospital admissions and discharges, postoperative consultations, office notes, and referral letters.This study was limited to the analysis of de-identified clinician notes and did not involve the collection of data from or engagement with human subjects.

Identification and selection of eligible notes
The data licensed included 45,414 records that included the term "myasthenia gravis."Of these 45,414 notes, 6,557 were written by a neurologist.Eligibility criteria for the analyzable notes included only one patient per text file (37 multi-patient text files were excluded), NLP analysis-ready structure and format (416 notes with unusual formats, including carriage returns in the middle of sentences or notes without any structure, were excluded), and a positive assertion of MG (923 notes with laboratory orders related to MG, suspected diagnoses, family history of MG, and referential discussions of MG were excluded).The final analysis dataset included 5,183 notes attributable to 3,085 patients.

NLP model development
A rules-based NLP model was developed using spaCy, an open-source library for NLP, to create an analyzable dataset focused on MG-related clinical phenomena, including clinical subtypes, disease status, and symptoms.The NLP model development consisted of four major steps, including a) knowledge framework, b) tokenization and named entity recognition, c) assertion status and clinical status detection, and d) section-specific analysis (Supplemental Material A).

Tokenization and named entity recognition
To facilitate NLP rule development for MG symptoms, a mapping of synonymous terms and phrases was developed.For example, "eyelid droop" and "drooping eyelids" were mapped to "ptosis."The listing of synonyms was iterated over a review of hundreds of notes during development (Table 7 in the Appendices).SpaCy was used to consume sentences and annotate each word or phrase as a named entity or cue to prepare for the application of the MG NLP model.

Assertion status and clinical status detection
To clarify the meaning of a symptom or condition within the context of a sentence, assertion status detection rules were developed.Types of assertion status included "negation" (e.g., "no dysphagia"), "hypothetical" (e.g., "possible MG crisis"), and "family history" (e.g., "father with a history of MG").Similarly, to clarify the trajectory of symptoms within the context of a sentence, clinical status rules were developed.Types of clinical status included "improving" (e.g., "ptosis is getting better"), and "worsening" (e.g., "dysphagia is increasing").Cue terms and phrases for each of the assertion status and clinical status categories were identified (Table 8 in the Appendices).
MG NLP rules were developed iteratively using the named entity lists, cue term lists, and logical expressions to perform assertion status and clinical status detection.These rules were then applied to sentences to generate a final analytical dataset.

NLP assessment
The NLP model's performance with regards to correctly detecting assertion status (negation, hypothetical, family history, and positive assertion) was assessed for accuracy, recall, precision, and the F1 score (a balanced measure of the model's performance regarding recall and precision) (Figure 1 in the Appendices).Because the dataset contained large amounts of content not relevant to this study, model performance was assessed against 16 datasets of 50 randomly selected sentences containing at least one mention of a concept of interest, including dysphagia, ptosis, generalized myasthenia gravis, myasthenia gravis exacerbation, or crisis.Each of the 16 datasets was then manually annotated using Prodigy, a proprietary software application, to create a gold-standard dataset, and the results were compared with the NLP model applied to the same sentences (Supplemental Material B).

MG crisis identification and abstraction
MG crisis was identified with a two-step process beginning with a search to identify notes using the NLP results that contained positive assertions for MG crisis or notes containing a positive assertion of "respiratory failure," "mechanical ventilation," "intubation," "bilevel positive airway pressure," or "BIPAP."All candidate charts were manually reviewed to confirm the history of a current or past crisis or the use of mechanical ventilation caused by MG (i.e., not related to COPD or other respiratory conditions).Patients with a confirmed history of MG crises had all notes reviewed and abstracted for further analysis by an internal medicine physician (JD).A framework was developed and used in the abstraction that focused on triggering events, and problems with MG medications, including adverse effects, interactions with comorbidities, and difficulties with access or availability.

Qualitative analysis of physician language of acute MG decompensation and risks
To examine the language used by physicians in the care of MG, one of the physician authors (JD) identified candidate concepts for clinical events, including MG-related symptoms, deteriorations, and crises, which were then reviewed by the authorship team, including an expert in neuromuscular medicine.Terms and phrases were normalized to major concepts to assist with NLP concept identification (Table 7 in the Appendices).
The thematic grouping of physician documentation regarding clinical deteriorations and MG-related risks was performed over three one-hour sessions over a six-week period.Existing frameworks for clinical severity were reviewed in advance of the sessions, including the Myasthenia Gravis Foundation of America and International Guidelines [22,38].Demographic statistics (age, sex) were calculated based on the Amplity dataset meta-data.Percentages of symptoms, clinical subtypes, and clinical events found in clinical notes were calculated at the patient level.The analysis of physician language was grouped by theme and visualized with representative quotes.

NLP performance
The NLP model's performance to detect assertion status (negation, hypothetical, family history, and positive assertion) was characterized by an accuracy of 0.96, precision of 0.98, recall of 0.98, and F1 score of 0.97 (Table 9 in the Appendices).

Notes removed Number of notes remaining
All available notes that were labeled "myasthenia gravis" or "MG" by Amplity Insights 0 45414

Neurology notes -6557
Notes limited to a single patient and encounter 37 6520 Notes limited to formatting compatible with NLP analysis** 416 6104 Notes limited to patients with confirmed diagnosis of MG 921 5183

MG crisis
Of the 137 patients who were identified with MG crises, 100 notes (73.0%) explicitly used the term "crisis," with an additional 37 (

Thematic analysis of clinical events
Thematic analysis of physician sentences used to describe MG clinical events yielded the following categories: symptom fluctuations occurring within 24 hours identified by phrases such as "worse in the afternoons and evenings" or "gets worse as the days go by."MG deteriorations were categorized into three categories: "Symptom deterioration with the intensification of therapy," "Deterioration with hospitalization or rescue therapy," and "MG crisis or deterioration requiring mechanical ventilation."Physician discussions of their assessments of MG patients that included uncertainty were characterized as "suspected MG deteriorations or crisis" (Table 4).

MG symptom fluctuations
Changes to specific symptoms within 24 hours that do not require intensification of treatment.
"He states the dysphagia is worse in the afternoons and evenings." "She states that ptosis is not so severe in the morning; however, it gets worse as the day goes by." "This ptosis also varies in intensity; sometimes it is more, sometimes it is less prominent and it can happen at any time of the day or night."

MG symptom worsening with treatment intensification
Changes in MG status or symptoms that require additions or modifications to treatment but do not require admission to the hospital or rescue therapy (i.e., IVIg or plasmapheresis).
"With increasing problems with dysphagia and dysarthria, she was started on prednisone 10mg daily." Symptom worsening terms include: "worsening", "increasing", "progressing", "aggravating" "He has episodic increases in diplopia and is utilizing a predominantly prednisone to treat his disease." "Based on the development of unequivocal extremity weakness, I started her on prednisone and she was recently up to 30 mg q.d." "This is an 82-year-old gentleman with diabetes, hypertension, hyperlipidemia and anemia, also has myasthenia gravis, which was treated last month with plasmapheresis for an exacerbation in which the patient developed difficulty swallowing and visual disturbance." MG status deterioration phrases include: "exacerbation", "flare", "out of control", "deterioration", "decompensation", "uncontrolled".
"Myasthenia gravis exacerbation: no need for intubation at this time, fvc 1.1 liters, able to communicate in full sentences, we will discontinue prednisone and continue IVIg, currently day 2 of 5." "She did receive IVIg this past year for a myasthenic flare." "A 29-year-old female admitted with exacerbation of bulbar myasthenia with respiratory insufficiency." "Looking through her medical record, she has been hospitalized nearly every month secondary to exacerbations of her myasthenia gravis."

MG crisis or deterioration requiring mechanical ventilation
Deterioration of MG requiring mechanical ventilation.MG status deterioration phrases include "crisis" "Impression: a 50-year-old gentleman with refractory myasthenia gravis resulting in recurrent episodes of respiratory failure requiring intubation and mechanical ventilation." "Reason for consultation: worsening myasthenia gravis requiring mechanical ventilation." "The patient has a history of refractory myasthenia gravis with several recent myasthenia gravis crises." "Myasthenia gravis, in crisis with acute respiratory failure status-post intubation and extubation."

Suspected MG deteriorations and crises
Concern for deterioration of MG status due to presenting symptoms but requiring more time or testing to be certain.
"Generalized weakness, suspected myasthenia gravis exacerbation with the potential for crisis." Suspected MG deterioration cues include a combination of MG deterioration terms and suspected cues including: "probable", "possible", "suspected", "differential diagnosis", "suggestive of", "monitor for", "impending" "Shortness of breath is concerning for possible exacerbation of myasthenia gravis." "However, an exacerbation of myasthenia gravis could not be ruled out completely." "Impression: Myasthenia gravis, possibly going into crisis, because her treatment was delayed for almost 2 to 3 weeks." "The impression is that of this gentleman with myasthenia gravis concern for impending myasthenia crisis." "At this point, he is also being worked up for the possibility of pulmonary effusion as well as possibility of pneumonia which could be contributing factors although i suspect that he is having a myasthenic crisis underlying at all."

Physician language, MG-related risks, and potential for clinical inertia
Neurologist discussions of MG-related risks were categorized into four groups: "Risk for changing therapy due to potential for toxicity and adverse side effects of new agents," "Risks related to toxicities of existing MG treatments," "Risk of changing existing MG treatments due to potential for worsening MG," and "Risk for not managing MG" (Table 5).Neurologist discussion or risks related to the toxicities of new or existing MG standard therapies ranged from short-term adverse effects, including increased respiratory secretions (5.8%), heart rhythm abnormalities (4.4%), decreased blood counts (4.4%), and renal failure (3.6%), as well as long-term effects related to corticosteroids and immunosuppression, including hyperglycemia (2.9%) and osteoporosis (1.5%).Discussions of reducing corticosteroids or other immunosuppression therapy were sometimes accompanied by neurologist and patient concerns regarding the potential for MG relapse and crisis, sometimes related to previous experience with deteriorations after corticosteroid taper.Neurologist concerns regarding adherence to therapy or non-treatment were often articulated in terms of the risk of MG deterioration, including life-threatening complications (Table 5).

Representative Text Examples
Risk for changing therapy due to potential for toxicity and adverse side effects of new agents "Again, from the point of view of his myasthenia, he is doing quite well.I do not think the risk of immunosuppressive agents, corticosteroids or even IVIG, there is a nationwide shortage of this material, are warranted by his relatively minor myasthenia complaints.Most of them are the more vague fatigue and just not as strong as he used to be and I think that that would not be a good indication for putting more toxic agent on.My advice is that he stay with the Mestinon.""I do not think he will be a good candidate for the Soliris because of the difficulties with the pancytopenia and the risk of concomitant infection with the complement inhibitor superimposed upon his chemotherapy."IVIg would not be recommended, especially due to high risk of hypercoagulation on IVIg in a patient who is actively suffering from pulmonary embolism, recently diagnosed about 2-3 weeks ago and on coumadin.

Risks related to toxicities of existing MG
treatments "This may increase his risk for recurrent malignancy if we are to continue azathioprine." "The patient is well aware about the risks of prednisone including avascular necrosis of the hips and our goal was to have the prednisone discontinued as soon as it is possible.""We may need to consider an alternate immunosuppressive agent such as azathioprine or mycophenolate for maintenance therapy to minimize longer term glucocorticoid toxicity, for which he remains at risk for."

Risk of changing existing MG treatments due to potential for worsening MG
"Reportedly, seropositive myasthenia gravis, on solid immunomodulatory therapy requiring refills.Has not had exacerbation in the last year.I will recommend to continue the course with a combined Imuran and prednisone.I would like to know what the Hematology has to say with regards to the Imuran.Also, any changes will certainly put her at risk of crisis." "We discussed decreasing the prednisone and other prednisone sparing activities, but he said he really is very comfortable now compared to especially when he first got it and he does not want to change things despite the possible risk of cataract opportunistic infections, aseptic necrosis of the hip, and osteoporotic fractures.""Certainly should he feel the imuran to be a risk, we could consult about possibly using another agent, though the patient has had myasthenia relapses when not being treated with imuran in the past."

Risk for not managing MG
I apprised him of the risk of leaving his generalized myasthenia untreated including respiratory arrest and myasthenic crisis, which could be fatal.The patient was strongly urged to stay compliant with her prednisone and mestinon for myasthenia_gravis and is fully aware of risks of not doing so.

Discussion
To better understand the care experience of individuals with MG and the challenges faced by practicing neurologists, we applied advanced analytic techniques as well as qualitative methods to clinical progress notes, which contain details not available in structured datasets.This investigation, the first of its kind, sought to assess the feasibility of extracting MG clinical subtypes and relevant symptoms from physician documentation and, at the same time, characterize the language used to describe MG clinical events as well as explore some of the challenges with medical decision-making and treatment in the care of individuals with MG.
Based upon the progress notes analyzed in the study, most neurologists describe the extent of myasthenia gravis through a description of clinical symptoms rather than using the terms "generalized" or "ocular."Neurologist notes frequently describe the presence, absence, characteristics, and trajectory of MG symptoms (e.g., ptosis, diplopia, dysphagia, and dysarthria).While numerous MG assessment instruments have been developed and validated for clinical research, including the myasthenia gravis activities of daily living (MG-ADL), myasthenia gravis disability scale, and myasthenia gravis quality of life 15 (MG-QoL15r), in our analysis, the use of these instruments was uncommon.
In this study, the language used by neurologists to describe clinical deterioration events ranged from mild fluctuations or worsening disease to more serious and substantial exacerbations of disease.to describe significant MG deterioration events including "exacerbation," "flare," "relapse," "deterioration," "decompensation," and "out of control" (Table 4).In contrast to this real-world clinical documentation, within the published literature, researchers have defined exacerbations using several different and more exacting methods.In a recent study of resource utilization, Phillips et al. used the ICD codes G70.01 (ICD-10) or 358.01 (ICD-9) to define MG exacerbations.The MGFA defines MG exacerbations and flares as worsening muscles throughout the body, but assistance is not required for breathing (i.e., presumably excluding MG crises); these may include worsened double vision, slurred speech, increased arm weakness, falling, unsteady walking, and difficulty swallowing [38].Other studies have used multiple criteria, including hospitalization, the use of rescue therapy such as IVIg or plasmapheresis, and changes in the quantitative MG score [3,39].
More serious deteriorations associated with respiratory weakness and mechanical ventilation are commonly and consistently referred to as MG crises [13,16,40].In our analysis of individuals with MG crises, the unpredictable and risk-laden nature of MG was emphasized.While some crises were associated with inciting events, primarily infections, more than three-quarters of MG crises were not associated with a specific trigger.Similarly, individuals with a history of MG crisis also commonly had a history of documented difficulties with MG treatments (43.8% of individuals with a history of MG crisis in this study).While the implications of some difficulties with treatment are relatively minor (e.g., stopping pyridostigmine due to excessive diarrhea or salivation), difficulties with receiving rescue therapies such as IVIg and plasmapheresis are potentially more serious.Neurologist documentation of medication difficulties in MG is consistent with published literature that discontinuations of MG medications due to adverse effects or poor response in the general population can run as high as 37% [7].
Our categorizations of neurologist documentation regarding risks and reluctance to treatment change, whether due to risk for deterioration or risk of treatment toxicity, identify potential drivers of clinical inertia in MG [31].Clinical inertia has been described in the treatment of hypertension, diabetes, hyperlipidemia, and heart failure, and in a recent qualitative study of living with MG, performed by patient advocates from seven countries, treatment inertia emerged as a major theme [4,[41][42][43][44].As reflected in the selection of neurologist statements (Table 5) as well as the analysis of myasthenia crisis patients (Table 5), many MG medications are associated with substantial toxicity, and prescribing providers may be reluctant to start new agents that expose individuals with MG to unwanted side effects.Further research is needed to better understand provider and patient perspectives regarding MG-related clinical inertia, risks related to treatment, and opportunities to improve communication and care.

Strengths
This study was the first application of NLP to examine the documentation of MG in neurologist clinical notes and provided an opportunity to examine physician language and the process of managing patients with MG, which is not feasible with coded datasets.This study sought to address the gap in knowledge of traditional observational research, which is routinely based on administrative (insurance claims) or structured clinical (EHR) data sources, and of qualitative studies of patient experience, which depend on patient-reported measures, by highlighting neurologist documentation of the clinical encounter.While MG is a rare condition, the clinical notes examined represented a wide spectrum of neurologists and patients from across the United States.The study results highlight the presence of important details contained within unstructured clinician notes (e.g., the presence of adverse effects to medications) as well as the variation in how providers describe their patients, symptoms, and outcomes, providing a baseline understanding of both the language of MG care as well as the limitations of existing documentation with regards to clinical subtyping and disease monitoring.Future research should seek to better understand physician needs and satisfaction with current documentation, leverage new digital capabilities to enhance documentation and overcome barriers to the use of patient-reported outcomes in routine MG care.

Limitations
The cross-sectional study design limited the ability to follow patient experience over time.Moreover, the dataset was limited to clinical notes only and thus provided little opportunity to assess the broader context of the clinical encounter.Future studies would benefit from using longitudinal data with additional data points to enable greater validation and learning.The NLP model did not seek to assess the timing of clinical events and symptoms in the clinical notes (i.e., prior versus current), limiting the interpretation of findings with regard to relationships between symptoms, treatment, and clinical events.The NLP development was heavily customized to this dataset to achieve high accuracy and precision and may not be generalizable to all physician notes.While this process supports the findings of this study, using the NLP rules for additional datasets would potentially return lower accuracy and precision.This study was limited to the notes that were available at the time of care and did not reflect the full set of information that was available to the practicing providers.As a result, clinical triggers for the MG crisis may have been present but not documented in the notes available in this study.

Conclusions
Neurologist documentation highlights the challenges of managing MG, a heterogeneous disorder with unpredictable and sometimes life-threatening manifestations.While mortality related to MG has improved over time, neurologist documentation reflects the ongoing challenges of fluctuating illness.Novel therapies and care delivery improvements may help achieve optimal outcomes for MG by matching patients to therapies that control symptoms, prevent exacerbations and crises, and limit exposure to adverse medication effects.The primary goal of the NLP model was to first to correctly identify the presence of concepts of interest within the progress notes and second to ensure correct interpretation of their meaning within the context of the sentence.The analysis focused upon identifying textual representing the concept of interest as well as the surrounding cues within sentences indicating the status of a that concept (e.g., hypothetical, negation, family history, or assertion) (see Supplemental Table 1).To further clarify the impact of cues upon the rulesbased model performance, all sentences in all notes were characterized by the number of cues present in the sentence.

Supplemental materials B -NLP assessment
Because the dataset contained large amounts of sentences with no mention of concepts of interest, the NLP model assertion status detection performance was assessed against randomly constructed datasets that included a mention of the concept of interest.To ensure the NLP model was assessed across a broad spectrum of the relevant concepts, 16 datasets of 50 randomly selected sentence containing at least one mention of a concept of interest including dysphagia, ptosis, generalized myasthenia gravis, myasthenia gravis exacerbation or crisis, were generated.For each of the concepts of interest, 4 datasets of 50 records were created that included 0-1 cue, 2 cues, 3 cues, and 4 or more cues.Each of the datasets was then manually annotated using Prodigy, a proprietary software application, and the results compared with the NLP model applied to the same sentences.

Figure 1 :
Figure 1: Definitions of accuracy, recall, precision, and F1 score using a confusion matrix

FIGURE 1 :
FIGURE 1: Definitions of accuracy, recall, precision, and F1 score using a confusion matrix

TABLE 5 : Physician language, risks of therapy, and potential sources of clinical inertia
*MG: Myasthenia gravis, IVIg: Intravenous immunoglobulin therapy

Table 8 :
Examples of cue terms-phrases

Table 9 :
Accuracy, Precision, Recall, and F1 scores for concepts by number of cues

TABLE 9 : Accuracy, Precision, Recall, and F1 scores for concepts by number of cues
F1 score is the weighted average of recall and precision which gives a more useful understanding of a model's performance especially with unbalanced datasets.A perfect F1 score is 1 and a failed model would score 0. *

TABLE 10 : Types of challenges or difficulties with specific MG therapies
*MG: Myasthenia Gravis, IVIg: Intravenous Immunoglobulin Therapy