The Impact of ChatGPT and LLMs on Medical Imaging Stakeholders: Perspectives and Use Cases

This study investigates the transformative potential of Large Language Models (LLMs), such as OpenAI ChatGPT, in medical imaging. With the aid of public data, these models, which possess remarkable language understanding and generation capabilities, are augmenting the interpretive skills of radiologists, enhancing patient-physician communication, and streamlining clinical workflows. The paper introduces an analytic framework for presenting the complex interactions between LLMs and the broader ecosystem of medical imaging stakeholders, including businesses, insurance entities, governments, research institutions, and hospitals (nicknamed BIGR-H). Through detailed analyses, illustrative use cases, and discussions on the broader implications and future directions, this perspective seeks to raise discussion in strategic planning and decision-making in the era of AI-enabled healthcare.

governments (G), research institutions (R), and hospitals (H), which provides a structured approach to dissect the complex interactions between LLMs and various stakeholders in the medical imaging landscape.
In the subsequent sections, we elaborate on the impacts of LLMs on each BIGR-H stakeholder, present several illustrative use cases, and delve into the broader implications and future directions of LLM integration in the medical imaging domain. By providing a comprehensive understanding of these elements, this perspective aims to contribute to the strategic planning and decision-making processes in the era of AI-enabled healthcare.

Impacts on Medical Imaging Stakeholders (BIGR-H)
The healthcare ecosystem is an intricate and dynamic network designed to address multiple layers of health needs, from prevention and primary care to specialized and palliative care. This complex, interconnected network of various entities is orchestrated around the key mission of delivering effective healthcare services to all individuals. This intricate network of relationships motivates us to introduce the BIGR-H framework for discussion, encompassing businesses (B), insurance entities (I), governments (G), research institutions (R), and hospitals (H). Each of these stakeholders brings unique values and objectives to the ecosystem. Businesses, including biopharmaceutical and medical device companies, fuel innovation and bring new therapies and technologies to the market. Insurance entities play a critical role in risk pooling and financing healthcare services. Governments, on the other hand, are primarily responsible for regulation, health policy formulation, and ensuring that the population's health needs are met. Research institutions generate scientific knowledge and insights that propel healthcare innovation, while hospitals and other healthcare providers deliver essential healthcare services directly to patients. The interactions among these stakeholders are often multi-faceted and highly interdependent.
In this complex web of interactions, LLMs such as ChatGPT are emerging as influential players. With their unparalleled ability to interpret health data, improve communication, and facilitate decision-making processes, these models are transforming the roles and operations of each stakeholder in the healthcare ecosystem [17,18]. Thus, the integration of LLMs into the healthcare ecosystem is not just enhancing existing capabilities but also redefining the interactions and balance among the stakeholders, heralding a new era of high-performance medicine [23]-a paradigm defined by technology-enabled, data-driven healthcare services.
In the subsequent section, we will examine the wide-ranging effects of ChatGPT and LLMs on various medical imaging stakeholders.

Businesses (B).
Within the medical imaging ecosystem, businesses play a critical role in bringing technological advancements to market. Medical device manufacturers often pioneer cutting-edge diagnostic equipment, while contract research organizations (CROs) expedite the development of new devices and treatments through clinical trials. These entities handle vast data volumes, making them ideal candidates for LLM integration.
Medical device companies are vital components in the evolution of healthcare by developing sophisticated equipment capable of capturing intricate details of the human body. Some have already incorporated AI into their offerings, crafting medical imaging tools that provide more accurate diagnoses and predictions [4]. LLMs can serve as valuable tools for analyzing copious amounts of user feedback and technical documentation, which could provide significant insights, leading to the evolution of existing devices. The applicability of LLMs can also optimize user manuals and device operation guidelines into talking agents, enhancing their clarity and comprehensibility for healthcare professionals. Besides, existing AI diagnostic tools augmented by LLMs can facilitate interactive patient consultation, thereby improving patient understanding and engagement.
In parallel, CROs could harness the power of LLMs to re-engineer their data processing and analysis workflows in clinical trials. Traditionally, clinical trial data management is labor-intensive and often prone to human error. By employing LLMs in data handling and analysis, these organizations can bolster the speed and accuracy of their research processes. The resultant streamlining could lead to shorter turnaround times and more reliable outcomes, thereby accelerating the translation of research into viable treatments.
Insurance (I). Insurance entities, including health insurance companies and public health insurance providers, serve as vital intermediaries, assuming the financial risk of healthcare expenses on behalf of patients and making healthcare services financially accessible to the public. They stand to gain significantly from the integration of LLMs into their operations. These models can enhance customer service, facilitate more accurate risk assessment, and improve fraud detection. With the ability to process large volumes of data, LLMs can identify patterns and anomalies that could signify potential fraudulent activities, providing an essential tool for insurance companies to prevent such occurrences. In addition, LLMs can aid in answering policyholder queries and providing personalized advice, improving the overall customer experience and ensuring policyholders receive the most accurate and helpful information.
Government (G). Government entities, encompassing regulatory agencies (e.g., NMPA and FDA), along with public health authorities (e.g., health commissions and centers for disease control and prevention), play a fundamental role in ensuring the safety and efficacy of medical products and overseeing public health initiatives.
The integration of LLMs could significantly streamline the operations of these entities. For regulatory agencies, LLMs can augment the regulatory review process, assisting in the meticulous scrutiny of medical product submissions. On the other hand, public health authorities could also leverage the analytical prowess of these models to scrutinize large volumes of health data. By identifying trends and patterns, LLMs could significantly enhance disease surveillance capabilities, providing valuable insights for proactive disease control and prevention measures. Moreover, these insights could inform the formulation of more targeted and effective health policies, optimizing resource allocation and contributing to overall public health outcomes.
Research (R). Academic institutions and industry research stand on the brink of a transformative leap with the integration of LLMs into their research endeavors. These models are adept at interpreting and analyzing extensive biomedical datasets, facilitating more precise conclusions and novel discoveries. In addition, their capacity to discern research gaps and generate new hypotheses can catalyze medical research and innovation.
In bridging the gap between research and education, particularly in medical imaging, LLMs hold substantial promise. They could function as personalized instructors, elucidating complex imaging concepts and results, thereby fostering a more intuitive understanding for students. By creating a learning environment that simulates real-world imaging scenarios and provides interactive interpretations, LLMs can enrich the learning experience.
Hospital (H). The delivery of clinical services takes place in various settings, but hospitals, encompassing a broad spectrum of medical examination institutions such as radiology service providers and physical examination centers, are at the core. These settings are integral parts of the healthcare service delivery chain where LLMs can have profound implications, significantly affecting the dynamics of medical imaging processes.
Within the scope of medical imaging in hospitals, stakeholders such as radiologists, referring physicians, hospital administrators, and patients can interact uniquely with LLMs. Radiologists and referring physicians can utilize LLMs as advanced decision-support tools. These models' prowess in interpreting vast quantities of radiology reports can potentially augment diagnostic precision and streamline imaging processes. Hospital administrators, often tasked with coordinating various hospital units, could incorporate LLMs to enhance administrative tasks such as scheduling and inter-departmental communication. On the other hand, patients, as the central figures in the healthcare landscape, stand to benefit from LLMs serving as advanced communication platforms. These models can distill complex medical information into more understandable terms, encouraging patients to participate actively in their healthcare journey.

Use Cases
In this section, we explore various use cases of ChatGPT and LLMs in medical imaging, illustrating their potential to enhance clinical workflows, consultations, and medical education and research. The examples are generated via prompt engineering OpenAI ChatGPT.
Optimizing Clinical Workflow. LLMs have the potential to significantly optimize clinical workflows, particularly in medical imaging contexts. Through their ability to interpret and analyze vast quantities of data quickly and accurately, they can streamline several time-consuming tasks in clinical workflows [1]. One critical application is in the structuring and generation of clinical reports. For example, interpreting scans and writing reports in radiology can be timeconsuming. As illustrated in Fig. 1, LLMs could help automate part of this process by transforming unstructured text into structured data (clinical report structuring) and vice versa (automated report generation).
In clinical report structuring, LLMs can convert a narrative radiology report into a structured format such as JSON, which is easier to process and analyze. This not only improves data management but also enhances interoperability across healthcare systems. On the other hand, in automated report generation, LLMs can utilize structured clinical data (e.g., JSON), which could be generated by another AI systems, to create a narrative report that can be easily Convert the radiologist's report below into JSON with indent. Don't output text other than JSON.
Enhancing Patient Communication. LLMs can serve as powerful tools to enhance patient communication. Often, radiology reports and medical jargon can be perplexing for patients, creating a barrier to understanding their own health status. LLMs can bridge this gap by translating complex medical language into comprehensible layman's terms, thereby improving patient understanding and engagement [3]. For instance, given a radiology report, LLMs can generate a simplified explanation and answer patient queries about their diagnosis in a natural language conversation (Fig. 2). The interactive diagnosis provided by LLMs, combined with their capacity to handle numerous and diverse patient inquiries, marks an advancement in personalized patient communication.

Supporting Medical Education and Research
LLMs are becoming increasingly impactful in the realm of medical education and research, notably in the specialized field of medical imaging. In the educational setting, LLMs can serve as virtual tutors [17], answering complex questions about different imaging techniques or specificities of certain pathological findings. For example, a student can ask about the relationship between various cardiac MRI sequences ( Fig. 3 (a)), and the LLM could provide a detailed, yet understandable explanation. This personalized learning assistance can supplement traditional teaching methods and support students in navigating complex medical subjects.
In terms of research, LLMs can assist in the design of research plans by providing insights on experimental design, data analysis methods, or even suggesting relevant literature. As illustrated in Fig. 3 (b), given a research problem, an LLM can help conceptualize the research structure, formulate hypotheses, and propose suitable methods for data collection and analysis. Notably, the answer aligns well with previous studies for the same research questions [29,26].
Your task is to explain the radiologist's report below to patients. First, you need to translate the report into comprehensible layman's terms. Then, if patient ask questions, you should answer them in comprehensible layman's terms.
Radiologist's Report: """ Patient: John Doe, Age: 55, Sex: Male, Date of Scan: 2023-05-01. The CT scan of the chest shows a 3.1 cm spiculated mass in the right upper lobe of the lung. There are also several enlarged mediastinal lymph nodes. The heart size is within normal limits. There is no pleural effusion or pneumothorax. There are degenerative changes in the spine. Conclusion: Findings suggest a lung malignancy. Recommend PET scan to further evaluate the mass and lymph nodes. """ Sure, I'll put this into more understandable terms for you. Patient Information: John Doe, Age: 55, Gender: Male, Date of the Scan: 1st of May, 2023. Your CT scan of the chest shows a growth (3.1 cm) with irregular and spiky edges in the upper part of your right lung. Additionally, there are several lymph nodes (small glands that filter out harmful substances) in the area between your lungs (known as the mediastinum) that are larger than usual. The size of your heart looks normal, which is good. There's no buildup of fluid around your lungs (known as pleural effusion), and there's also no sign of a collapsed lung (pneumothorax). However, there are signs of wear and tear in your spine, which is common as we age. In conclusion, these findings suggest there might be cancer in your lung, but more testing is needed to confirm. It's recommended that you have a PET scan, which is a type of imaging test that can help to evaluate the growth in your lung and the enlarged lymph nodes more clearly.
What is a PET scan? Is it bad? A PET scan, or Positron Emission Tomography scan, is a type of imaging test that doctors use to check for diseases in the body. It uses a special dye containing radioactive tracers. These tracers are injected into a vein in your arm. Your organs and tissues then absorb the tracer. When highlighted under a PET scanner, the tracers help your doctor to see how well your organs and tissues are working. The scan can measure important functions such as blood flow, oxygen use, a nd glucose metabolism, which helps doctors to identify abnormal from normal functioning cells. In the context of you, it can help to provide more information about the growth in your lung and the enlarged lymph nodes. As for your second question, PET scans are generally safe. The radioactive material in the tracer is small so the risk of developing side effects from it is low. However, like any medical procedure, it does have some risks. People with certain conditions, like kidney disease or diabetes, may need to take certain precautions. You'll also need to stay well-hydrated to help flush the radioactive tracer out of your body. It's important that you discuss any concerns or questions you may have with your doctor. They can provide information tailored to your personal medical history and condition.
What does "enlarged mediastinal lymph nodes" mean? Sure, let's break that down: The mediastinum is the space in the chest between your lungs. It contains various organs and tissues, including your heart, esophagus, trachea, and lymph nodes.
Lymph nodes are small, beanshaped organs that produce and store cells that fight infection and disease. They are a vital part of your immune system. When lymph nodes are referred to as 'enlarged,' it means they are bigger than normal. This can occur for several reasons. The most common cause of enlarged lymph nodes is infection, as they may swell while producing extra white blood cells to fight off the infection. However, other conditions such as inflammation, an immune response, or cancer (like lymphoma or spread from another location, called metastasis) can also cause lymph nodes to enlarge. In your case, the report suggests that several lymph nodes in the mediastinum area of your chest are larger than usual. This could be related to the growth identified in your lung. The PET scan recommended by the radiologist will help doctors better understand why these lymph nodes are enlarged.

Implications and Future Directions
The incorporation of LLMs like ChatGPT in medical imaging has begun to profoundly reshape this field, presenting immense opportunities and transformative potential for all involved parties. The advantages offered by these models are twofold. Firstly, they bring an extraordinary capability to decode complex language structures, leading to simplified understanding and generation of diverse and intricate medical jargon. This novel approach heralds a new era of communication and data interpretation, essentially making the discourse in medical imaging more accessible and meaningful to a broader audience-from seasoned clinicians and radiologists to healthcare staff and patients. Secondly, LLMs empower entities with logical programming abilities traditionally reserved for computer scientists and software engineers. By making these capabilities more accessible, LLMs democratize advanced technology, thereby providing an equal opportunity for all healthcare stakeholders to contribute to and benefit from AI's progress.
These advantages manifest as tangible improvements in healthcare delivery. LLMs can improve diagnostic precision by providing more nuanced and comprehensive interpretations of medical imaging data. Efficiency in clinical workflows is enhanced as LLMs automate the reading and interpretation of diagnostic imaging reports, reducing manual workload and allowing healthcare professionals to devote more time to patient care. Furthermore, by generating patient-friendly explanations of imaging results, LLMs can enhance patient engagement, making healthcare a more collaborative and holistic experience. Another major advantage of LLMs is their potential to revolutionize medical education. They can provide personalized, intuitive, and interactive learning experiences, adapting to individual learning styles and pacing. This dynamic environment fosters a deeper understanding of complex medical imaging concepts and enhances skill development.
What's the relationship between various cardiac MRI sequences?
Magnetic Resonance Imaging (MRI) is a powerful tool in the diagnosis and management of many cardiovascular diseases. It utilizes various sequences to acquire different information about the heart. The main cardiac MRI sequences include: 1. **Cine sequences:** These are used to assess cardiac function, including the evaluation of ventricular volume, mass, ejection fraction, and wall motion abnormalities. Cine imaging is typically performed using steady-state free precession (SSFP) sequences due to their high signal-tonoise ratio and good contrast between blood and myocardium. 2. **T1-weighted imaging:** It's useful for the assessment of fat and water content in tissues. In the heart, T1-weighted imaging can be useful to detect fat in the myocardium (as in arrhythmogenic right ventricular cardiomyopathy) or to identify edema (as in myocarditis or early ischemia). 3. **T2-weighted imaging:** T2-weighted images are sensitive to water content and are used to detect edema in the myocardium, which could indicate inflammation or acute injury. 4. **Late Gadolinium Enhancement (LGE) imaging:** Gadolinium is a contrast agent that is distributed in the extracellular space. It's typically used to assess myocardial viability, fibrosis, or scarring. Areas of the myocardium that have been replaced by fibrous tissue (as in infarction or cardiomyopathies) will show late enhancement. 5. **T1 and T2 mapping:** These techniques are used to quantify the relaxation times of the tissues, which can provide information about their composition. For example, an increase in T1 relaxation time can indicate increased water content or fibrosis, and changes in T2 relaxation time can indicate inflammation or edema. 6. **Perfusion imaging:** This involves the use of gadolinium contrast to assess myocardial blood flow. Areas of the myocardium that are ischemic due to coronary artery disease will show reduced perfusion. 7. **Phase-contrast imaging (velocity mapping):** This technique measures the speed and direction of blood flow within the heart and great vessels. It can be used to quantify valvular regurgitation or shunts. 8. **Black-blood sequences:** These are used to suppress the signal from blood, and are particularly useful for evaluating the morphology of the heart and great vessels, or for assessing pericardial disease.
Each sequences provides a different aspect of cardiac anatomy, function, or pathology, and the choice of sequences depends on the clinical question.
I would like to design a research study using deep learning to investigate the relationship between lung nodules in CT images and their postoperative pathological infiltration degree. How should I design the research plan? Additionally, which experts' assistance do I need? Please list specific roles and capabilities. **1. Formulation of the research plan:** **a. Problem Definition:** You aim to investigate the correlation between the characteristics of lung nodules in CT images and the degree of postoperative pathological infiltration. **b. Hypothesis:** The characteristics of lung nodules in CT images can predict the degree of postoperative pathological infiltration. **2. Data Collection:** You will need a large and diverse dataset of pre-operative CT images of lung nodules, along with corresponding post-operative pathological data. The dataset should contain patient demographics and other clinical parameters. **3. Data Preprocessing:** The data needs to be cleaned and formatted into a usable form. Any irrelevant or sensitive patient information should be removed or anonymized. **4. Building the Model:** Train a deep learning model using your dataset. Various models can be used, such as convolutional neural networks (CNNs), which are particularly effective for image analysis tasks. **5. Model Evaluation:** Once the model is trained, evaluate it using unseen data (test set). The metrics used to evaluate the model might include accuracy, precision, recall, F1 score, and area under the receiver operating characteristic (ROC) curve (AUC-ROC). **6. Analysis:** Analyze the results and identify the most relevant features the model uses to make predictions. This can provide insights into which features of the nodules are most indicative of pathological infiltration. **7. Validation:** Validate the model using an independent external dataset. This will help ensure the generalizability. **8. Interpretation:** Use the results of your study to draw conclusions and make inferences about the relationship between lung nodules in CT images and their postoperative pathological infiltration degree. **9. Documentation and Publication:** Finally, document all your findings and submit them for publication. **Experts needed for the research plan:** 1. **Radiologist:** To help with understanding and interpreting CT images of lung nodules and to ensure that the data set is appropriately labeled. 2. **Pathologist:** To help with the interpretation of the postoperative pathological data and to validate the ground truth labels of pathological infiltration degree. 3. **Data Scientist or Machine Learning Engineer:** To handle data preprocessing, model development, training, testing, and validation. 4. **Medical Statistician:** To help with the statistical analysis of the data, including validation of the model's results. 5. **Ethicist or Legal Expert:** Since you're working with medical data, you need to ensure the privacy and confidentiality of patient data are maintained, and the study adheres to necessary ethical guidelines. 6. **Domain Expert (Thoracic Surgery or Oncology):** To provide insight into clinical relevance and applicability of the findings, as well as assist in interpreting results in the context of current clinical practice. 7. **Research Coordinator:** To manage the logistics of the project, including coordination between different experts, schedule management, and ensuring the project is on track. 8. **Medical Writer/Scientific Writer:** To assist in the documentation and publication of the findings in journals or conferences. Despite the considerable benefits, the advent of LLMs in medical imaging raises several significant challenges [17,21]. Ethical issues abound, primarily concerning the degree of trust and responsibility placed in AI technology. It's crucial to contemplate the potential ramifications when AI errs or fails to deliver the expected results. Alongside this, the safety and privacy of patient data are of paramount importance. LLMs deal with vast amounts of sensitive patient data, making robust security measures and strict adherence to privacy regulations non-negotiable.
Bias and fairness are also critical issues [30,20]. LLMs learn from the data they are trained on; if this data is inherently biased, these biases could be replicated in AI predictions, leading to unfair outcomes and exacerbating existing healthcare disparities. Regulatory and legal challenges also emerge, as the integration of LLMs requires clear, comprehensive guidelines from regulatory bodies to protect patient safety and ensure data integrity.
Looking ahead, future directions for LLMs in medical imaging could include exploring multi-modality [9,18], wherein LLMs are trained on a combination of text, medical imaging, and other omics data [16,28,10] for more nuanced analysis and interpretation. Improving liability and reducing malpractice is another crucial aspect [11]. As LLMs become increasingly integrated into patient care, systems must be put in place to deal with potential errors and their repercussions.
In conclusion, the deployment of LLMs like ChatGPT offers an exciting new frontier in medical imaging, yielding promising use cases that can revolutionize clinical workflows, patient communication, and medical education and research. Despite the considerable challenges, with continued research, collaboration, and careful consideration of ethical and regulatory guidelines, the future of LLMs in medical imaging holds remarkable promise.