According to a recent study published in Radiology on March 26, Large Language Models (LLMs) have shown promising results in simplifying radiology report impressions, thus making them more comprehensible for patients. The research, conducted by a team from Yale University led by Rushabh Doshi, involved analyzing 750 radiology reports. The team tested four different LLMs, including ChatGPT, Bard (now known as Gemini), and Bing (Microsoft), finding that all four models were successful in significantly simplifying report impressions. However, the effectiveness of each model varied depending on the wording of the prompt used.
The researchers aimed to address the challenge posed by the complex medical terminology often found in radiology reports, which can be confusing or anxiety-inducing for patients. They collected 150 reports each for CT, MRI, ultrasound, and diagnostic mammography from the Medical Information Mart for Intensive Care (MIMIC-IV) database. Using prompts such as “Simplify this radiology report” and “I am a patient. Simplify this radiology report,” they evaluated the performance of the LLMs in simplifying report impressions.
The study revealed that compared to the original reports, all four LLMs significantly improved the readability of the report impressions for each prompt used. However, certain models demonstrated better performance than others based on the specific prompt. Additionally, prompts providing additional context, such as “I am a patient” or “simplify at a 7th-grade level,” resulted in better performance across all LLMs.
It’s important to note that the researchers did not endorse any particular LLM, as each model has its strengths and weaknesses. They emphasized the importance of careful fine-tuning and customization for each LLM to ensure optimal simplification while maintaining the clinical integrity of the reports. The authors recommended further longitudinal studies with diverse datasets to enhance the validity and generalizability of these findings.
In an accompanying editorial, Dr. Amir Ali Rahsepar of Northwestern University highlighted the necessity for careful development and oversight of LLMs in healthcare. He stressed the importance of incorporating expert medical advice during the development process to ensure reliability and patient-centered outputs. Dr. Rahsepar emphasized the need for large-scale, diverse studies to identify areas requiring LLM fine-tuning and to establish specific guidance for generating accurate, simplified, and empathetic reports, ultimately improving healthcare outcomes.
For those interested in further exploring advancements in mammography and AI, consider checking out the BIE mammography boot camp with AI online 2024 for additional insights and training opportunities.
