Deep-Learning Natural Language Processing Tool Identifies Radiology Findings to Reduce Risk of Delayed Care


Leading-edge artificial intelligence sorts through radiology reports to pull critical results that could be missed and alerts physicians to expedite next steps.


Each year, thousands of radiology reports are generated at hospitals. While primary results are shared with clinicians and patients, the full report can contain other critical yet unexpected findings. But without an efficient, standardized method of identifying and communicating these findings — often buried in clinician notes — patients are at increased risk of delayed diagnosis and treatment. At the beginning of 2020, Northwestern Medicine scientists set out to develop a solution to this nationwide healthcare issue, starting with lung and adrenal scans.

Tracking Unreported Findings

When clinicians order a test, such as a chest X-ray for suspected pneumonia, a radiologist reviews the images and adds the findings to the patient’s chart. Once the primary diagnosis is addressed, any additional findings might go unreported because they are often hidden in technical, nuanced language. With no existing scalable solution, Northwestern Medicine looked to a form of artificial intelligence (AI) called deep-learning natural language processing (NLP) to create a tool that sorts through thousands of patient notes; identifies critical findings, such as lung or adrenal nodules; and alerts physicians.

The first step in building this solution involved manually annotating and classifying thousands of radiology reports. Next, using the annotated reports, scientists trained the algorithm to look at word patterns to determine if a radiologist was describing a potential nodule. A critical part of the process was ensuring the NLP tool did not flag stable lung nodules, which required running multiple models to improve its learning and precision.

Sharing Results

Scientists, nurses, human resources, information services, performance improvement specialists, and patient and family advisor councils worked as a team to build the NLP tool into Epic electronic medical records, making it easy to integrate into existing workflows. Findings are presented to the patient’s physician within minutes of the tool’s review. Once an alert is issued, several options are available for next steps, including opening the SmartSet documentation and ordering system, establishing a reminder to follow up with the patient, or indicating the patient has been contacted.

Moving Beyond the Lung

In addition to lung nodules, the NLP algorithm is also being used to find adrenal nodules at Northwestern Medicine. In one case, the tool identified an adrenal nodule and alerted the ordering physician, who referred the patient to the Endocrinology Department, all within three days.
As the first hospital in the country to develop an AI that can flag critical results in radiology reports using clinician notes, Northwestern Medicine is actively training the algorithm on new areas (e.g., liver, ovarian and thyroid findings). In addition to pulling out findings that could otherwise go undetected, the algorithm expedites the process for determining next steps in care and allows for faster treatment, improving outcomes for many patients.