A natural language processing algorithm, designed to scour emergency department records for fracture cases, has the potential to improve treatment of osteoporosis and prevent future, more severe fractures.
The approach led to a notable increase in referrals to the osteoporosis refracture prevention service at the Prince of Wales Hospital in Sydney, where the work was done.
The strongest predictor of a future fracture is a recent previous fracture, said Christopher White, MBBS, the hospital’s director of research, who presented results of an analysis at a virtual news conference held by the Endocrine Society. The study was slated for presentation during ENDO 2020, the society’s annual meeting, which was canceled because of the COVID-19 pandemic.
“We have really effective therapies that can reduce the risk of [future] fractures by 50%, and yet 80% of osteoporotic patients leave the hospital untreated after fracture,” said Dr. White.
That, he explained, is because of a fundamental disconnect in fracture care – emergency department physicians tackle the immediate aftermath of a broken bone, but they are not tasked with treating the underlying condition. As a result, many patients who would be candidates for follow-up care are not referred.
The current work grew out of Dr. White’s frustration with not being able to recruit patients for osteoporosis clinical trials. In fact, he got so annoyed trying to recruit and not getting patients referred to him – even though he’d find they were actually in the hospital – that he decided “to start an AI [artificial intelligence] program that would read the radiology report and bypass the referrer,” he said.
To that end, with the help of an industry partner, he developed a software program called XRAIT (X-Ray Artificial Intelligence Tool), which analyzed the reports and, with Dr. White’s iterated guidance, learned to identify fractures.
The system performed a little too well. “You have to be careful what you wish for, because suddenly I went from 70 referrals to 339,” he said.
That influx is a potential downside, however, according to Angela Cheung, MD, PhD, director of the Centre of Excellence in Skeletal Health Assessment and Osteoporosis Program at the University of Toronto’s University Health Network. Natural language processing can help identify patients that a human reviewer would miss, because reviewers tend to focus on cases in which the fracture was the reason for the hospital visit, rather than on incidental findings. But not all incidental findings are clinically important. “A pneumonia patient might have had the fracture 30 years ago, falling off a tree as a college student. It may not pick up the highest-risk group in terms of fractures, because we know that recency of fractures matters,” Dr. Cheung, who was not associated with the research, said in an interview.
“It means the fracture liaison coordinator would need to review [more] numbers in trying to figure out whether the patient should get attention and whether they should be treated as well,” said Dr. Cheung, adding that more studies would need to be done to determine if the approach would be cost effective.
The researchers performed a technical evaluation of 2,445 nonfracture and 433 fracture reports, in which the tool performed with more than 99% sensitivity and specificity.
In a clinical validation, a fracture clinician and XRAIT reviewed 5,089 x-ray and computed tomography reports from ED patients who were older than 50 years. The ED referred 70 cases, leading to identification of 65 fractures. The combination of ED referral and a fracture clinician’s review of 224 cases revealed 98 fracture cases. By contrast, XRAIT nearly instantaneously analyzed 5,089 reports from 3,217 patients, and identified fractures in 349 patients – a nearly fivefold higher number than the manual case finding of 70. Of those 349 patients, results for 10 were false positives, leading to a total find of 339 patients.
In all, 57 cases were found both by XRAIT and the ED referral/fracture clinician, resulting in 282 unique cases identified by XRAIT alone. That translated to a 3.5-fold increase in cases that were identifiable using XRAIT.
In an external validation, the researchers tested the system on 327 reports from a subset of the Dubbo Osteoporosis Epidemiology Study, based in the city of Dubbo in New South Wales, Australia. In that cohort, XRAIT identified 97 positive cases, of which 87 were true fractures (10 false positives). Of 230 cases that it considered not to be fractures, there were 38 false negatives. Those numbers translated to a sensitivity of 69.6% and a specificity of 95.0%.
All of those hits have the potential to overwhelm osteoporosis services. “I now have to adjust to that, and further development will be to link the AI with clinical risk factors and treatment data to assist my fracture coordinators to target the right patients. We’ll increase the number of patients with osteoporosis on treatment, improve productivity and safety, and reduce the burden of care,” said Dr. White.
The study was funded by The Sydney Partnership for Health, Education, Research and Enterprise and the Musculoskeletal Consumer Advisory Group. The researchers reported no financial conflicts of interest, as did Dr. Cheung.
The research will be published in a special supplemental issue of the Journal of the Endocrine Society. In addition to a series of news conferences on March 30-31, the society will host ENDO Online 2020 during June 8-22, which will present programming for clinicians and researchers.