Using Chat generative pre-trained transformer (GPT) to understand the public perception of interventional radiology (IR), Chloe Cross (Icahn School of Medicine at Mount Sinai, New York, USA) and colleagues presented new data which provide “exciting” insights into how and when IR is suggested for treatment for patients using artificial intelligence (AI).
Cross et al detailed their study titled ‘Settling turf wars: When does ChatGPT refer you to an interventional radiologist?’ at the Society of Interventional Radiology (SIR) annual scientific meeting (23–28 March, Salt Lake City, USA), in a session dedicated to emerging technologies within IR.
Developed by OpenAI, ChatGPT is a conversational AI-based chatbot which provides answers to questions or prompts. “Although it’s not specifically designed for medicine,” Cross explained, “many patients are using ChatGPT for medical decision-making—similar to how patients use WebMD. Our purpose was to understand public perception of IR and when IR treatments are offered.”
Cross and colleagues developed a standardised prompt that reflected a realistic patient query, adhering to a ‘I have a disease process, what type of doctors can treat this?’ formula. The researchers created a list of disease processes in various systems treated by IR which were selected from a literature review. Each prompt was repeated three times and then the number of times IR was suggested and an average rank was recorded.
Their results showed that ChatGPT mentioned IR in 51 (73.9%) of 69 prompts, with an average rank of 3.3. Of all the disease states, IR was consistently ranked first for stroke angiography and splenic artery aneurysm embolization. Cross offered that for the splenic artery, this result may be due to the level of literature published on this diagnosis in IR. “We are pretty much the only ones who are doing splenic artery aneurysms in the recent past, so you can see where this result is coming from. It speaks to the importance of having our name in new searches published in these realms.”
Their data, however, showed that IR was not suggested for the treatment of pulmonary embolism (PE) or image-guided bone biopsy, and was only suggested once for the treatment of benign prostatic hyperplasia. Interestingly, Cross added that IR was only mentioned once and ranked far behind other specialties for image-guided percutaneous ablation of renal carcinoma, a treatment frequently performed by interventional radiologists.
“It’s important to note that IR was cited for the treatment of many diseases, which is quite exciting. This may help to improve patient awareness of IR, especially for referrals,” said Cross. Yet, she stated that these data indicate a “need for more public outreach” concerning the disease processes such as PE where IR was not indicated.
One limitation of their study was that they do not know the “inner workings” of ChatGPT, Cross relayed, so any conclusions drawn may be speculative. There is also little understanding of how patients are interacting with the technology as it is still relatively new.
“New versions of these large language models are coming out constantly. We may see changes in the generative answers that we obtain from this research, however for now, ChatGPT can help us understand public perception of IR and we are being suggested for many diseases we are able to treat, so that is very exciting,” Cross told attendees. She concluded that future research should explore ChatGPT’s role in medicine in order to better understand how the patient pathway to IR can be expedited through this new technology.
