It is time for interventional radiologists to leave behind the “cowboy culture” that was the foundation of the discipline’s early, exploratory days, and to become “legionaries marching in service of data”, Ziv Haskal (University of Virginia, Charlottesville, USA) argued in his 2021 Dotter Lecture, delivered during the Annual Scientific Meeting of the Society of Interventional Radiology (SIR; 20–26 March, online).
“IR knowledge is not atomic: it is complex,” the former Journal of Vascular and Interventional Radiology (JVIR) editor told virtual attendees. “You cannot live with ‘I read the abstract’ or ‘I cruised the conclusion’”, he joked, cautioning SIR delegates against accepting certain ideas as gospel without first scrutinising the data. Indeed, Haskal’s lecture, “The reluctant cowboy: Moving past myth and dogma” deconstructed commonly held interventional radiology (IR) “myths” and dogmatic approaches to practice, and called for a more rigorous culture of data and research.
“We need to be accountable and adopt a sceptical approach that can only be satisfied through repetition and replication,” he said.
In order to do this, he advised his audience to act at various levels: “On a personal level: embrace this constant self-query, self-reflection, scepticism, and patience. Read in detail. Embrace the dialectic.
“On a local level: cheer on developing prospective research; it is five to ten times more effort to write protocols, but they are durable, and impactful, and can be built upon, or will lead to other work.
“On a national level: support initiatives for data that can be congregated, through registries, through standardised reporting, through NIH [National Institute of Healthcare] efforts. Open your pocketbooks with patronage; lend your financial support to societal initiatives.
“And on the big blue planet, think about the large-scale questions, so we can get together with other groups. We have to mine the facts for truth and work daily to be clinical scientists.”
Alan Matsumoto (University of Virginia, Charlottesville, USA), who delivered the 2019 Dotter Lecture, presented the Dotter Lecture award to Haskal on 22 March.
The apodictic versus the apoplectic: Moving beyond a narrow, dogmatic vision in IR
He began his talk by noting that “many or most of us [interventional radiologists] remember that signal case” that led them to pursue a career in IR, which he likens to skydiving due to the “adrenaline thrill”. However, he warned that now is the time for the speciality to turn towards “creating data and our own science”.
Haskal made the distinction between “the apodictic—knowledge we are certain of” and the apoplectic, the “you will do it this way because I was trained this way, and therefore you will be trained this way” school of thought.
He highlighted some specific examples of “apoplectic” thinking “to highlight the types of narrow vision that we have on a daily, or on a field-wise basis”. One such belief he questioned was the idea that pulmonary arteriovenous malformations (PAVMs) less than 3mm do not need to be embolized to prevent stroke. “Are bacteria just too big to fit through a 3mm PAVM?” Haskal asked rhetorically. He traced the origins of this belief to an abstract presented orally at the 1992 meeting of the Radiological Society of North America (RSNA), which did not state that there was no need to embolize smaller than 3mm, but rather found clinically-evident stroke in four patients with feeding artery measuring 2.9–4.5mm. “That is pretty shaky ground for decision-making,” Haskal opined.
“How about gall bladder catheters—they have to be transhepatic, right?” he continued. He cited a 1988 article in the American Journal of Roentgenology (AJR) and a 1994 paper in JVIR that refute this, demonstrating the safety of a transperitoneal route. “If that was not enough, I was recently asked by a plaintiff’s lawyer to be a medical expert on a case in which there was a hepatic bleed, and she insisted the interventional radiologist was negligent because they had gone through the liver, and there was a bleed.”
He also addressed common practice assumptions. “Why do we always get called at night for nephrostomies or arterial lysis?” he asked. But according to a review of cases conducted at the 800-bed University of Virginia Hospital over the course of one year, 84% of nephrostomies and 80% of lysis procedures took place between 8am and 8pm.
According to Haskal, this is due to frequency bias, also known as the Baader-Meinhof effect—after you notice that you have done several night-time lysis procedures, you will notice it more often and believe that it has a high recurrence.
The risks of skimming
In addition to these commonly-held IR “myths”, Haskal also warned his audience of the dangers of “reading light”. Presenting an example, he discussed an “excellently-written, multicentre trial, [conducted] under the auspices of the FDA [the US Food and Drug Administration]”. He published the study, which was an approval trial for a vena cava filter, when JVIR editor. “Here are the conclusions from the abstract,” he read: “The rate of freedom from new symptomatic PE [pulmonary embolism] through 60 days was 100% (n=129, confidence interval [CI] 97.1–100%). [….] There were no cases of PE through 12 months for either therapeutic or prophylactic indications.”
Commenting on this, he said, “So that is pretty good, we want that device,” before adding, “But here is what happens when you read inside [the paper].” A third of the patients were purely prophylactic, “that means they had no DVT [deep vein thrombosis] or PE to begin with”, Haskal said. “That is like taking people with cancer and people who might get cancer, and following them for a year, and then saying ‘Look, we did not have cancer [amongst the total cohort]’. But you put those people in! [Referring to the cohort without any cancer at the start of the study].”
“Papers are this long for a reasons—we have got to read the detail,” he emphasised.
“Three out of four meta-analyses are garbage in, garbage out, cannot be trusted”
“Surely meta-analyses are the path to truth?” Haskal posited. “It is the analysis of the analyses, it is integration and power, it is rigour over narrative, it is signal over noise, it is the superman!” he enthused.
However, he showed a meta-analysis that had to pool 95 trials to show that neutropenic cancer patients have lower mortality with antibiotics, saying “even large, individual trials were not enough to pull it together [and reach this conclusion]. That is a life-altering signal.
“But guess what? In one analysis of 7,200 systemic reviews, once you take out the Cochrane analyses, one quarter, 27%, had low risk of bias, compared to 87% of Cochrane analyses. That means that three out of four meta-analyses are garbage in, garbage out, cannot be trusted.
“You have to stay vigilant as a reader,” he advised, counselling his listeners to have a mental checklist when reading papers:
- Study question or hypothesis is clearly defined prior to analysis
- Study design has been defined before analysis and search is clearly defined
- Clear inclusion and exclusion criteria
- Methodological quality of the studies has been checked before analysis by a predefined quality standard (and by more than one reviewer)
- Publication bias is checked (for example by funnel plot)
- Heterogeneity has been properly investigated (graphically or statistically)
- A common effect size is used (directly quoted or recalculated) for pooled result
- Adjustment for potential confounders has been considered (mainly for observational studies)
- Results are clearly reported and properly interpreted
- Are statistically significant results also clinically significant?
“We must train scientific methods into the next generation of interventional radiologists”
To stray from the IR echo chamber, Haskal noted, interventional radiologists have to have scientific flexibility.
“The perfect gold standard is not possible in clinical research,” he said. Iteration, reiteration, renewal, and replication is the best that can be done. “We have to have ongoing scepticism and constrain exuberance,” he said.
He closed his lecture with a call to arms: “We must train scientific methods into the next generation of interventional radiologists, and we have to standardise it.”
Writing in JVIR in 2016, Haskal put: “We have new trainees now entering the specialty; we have five years of opportunity to inculcate them with methodical training and research exposure—to methods, critical reading, analysis and study design”.
He continued: “This will require a fundamental and intentional mutation in IT training DNA—potentially a centralised CRISPR insert available to all residencies. We cannot expect that each programme will have the ability to teach this, we need to develop this nationally, and make this available to our trainees [in the USA], and across the planet, because this is our long-game.”
IR and COVID-19: “When diagnostic volumes cratered, […] our volumes went up”
Speaking more broadly of the hardships SIR members and attendees have experienced in the past year due to the ongoing COVID-19 pandemic, and in particular noting how this has affected the SIR meeting, Haskal commented: “On a professional level, 25 years of the Dotter Lecture means sitting in that giant hall, surrounded by thousands of our colleagues, the lights go down, an extraordinary speaker comes out, and something revealing, and visionary, pulls us all together. Instead, we are all separated, watching this on screens, alone.
“But I want to remind everybody what we have done as interventional radiologists this year. Pulling protocols across the planet, trying to figure out how to manage in this plague year; doing all those procedures—PD catheters at bedside, we had not done that before, dialysis catheters, gastrostomy catheters in acute or chronically-ill patients, arterial lysis for hypercoagulability or pulmonary embolism. When diagnostic volumes cratered and people were furloughed and elective surgeries were cut, whether you were in a big metropolitan area or in smaller hospitals hit by COVID patients, our volumes were maintained or went up.”
“We are always going to have the professor Dotter spirit,” Haskal said in his closing remarks. “But we have to embrace the next evolution and work daily to be clinical scientists.”
