Learning from Your Feedback on the IR/DR Certifying Oral Exam
By James B. Spies, MD, MPH, ABR Associate Executive Director for Interventional Radiology
In May, we successfully completed the first regular remote administration of the IR/DR Certifying Oral Exam, and we are now reviewing the feedback from participants. One of the benefits of our current electronic format for the oral exam is the timeliness of post-exam surveys, which were sent to candidates and examiners at the conclusion of the exam. We had a very good response rate and received many constructive comments and suggestions.
Most comments from candidates about the software platform and exam content were positive. Many commented on the benefit of not having to travel and the associated cost and time savings. The most common concern of respondents was a lack of feedback from the examiners. We train examiners to focus on ascertaining the candidate’s knowledge about the cases presented and to avoid confirming or praising candidate responses. This has always been the case in the traditional in-person oral exam, but it seems the effect on the candidates was magnified by the virtual nature of the current exam. Some candidates were unsettled by what they felt was a one-sided discussion, and they were sometimes unsure if they were answering the question asked.
In our orientation of the candidates, we tell them that the examiner will not mislead or trick them but may not confirm their discussion is correct. Examiners will likely prompt the candidate if needed but will not necessarily lead them to the correct answer. With the broad input we received, we learned that some candidates felt they were unable to fully demonstrate their knowledge without engaging in more of a dialogue with the examiner. We will adjust our examiner training to encourage better engagement with candidates and work to make this a more comfortable experience for them.
From the examiner’s perspective, the logistics of the exam were more complicated, as it required entering two platforms for each section, while the candidate stayed in one virtual WebEx room. The primary examiner having difficulty logging in to the correct WebEx room was the most common reason a secondary examiner had to take over. Still, more than 95% of exams were completed by the primary examiner. Regarding the exam itself, many examiners commented that some candidates were not prepared to discuss the range of IR procedures. This is perhaps understandable, as the standard path is for candidates to take the exam after a minimum of one year in practice, and many have been out from training for several years. It is natural to form your view of the “range” of IR based on your own experience.
However, the IR/DR certifying process must determine competence over the range of clinical conditions and interventions that interventionalists may encounter. In practice, one may not do peripheral arterial interventions, TIPS, thoracic duct embolization, or any number of other procedures, but that does not excuse a lack of at least a working knowledge of the indications, technical aspects, patient management, and complications from a procedure. The knowledge level for the oral exam is not the “walking-around knowledge” standard that typifies the Continuing Certification OLA process. The expectations for an initial certifying exam such as this are higher. We provide guidance on the topics that might be covered in the certifying exam on the ABR website. We urge candidates to take the time to review those areas they are less familiar with, as they are likely to be part of the exam.
We continue to learn from candidate and examiner input, and we will continue to update and refine examiner and candidate orientations and training sessions to improve what we believe is an already very strong process. Our goal remains to provide a fair process to assess the competency of our candidates for certification, while doing all we can to reduce the stress of the experience.