ABR Executive Director: DR Oral Certifying Exam Planning Well Underway

The start of a new year is a good point to reflect on the recent past and look to the future. ABR Executive Director Brent Wagner, MD, MBA, recently took time to address questions about a few of the organization’s accomplishments and opportunities.

The new Diagnostic Oral Certifying Exam is coming in a couple of years. What are the ABR’s priorities for 2026 as it continues preparing to give the exam?

Two parallel group efforts are underway. First, our IT team is working to enhance the platform and user interface for case development and presentation that incorporates many display functions that are part of a standard PACS system. Second, volunteer subject matter experts are actively working on case development to craft an exam that will be a relevant and broad sampling of the domain for each category and include standardized content for enhanced objectivity and fairness.

In 2029, interventional radiology candidates will take both the IR and DR certifying exams. How vital was stakeholder feedback in making that decision and setting the timing for the exams?

In addressing this issue, external stakeholder input was considered above everything else. The ABR governing board (and the psychometric staff) fully supported the fundamental reasoning: the two certificates both include “diagnostic radiology,” thus the exams should be equivalent to avoid the possibility of a perceived imbalance or reduction in the value of the certificate. We were not surprised to learn that the same rationale was consistently part of the position expressed by radiologists, including those who are recent residency graduates. The timing of the first opportunity was also intended to coincide with timing for the DR candidates (at the end of the fourth year of residency) to enhance the fairness and relevance of the process in a combined cohort.

What persuaded the ABR to continue giving remote oral Initial Certification exams after the pandemic?

Although the move to remote exam administration in 2021 was in direct response to circumstances arising from the pandemic, ABR staff and board members have adopted the organizational mindset to continue the model indefinitely. The obvious advantages to the candidate — avoiding the costs and inconvenience of travel — support the continuation of remote administration. Unless concerns arise (for example, related to security, software development costs, or psychometric reliability), we intend to continue to refine and continue the process for the examinees.

Online Longitudinal Assessment has been in effect since 2019. In what ways does the organization continue to refine and improve the platform?

Many of the improvements are not obvious because they are either incremental small changes, improvements in hidden infrastructure functions, or both. For example, we use a set of automated algorithms that quickly remove flawed content (e.g., a question with more than one acceptable answer) and send it back for review by a volunteer subject matter expert. Such adjustments to the content pool are designed to err on the side of the diplomate, so there is no score penalty, and questions are extracted for deletion or modification.

AI is constantly in the news. In what ways does it impact the ABR?

The AI landscape is evolving very rapidly, and, like many organizations, the ABR is still finding its way in this space. We are actively working with internal and external stakeholders to thoughtfully consider ways in which we could use AI tools within a sound and ethical governance framework. This includes the recent retention of an outside firm for a brief engagement to facilitate those discussions and, ultimately, allow us to consider innovative ways to more effectively and more efficiently assess the knowledge and skill of our candidates and diplomates. For the foreseeable future, despite understandable interest from many of our volunteers, copyright and validity concerns preclude use of generative AI to write exam questions.

The ABR recently updated guidelines to help its volunteers avoid conflicts of interest. What was the impetus for providing that guidance?

The fundamental principle is that individuals who choose to serve as volunteers may not be engaged in activities that purport to “teach to the test” (sometimes called “board prep”). One of our stated obligations to our candidates for certification is that the exams will be fair, so that everyone has the same opportunity to demonstrate their competence in their discipline, and no candidate will indirectly gain an advantage from a volunteer’s participation in the test development process. We trust our volunteers to keep internal elements of process and all exam content, whether in draft, committee review, or test assembly phase, in strict confidence. However, we understand that most ABR volunteers have teaching responsibilities as part of their faculty roles, and many lecture to national or international audiences. Our guidelines are intended to address perceived conflicts that arise when a diplomate is interested in not only serving as an ABR volunteer but also engaging in parallel activities as a subject matter expert in their field. After the 2023 announcement that we would return to the oral exam model for diagnostic radiology, we received questions from current volunteers regarding what was permissible as they attempt to balance their ABR efforts with the educational functions in their departments. Recent revisions have also relaxed prior guidelines (addressing the increasing frequency of shared faculty via technology platforms), but we must remind volunteers to separate their activities as a volunteer in design and administration of an exam from what they might do, in their role in their institution, to teach the elements of sound and effective clinical practice.