Months of Development Led to Successful Oral Exam Administrations
by Brent Wagner, MD, MBA, ABR Executive Director
2021;14(3):3
Last month, the ABR hosted hundreds of candidates for oral board certification exams in medical physics, interventional radiology, and radiation oncology. The result of almost a year of design and iterative development, the exam interface combined standard video conference software with an interactive platform developed by the ABR. This created an exam experience and scoring model as similar as possible to the previous in-person oral exam without the risk or inconvenience of travel, lost time from work, etc.
ABR staff at multiple levels were critical to the execution of the process, which involved as many as 100 candidates in a single day, connected with examiners from around the country. After content development by subject matter experts, individually and in committees, exam development staff incorporated the cases into a standard format. Testing and modification addressed the need for examiners to have an intuitive interface to proceed through the cases with the candidates, while participating in a rigorous scoring methodology that would achieve the highest level of fairness.
As we mapped out the workflow and the potential impact of transitions between exam sessions, we decided to introduce a “navigator” model that would require only one check-in for the candidates, followed by a series of one-on-one sessions with examiners over the course of the exam day. Because this was a novel process, in a high-stakes environment, the primary role of the navigators was to reduce stress for the candidates by facilitating the exchanges and providing direct IT support. With navigators working from a command center in the ABR office in Tucson, with multiple levels of oversight, we were able to successfully administer nearly all scheduled exam sessions.
While pleased with the success of the new oral exam platform, the staff recognized the learning opportunities from the experience. Both formal and informal conversations have been directed toward making the process better for candidates and examiners. Much of our approach has been targeted to support ongoing remote administrations, and the Board remains cautiously optimistic we will be able to continue to use and refine this model. As in 2021, all computer-based and oral exams for 2022 will be remote.
