Radiation Oncology Remote Oral Exam Development
by Paul E. Wallner, DO, ABR Associate Executive Director for Radiation Oncology; Brian J. Davis, MD, PhD, ABR Trustee; and Anthony Gerdeman, PhD, ABR Director of Exam Services
2021;14(1):10
From the 1970s through 2018, the certifying (oral) exams in radiation oncology were administered in a relatively similar manner each year. The Executive West Hotel in Louisville, which changed its name to the Crowne Plaza Louisville Airport Kentucky Expo Center, served as the site for the exam during nearly all of this period. With the move to Tucson in 2018, the venue was different, but exam administration proceeded as it had for decades. With the restrictions necessitated by the international COVID-19 pandemic, however, previous standard operating procedures became moot.
Immediately upon cancellation of the May 2020 certifying exams, the Board set aside dates in late July for rescheduled exam administrations. Public health and safety restrictions and institutional travel bans for volunteers and candidates soon made that plan untenable. Working alongside colleagues in interventional radiology/diagnostic radiology and medical physics, the Board launched an “all hands” initiative to develop a remote oral exam delivery platform for rollout in 2021. In the absence of completely acceptable commercial, off-the-shelf products, much of the development was carried out in-house by ABR IT and exam development staff, trustees, and an extraordinary group of motivated volunteers.
Exam development proceeded along several parallel but constantly intersecting paths. As the software for exam delivery was being produced by the IT team, other groups were working on exam logistics and examiner and candidate experience issues such as scheduling, hardware specifications, item inventories, and requirements for category and panel meetings. Constant meetings of the task-specific teams and larger group kept all involved aware of developmental progress and questions. Innumerable issues crossed team responsibilities, such as session scoring: the look and utility of score sheets, how they would be routed through the exams, category and panel session access, and then on to staff for final tabulation and performance distribution. Other teams worked on developing examiner instruction manuals, orientation sets for web sessions, examiner software training, and candidate orientation tools.
To thoroughly vet the new oral exam platform, a full dry run will be conducted this month. The dry run will involve staff, trustees, and volunteer diplomates as monitors, examiners, and candidates to test the functionality of the new software. In March, pilot exams will be administered by seasoned examiners to candidates seeking board certification. Although the sessions will be limited in number and participating candidates will be volunteers selected by lottery, all aspects of the exam, including scoring, will be real. Because the candidates have volunteered to participate in this final stage of exam development, they will have no risk for the exams: if they pass, they will become certified in radiation oncology. If they condition or fail, they will be able to take the exams again without prejudice, cost, or an adverse record. In May, the entire cohort of candidates who were eligible for the postponed May 2020 administrations will be able to take the new oral remote platform if they have chosen to do so. A second full exam administration will be available in September 2021 for 2021-eligible candidates.
Throughout development of the new exam platform, a primary focus has been on maintaining an acceptable candidate experience while retaining exam security, validity and credibility. In that regard, candidates and examiners will be able to see each other on split screens and to communicate freely. Back-up examiners will be available in the event of an internet disruption, and ABR “navigators” will be present to assist with issues that might arise. Two administrations of both the qualifying (computer-based) and certifying (oral) exams will be available in 2021 to accommodate all candidates who were disadvantaged by pandemic disruptions, but in 2022, the Board will resume single annual administrations of each of the exams.
