Successful Oral Exam Development Required Teamwork, Quick Action
by Paul E. Wallner, DO, ABR Associate Executive Director for Radiation Oncology; Lydia Malis, ABR Associate Director of Exam Services; and Anthony Gerdeman, PhD, ABR Director of Exam Services
2021;14(3):8
ABR radiation oncology trustees had been considering alternatives to in-room, hotel-based certifying (oral) exams for some time, but when the May 2020 exams had to be postponed, it became apparent that a solution enabling remote exam administration was essential expeditiously. What might otherwise have been a several-year development process had to be completed within a timeline that permitted the 2020 eligible exam cohort to maintain a pathway to initial certification generally compatible with previous schedules.
The primary goal of exam development was to create a platform that provided the best possible experience for candidates while maintaining necessary content security and examiner satisfaction. Teams comprising ABR staff and volunteers were formed to focus on developing the examiner and candidate software, essential hardware specifications, content requirements and development, user interface, and a potential variety of exam scheduling iterations because of time zone considerations. To accomplish these tasks in the most efficient and expeditious manner, teams began their tasks independently but were in constant communication and cross-fertilization through almost daily videoconference calls. Team leadership conferred on a weekly basis to review progress, consider alternative solutions for defined problems, and test development at each step of the process.
As all aspects of exam development began to reach critical inflection points, staff and volunteers carried out dry runs to test development components. After each dry run, debriefing sessions identified elements that were in need of alteration. Those deemed problematic were assigned timelines for correction and subsequent testing.
In March 2021, all elements of RO exam development were in place to administer a pilot exam to 21 volunteer candidates. Although minor problems were identified during the two-day administration, all candidates were able to complete their exams satisfactorily. Post-pilot debriefing identified additional areas of improvement, some of which were completed immediately and were implemented for the May, large cohort administration. Some modifications will require longer-term correction but should be available for the September 2021 exam administration.
For updates on remote exam development and administration, please visit our website.
