Radiation Oncology Certifying Exam Development and Administration
By Paul E. Wallner, DO, ABR Associate Executive Director for Radiation Oncology; and Brian J. Davis, MD, PhD, ABR Trustee
2022;15(2):8
The ABR strives to be transparent in exam development and administration procedures. Processes specific to radiation oncology (RO) exam development have been published.1-4 A recent RO Initial Certification (IC) Town Hall indicated significant knowledge regarding the qualifying exam (computer-based) parts, but some lack of understanding of the processes involved in developing the certifying (oral) exam. A manuscript detailing development and administration of the new remote exam platforms has been accepted for publication by the Journal of the American College of Radiology and should be available shortly, but that piece does not describe the creation of case material or examiner selection and candidate scheduling.
RO IC exam content is developed by eight clinical category committees. Each committee is divided into two teams: one responsible for submission and refinement of qualifying exam questions and the other tasked with submission and refinement of clinical case material for the certifying exams. The blueprint for exam weighting is created by the RO trustees, committee chairs, and ABR staff, based on disease incidence in the general population and involvement in management of that disease site or organ system by RO. Once submitted, cases are selected for the exam inventory, and image quality is reviewed by ABR staff. Text of clinical presentations is reviewed by senior volunteers and ABR editors for clarity and to remove extraneous material. The exam case material is then entered into appropriate sections of the remote exam platform in a distribution such that each exam session has different cases but similar topics.
Members of both exam development teams are eligible to serve as oral examiners. Invitations are made based on experience, geographic distribution, and numbers needed for a given set of exams. Examiners have extensive training by web-based presentations and manuals, and pre-exam category meetings review each case to maximize cross-examiner consistency. On exam days, each category convenes to review candidate responses, and then scoring is reviewed by multi-category panels. Exam sessions are recorded for training and quality assurance, but the recordings are deleted prior to the distribution of exam scores. Candidates can raise concerns about bias or examiner demeanor prior to the distribution of scores and may request clarification of their scoring at any time prior to the next semi-annual exam session.
The ABR currently anticipates that candidates will continue to sit for the certifying exam remotely, and that semi-annual administration will also continue. The decision to deliver both spring and fall oral exams was made to enable candidates with life-event constraints to have additional opportunities to take the exams, and to give candidates who have failed or conditioned exams to no longer be required to wait a full year for another opportunity to attain IC. Candidates have six years from completion of training to attain IC, so timing of their opportunities to take the certifying exam is at their discretion. Spaces for the fall oral exam administration will be limited, so candidates are encouraged to register for the spring exam.
RO trustees and staff continue to seek input from the IC Advisory Committee members and stakeholders for suggestions for improvement in exam administration.
- Yang, JC, Wallner, PE, Becker, GB, et al. Reliability of oral examinations: radiation oncology certifying examination. Pract Radiat Oncol. 2013;3(1):74-78.
- Wallner, PE, Gerdeman, A, Willis, JM, and Zietman, AL. The American Board of Radiology radiation oncology maintenance of certification part 3 modular examination: evaluation of the first examination. Pract Radiat Oncol. 2016;6(6):436-438.
- Wallner, PE, McGeagh, AM, Gerdeman, AM, et al. Snapshot of a specialty: results of the ABR 2016 radiation oncology clinical practice analysis. Jour Amer Coll Radiol. 2019;16(4):513-517.
- Wallner, PE, Frey, GD, Roberts, AC, et al. Gender diversity within the American Board of Radiology: history, current status, and future implications. Jour Amer Coll Radiol. 2019;16(12):1713-1715.
