Pandemic Causes Tough Exam Administration Decisions
by Paul E. Wallner, DO, FASTRO, Associate Executive Director for Radiation Oncology, and David Laszakovits, MBA, Director of External Relations
2020;13(4)4
Initial certification (IC) in radiation oncology (RO) requires that residents or candidates pass four exams: three computer-based qualifying and one final oral certifying. Computer-based exams in medical physics for RO and radiation and cancer biology may be taken after 32 months of residency training, and the computer-based exam in clinical (general and radiation) oncology may be taken after 48 months of residency training. The final step in attaining initial certification, the oral certifying exam, may be taken the year after residency is completed if the three qualifying exams have been passed. Traditionally, oral exams have been administered in early May, and the computer-based exams in early July.
In early March 2020, it became evident that the novel coronavirus (COVID-19) was spreading globally, and on March 11, the World Health Organization (WHO) officially designated the outbreak as a global pandemic.1,2 The ABR and other ABMS Member Boards were faced with a series of decisions that would have significant impact on the organizations, their staff and volunteers, and, critically, residents and candidates in the process of initial certification.3
The process of developing ABR RO IC assessment instruments and delivery logistics occurs over a period of several years. Contracts for the Pearson VUE (PV) test centers used to administer the IC qualifying exams in physics and biology are executed a minimum of two years in advance of each administration, as are contracts for the hotels used for administering the certifying exam. During the summer and fall of the year preceding exam administration, computer-based items are submitted by groups of volunteers and edited by ABR staff for clarity, consistency, and psychometric validity. Clinical cases for the certifying exam are submitted by committee members, reviewed by internal ABR editing and imaging staff, and then developed into the appropriate clinical case presentation format. Invitations to examiners for each certifying exam are sent at least six months in advance of the May administrations. In the late winter or early spring of each year, volunteers meet at the ABR Tucson office for a test assembly (TA), during which each of the more than 400 items intended for use in the upcoming July qualifying exams is reviewed, re-edited for clarity, discarded when appropriate, and replaced as needed. A final Angoff score is then assigned to each item. This intensive TA process typically takes two to three days of volunteer and staff time.
During the first week of March, as reports of COVID-19 cases in the United States increased, the ABR began to receive notification of flight disruptions and prohibitions on nonessential travel by volunteers’ home institutions. As the number of committed oral examiners declined, it became evident that it would be impossible to muster a complement of examiners to administer the May oral exam to more than 200 registered candidates. In addition, the ABR anticipated that candidates might also have difficulty traveling to Tucson to take the exam. On March 5, with mounting evidence of health risks and logistical challenges, and a full understanding of the inherent implications of the decision, the ABR decided to postpone administration of the May RO oral exam. Candidates who had registered for the May exam were notified as were volunteers impacted by the decision.
The 2020 RO TA was scheduled to take place in Tucson on March 15-16. Volunteers who had been prevented from travel in May were also prohibited from travel to this function. Thus, the ABR decided to change the onsite Tucson RO TA to a remote meeting. Working with dedicated ABR IT and exam development staff, the TA was carried out in its entirety by a secure webinar over two days, during which each of the more than 400 items was reviewed and scored in a manner similar to the onsite process it replaced. On April 13, with a continued increase in incidence and deaths related to the virus, increasing reports of disruption in training programs, and the presumed unavailability of the PV test centers, the ABR decided to postpone the July qualifying exam administrations. That decision was immediately transmitted to exam registrants and program directors.
After the decision had been made to postpone the May oral and July computer-based exams, ABR staff and volunteer leaders were faced with determining how and when to reschedule the exams and clarifying the status of residents and candidates who had registered for those postponed sessions. The option of administering both the computer-based and oral exams by commercially available remote, web-based platforms was considered, but in consultation with PV, these options were determined to be unfeasible at the time. A plan to administer the oral exam in Tucson between October 11 and 13, 2020, was announced. Rescheduling the computer-based exams presented a more difficult logistical challenge because of the ABR’s reliance on PV test centers, which had been closed from March through June. Discussions with PV indicated that when their centers did reopen, they would be following social distancing procedures, necessitating reduction in seating capacity. The Board was ultimately able to obtain a commitment from PV for rescheduling of the exam centers. On May 18, candidates were notified that the medical physics for radiation oncology and radiation and cancer biology exams would be administered on Monday, December 7, and the clinical oncology exam would be administered on Tuesday, December 8. The ABR is aware that the revised dates may be problematic for some individuals previously registered for the May or July exam administrations. Those unable to attend the rescheduled exams will be able to register for the 2021 sessions without additional fees. All ABMS Member Boards with spring and summer exam schedules were faced with similar concerns and decisions, some of which were driven by closures of PV or Prometric, the two commercial test administration centers used by many boards for their computer-based exams.
As additional details regarding the spread of the COVID-19 virus and associated travel restrictions and health concerns became evident, ABR Governors and staff became more concerned about the ability of the ABR to successfully administer the RO oral certifying exam on the anticipated dates and in a safe and reliable manner. Thus, on June 22, the Board announced its decision to cancel the 2020 certifying exam administrations and to transition future oral exams from a face-to-face experience to one based on a remote, disseminated platform. The dates and logistics of the postponed exam, anticipated for the first half of 2021, will be announced as soon as they become available. Candidates will be informed when the dates are finalized.
ABR senior physician leaders and staff recognize the personal, family, career, and program implications of the decisions they make regarding IC exam administration. On May 28, the ABR convened a webinar with appropriate RO stakeholders to consider future exam scheduling and alternative exam delivery methods. As in the past, decisions will be based on supporting the best interests of stakeholders while at the same time, preserving the value and credibility of the IC process.
References
- World Health Organization. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/technical-guidance/naming-the-coronavirus-disease-(covid-2019)-and-the-virus-that-causes-it Availability verified March 19, 2020.
- World Health Organization Director General’s remarks, March 11, 2020. https://www.who.int/dg/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19—11-march-2020 Availability verified March 19, 2020.
- American Board of Medical Specialties COVID-19 statement. https://www.abms.org/news-events/abms-statement-to-dios-regarding-coronavirus-disease-2019-covid-19/ Availability verified March 22, 2020
