Exam Developers Roll with the Changes Caused by COVID
by Heather S. Hopkins, ABR Communications Coordinator
2020;13(6):10
ABR exam and Online Longitudinal Assessment (OLA) content is created by more than 600 volunteer radiologists, radiation oncologists, and physicists on numerous specialty-specific committees. Volunteers meet throughout the year to refine questions (“items”) and organize the material. Before the COVID-19 pandemic, each committee met annually in Tucson or Chicago, and committee chairs came to Tucson for annual test assembly meetings.
All of these meetings are organized and run by exam developers. These ABR staff oversee the life cycle of an item from initial submission through editing, image processing, review, revision, Angoff, and, finally, export from the database onto an exam. The ABR has 12 exam developers: eight who work on content for computer-based and oral exams and four who work with the OLA committees. Exam developers work in teams of two, and each team handles about a dozen committees.
Because of the pandemic, all committee meetings since March have been conducted remotely, with WebEx calls replacing in-person gatherings. For individual committee meetings, this did not impact the process very much. Exam developers still prepare the same documents and tools they would have used in a standard meeting. Some committees have chosen to break up their meetings into multiple shorter sessions; the rest complete the process in the same amount of time as they would have in person.
“It is important that these specific meetings are conducted as close as possible to the in-person meetings,” said Lydia Malis, associate director of exam services. “The integrity of our process cannot be jeopardized just because it is conducted remotely. We are committed to creating the same high-quality content that our candidates and diplomates expect and deserve.”
Nothing additional is required of volunteers to participate in the remote meetings. Because they regularly participate in WebEx calls, they are familiar with the process. One challenge for the radiation oncology and medical physics TA meetings was conducting the standard-setting process remotely. The exam developers found a way to collect responses within WebEx, and no additional calls were required.
However, lost in the remote process are the networking opportunities and social aspects of annual meetings, like having dinner together after a long day of exam-building. According to exam developer Casey Sankey, “These things build a strong foundation for interacting with each committee. We all miss the excitement of travel and meeting face-to-face in Chicago, and it seems to be generally agreed upon that it is a worthwhile and highly beneficial goal to return to eventually.”
In addition to conducting meetings remotely, the entire exam developer staff has been working from home since March. Lydia said the team adapted very quickly to remote work and have become proficient at using Microsoft Teams to stay in touch. In addition, flexible hours help developers with small children balance working from home with remote school, and the lack of a commute saves time.
“I can see my two boys before school and pick them up after. We get to do homework while we cook dinner. I no longer feel like I am rushing all the time,” said Anyalisa Samaniego.
On the other hand, they miss seeing their teammates and other ABR staff. “We have become a family in many different ways and not being able to interact with each other in person was an adjustment, especially since we are a very social group,” said Amaris Castellanos.
“I interact with the exam development team all the time now but rarely get to talk to anyone else,” Terri New said. “I miss the bigger ABR family.”
Lydia takes that into account when she holds staff meetings. A little levity at work helps keep the team going strong.
“It has been important to me to allow some socialization in meetings, to replace what they are missing in the office, and to remind them that we are all on the same team,” she said. “Laughter relieves the stress. I’m a big fan of laughing.”
Learning a little about everyone’s home life has also proven to be bonding.
“Everyone is human, and we’ve allowed each other a window into our lives,” Casey said. “For example, seeing a trustee’s cat walk across their screen during a meeting provided levity and a sense that we’re all in this together. Meeting volunteers’ children and pets onscreen is heartwarming and provides some joy during this difficult time.”
