By Kristen McConnell, PhD
As the plane touched down in Tucson, I did not feel the fear or trepidation that many before me have experienced when flying into Arizona. Instead of taking an ABR exam, I was there to look at giant cacti and see what happens at the organization’s headquarters. I checked into the Courtyard Marriott Hotel, where I could almost feel the anxiety and stress of candidates from years past as they walked into an examiner’s room to answer oral exam questions.
The next morning, I woke up in need of coffee, so I went on a walk to a shop and passed the ABR building. It’s a big square structure, rather unassuming with the desert-appropriate cacti out front. I wouldn’t get to go inside for a few more days. I had taken time off work to come early, so I spent the next few days exploring what Tucson has to offer. I saw an oasis nestled amid the dry landscape at the Arizona-Sonora Desert Museum, ate a delicious Sonoran hot dog from a food truck, had the best noodles I’ve had in years, visited the Pima Air & Space Museum, toured the Titan Missile Museum, and saw so much beautiful desert landscape.
When it was finally the day to visit the ABR, I walked into a building that was not what I expected. It has a big, inviting reception desk with a large open staircase going up the side of the room. There is a cool hanging art piece down the middle of the staircase. Pictures of ABR board members line the walls, and a large conference room has huge windows that reveal the mountain landscape.
The ABR staff had put together a program for the next few days, and we didn’t waste any time diving in. We had presentations from certification services and exam services, were walked through a demo of the new online exam software, learned how scoring and results communication takes place, and found out how exam standards are set. It was eye-opening.
Preparing Exams is Hard Work
Having taught a few semesters of radiation physics, I have created many quizzes and exams. But what I saw at the ABR put all my work to shame. These exams are not something you cobble together a few days before the final exam while you eat dinner. The ABR takes this process very seriously. Each question goes through many filters to remove sex/gender language, ensure focus, remove acronyms, ensure linear delivery of information, scrutinize answer choices for anything confusing, and check grammar among many others. I can’t bear to open any of my old exams and see if what I made holds up to the ABR standard.
We also talked through what it takes to score an exam. This is a six-step process! In the first step, collect, responses are verified and collected. Then an initial scoring with a computerized routine is run. To examine question quality assurance, poor performing questions are identified by psychometric staff and reviewed by subject matter experts. Then, a secondary scoring is run to remove “poor performing” questions. Finally, the exam results and feedback are validated and transferred to myABR before results are released.
I think something everyone wonders is…”How does the ABR decide who passes and who fails?” This was interesting because generally I just know one way, which is to say everyone who gets a 70 or above passes. (I’m clearly not creative, and I didn’t know that much about the topic).
This is not how the ABR scores computer-based exams. They use a criterion-referenced standard setting process. It is decided how many questions a person must answer correctly to be classified as “minimally competent,” and the number becomes the cut score – the number at which you have demonstrated that you are a minimally qualified candidate and will pass the exam.
This means a couple things that are important to understand: (1) everyone can pass each individual ABR exam in a given year and (2) your peers’ performances do not affect your exam result. Standardized exams in your academic career likely used “norm-referenced” scoring, and your percentile (or how well you did compared to others) very much mattered to how well you did on that exam. But that is not the case with criterion-reference scoring.
So, study with your friends! Their success does not change your ability to pass this exam. You might ask yourself, “Who decides the cut score?” And the answer is the ABR volunteer subject matter experts for your profession. They review the exam ahead of time using the Angoff Method of deciding for each question whether a minimally qualified candidate will answer it correctly. ABR staff do not set this cut score – it is your mentors, your attendings, your professors, and your clinical physicists.
Shining Light on a ‘Black Box’
Admittedly, I came to this event a little skeptical of the ABR because there is a lot of noise in the field about how bad the process is or how the ABR is a black box or how it is so weird that you used to take oral exams in someone’s hotel room. But what I heard and learned at the ABR changed my mind. ABR staff said that people have called them “the guys in the ivory tower.” First, there are a lot of women who work there, and second, I didn’t see people who are unaware of the challenges of the real world.
I saw a dedicated group of professionals, many of whom have spent their careers ensuring initial certification exams are delivered and candidates are certified each year in a fair and unbiased process. I think it is easy to toss around “ivory tower” when you don’t have a face to associate with the ABR or a true understanding of the checks and balances put in place to maintain quality and fairness. We met about 15 staff members and you could see the passion and excitement each of them had for their jobs. More important, they candidly discussed the challenges they face – many of which are communication-based. Most important, they took notes, asked us questions about changes that would help us, and presented roadmap items they hope will improve the process.
The takeaway I had is that the ABR wants us all to pass and have a good experience. Achieving that requires communication, something that’s hard for physicists sometimes. We tend to suffer in silence, accepting our fate because maybe we are pretty sure we understand everything and that nothing can change.
But don’t do that! The ABR proved their commitment to communication and change at this meeting (and during COVID, remember they changed to remote oral exams!), but it is a two-way road. We need you to reach out to your ICAC committee members (there’s one for each specialty) so we can address your concerns, thoughts, challenges, and suggested changes in a forum with the ABR. You also may contact an ABR Certification Services Manager at firstname.lastname@example.org or (520) 790-2900.
Dr. McConnell has served on the ABR Medical Physics Initial Certification Advisory Committee (MP-ICAC) since July 2019. She’s a medical physicist at the Miami Cancer Research Institute.