ABR Methods and Motives Explained in Q&A with Executive Director
By Brent Wagner, MD, MBA, ABR Executive Director
2024;17(6):3
Comments and questions from ABR candidates and diplomates often reveal misperceptions about our processes and procedures. We continually learn from questions that come from external sources and modify or enhance our communications accordingly. Although most of these inquiries are related to the “how” and “what” of certification, some reflect a disconnect between the ABR’s motivations (the “why”) and the way the board’s construct and actions are perceived.
Why are initial certification qualifying exam questions so difficult?
We hear this frequently in our post-exam surveys. We do not design our exams to measure how close candidates are to “perfection.” Instead, we are attempting to establish competence by measuring knowledge of the discipline domain and the skills needed to practice safely and effectively. The Angoff rating system is used to estimate the difficulty of individual questions and, in aggregate, the passing threshold (“number of correct answers needed to pass”). This method supports the fairness of our process by adjusting for exams that have a higher proportion of difficult questions (because on these exams, fewer correct answers will be required to reach the passing threshold). The ABR Angoff committees comprise practitioners in private and academic practices, including directors of training programs. After item-writing committees create questions, and the forms (the set of questions that populate a particular exam) are organized for exam delivery, the Angoff committees rate the items to set the passing threshold.
Why do medical physics exams have lower pass rates than those for the physician disciplines (diagnostic radiology, interventional radiology, and radiation oncology)?
The passing threshold for a specific set of questions (“an exam”) is established for all four ABR disciplines by using the Angoff method. Compared with the physician disciplines, which are supported by decades-long accreditation and testing practices of the AAMC, ACGME, and NMLE, there is more inherent variability not only in training among different CAMPEP residencies but also in the graduate school study that precedes the residency. It is therefore not surprising that robust psychometric standards in the development and assembly of ABR exams would result in a broader range of performance on the qualifying exams in medical physics.
What is it like to be “in charge” of the ABR?
I really like this question because it allows me, as the executive director, to give appropriate credit to the Board of Governors who oversee development policy and strategy. Under the ABR bylaws, the executive director is “responsible to the Board of Governors through the President for the effective conduct of the affairs of the ABR. The Executive Director will be responsible for the implementation of the policies of the Board of Governors relative to the Corporation’s mission, goals and objectives.” Although I am grateful for the opportunity to serve the board, the ultimate authority rests with the volunteer board itself (my “bosses”), not with me.
What is the most challenging part of making decisions at the board?
Striking a balance in our certification processes so that they are not only sufficiently rigorous to be of value to the public but also reasonable in what we ask of our candidates and diplomates. We know that our diplomate population is exceptionally dedicated, extremely knowledgeable, and highly skilled. We seek to provide assessments and other tools, especially as part of Continuing Certification, that provide reassurance to patients, credentialling bodies, and members of the healthcare team that specific standards, as defined by the profession, have been met to support optimal care delivered by the radiologic professions.
Why doesn’t the ABR have an elected board that focuses on the diplomates?
The ABR’s mission is “to certify that our diplomates demonstrate the requisite knowledge, skill, and understanding of their disciplines to the benefit of patients.”
There is a foundational principle that independent governing boards serve the mission rather than a particular constituency. This concept also applies to individual board members and avoids the pitfall of representation of groups or interests. This means that most nonprofit boards are composed of members who are not elected by external stakeholders. Independence is essential to the board’s functions, as summarized in this post:
Mission is what distinguishes nonprofits from their for-profit cousins: Nonprofits have missions instead of owners or shareholders. While the prime directive for board members of for-profit organizations is to ensure the highest possible value for owners, by contrast, nonprofit board members’ prime directive is mission fulfillment. Board independence and board attention are of paramount importance in good nonprofit governance. The independence of the board is key because of the non-distribution constraint – nonprofits exist to serve the public interest, not to benefit owners or other private parties.1
Was the decision to return to the Diagnostic Radiology Oral Exam merely intended “to justify the ABR’s existence”?
Organizations evolve. When circumstances (or stakeholder understanding) change, we have a responsibility to adjust by changing the construct of assessment instruments. Nearly 20 years ago, as increased subspecialization represented a widely recognized trend, the idea to create an exam that would allow a “practice profile” seemed reasonable, especially when coupled with the “Core” (Qualifying) Exam that was, unlike its predecessor written exam, “image rich.”
Over the past four years, after informal conversations regarding the limitations of a multiple-choice format in replicating clinical practice, the ABR asked hundreds of external stakeholders for feedback and input on whether the assessment method should be modified. It would have been easier for ABR staff (including me) and our volunteers to continue the status quo. Instead, the ABR, under the direction of the board, embraced the opportunity and responsibility to build an exam that would test the application of knowledge and skill in ways that are inherently nuanced but closer to clinical practice.
Why do the Trustees and Governors who sit on the ABR board presume to set rules for diplomates when they do not understand the challenges of everyday practice?
Contrary to a widely held perception, nearly all the board members spend most of their time in a combination of clinical work, teaching, and leadership. While it is important for board members to grasp the “big picture” issues facing the profession as they relate to the complicated landscape of healthcare delivery, they also understand – from personal experience – that increasing demands of delivering quality patient care amidst current workforce shortages and regulatory requirements are difficult to sustain. Discussions among the board members take this into consideration, based on their own perspectives and what they hear from their colleagues, when making decisions about certification requirements.
- Harvard Law School Forum on Corporate Governance. Nonprofit Corporate Governance: The Board’s Role
