On this page
- DR Oral Exam Reenactment (November 27, 2024)
- New Exam Builds on Previous Model with Scoring Rubrics (November 4, 2024)
- Exam Development Update (May 9, 2024)
- Differences in Qualifying (Core) and Certifying Exams (May 3, 2024)
- First-year Resident Discusses the Exam (April 17, 2024)
- Oral Exam Update (November 14, 2023)
- Addressing Questions About the New Exam (June 28, 2023)
- Town Hall Recording Available (May 11, 2023)
- Transition to New DR Oral Exam (April 13, 2023)
- New DR Oral Exam FAQs
- Evaluating Potential Solutions (October 2022)
- Call for Stakeholder Comments (May 2022)
2024
November 27
DR Oral Exam Reenactment
ABR volunteers Desiree Morgan, MD, and Derek Sun, MD, run through a simulated DR oral exam. Dr. Morgan and ABR Associate Executive Director for Diagnostic Radiology Mimi Newell, MD, then take part in a mock panel discussion. We also have posted oral exam sample cases on our website.
November 4
New Exam Builds on Previous Model with Scoring Rubrics
The development of the new DR Oral Exam model benefited from iterative contributions of a wide range of external stakeholders, and the subsequent decision to return to an oral exam format was largely focused on a specific goal: to create an exam that assesses the higher order skills that are part of clinical practice (in contrast to knowledge assessable on a multiple-choice exam).1
In communications related to the change, the ABR compared the 2028 model with the legacy ABR oral exam that existed for decades through 2012. In conveying that the 2028 version will be different from the old DR oral exam, we have unintentionally created the impression for some faculty and candidates that we have not retained some of the attributes and principles of the legacy exam.
The previous and upcoming models both represent composite assessments based on cases presented by and discussed with several examiners during a series of 25-minute one-on-one sessions covering the different subspecialty content areas of radiology. Candidates analyze cases based on their observations of the imaging features and available history and summarize their findings in the form of a reasonable differential diagnosis before suggesting next steps (e.g., additional imaging, urgent referral for consideration of surgery, etc.). The analysis by the candidate resembles a succinct summary that might be part of a multidisciplinary conference, a concise but complete report, or a phone conversation with the treating physician. The examiner might ask clarifying questions or redirect the candidate to the major findings or alternative diagnostic possibilities. Each candidate is assessed by the specific group (panel) of examiners who discussed the cases with them. In the panel discussions, scores for individual sections may be raised if low performance in a particular subject area is considered an outlier.
The most obvious difference in the new exam is that it will be administered remotely using a one-on-one videoconference format. Less apparent to the candidate will be our efforts to mitigate subjectivity and bias in the exam. For example, cases will be given in the same order for each session, and each category will have an identical case set for a given oral exam date. As a result, all candidates examined on a Tuesday will see very similar content (although the number of cases might vary, depending on how quickly the examiner and the candidate can complete a case and move to the next).
Examiners in the old model used a general scoring grid that included three broad categories: observation (identifying the abnormality and pertinent negatives), synthesis (differential diagnosis, including the most likely diagnosis), and management (for example, additional imaging or urgent referral). The new model will build on these by defining specific elements (a rubric) to be used by all examiners as part of the electronic score sheet for each case.
For example, a conventional set of abdominal radiographs in a young adult with vomiting might demonstrate distended loops of gas-filled small bowel. A pertinent negative would be the absence of free air. A second finding would be a subtle lucency overlying the right inguinal region. Additional testing would be a CT scan with intravenous contrast. The major CT finding would be a right inguinal hernia containing thickened ileum with diminished enhancement. The overall management (urgent surgical consultation) would be based on a presumptive diagnosis of ischemic bowel within an incarcerated inguinal hernia. More detailed examples of performance rubrics are being prepared for the seven content areas by the ABR’s subject matter experts in diagnostic radiology and are scheduled to be posted on the ABR’s website by November 25.
The major goal of the new exam model is to assess the knowledge and skill of diagnostic radiologists. To that end, residents should prepare for the exam in a way that allows them to identify and appropriately communicate the presence and nature of imaging findings, the potential significance of those findings, and reasonable recommendations for next steps. The use of defined rubrics for each case will enhance standardization of objective scoring of the discussions.
- Larson DB, Flemming DJ, Barr RM, Canon CL, Morgan DE. Redesign of the American Board of Radiology Diagnostic Radiology Certifying Examination. Am J Roentgenology 2023;221(5). https://www.ajronline.org/doi/10.2214/AJR.23.29585
May 9
Exam Development Update
May 3
Differences in Qualifying (Core) and Certifying Exams
The Diagnostic Radiology Qualifying (Core) Exam will continue to assess the full scope of the discipline with an organ system approach and includes a robust examination of DR physics, radioisotope safety, and noninterpretive skills (NIS).
The new Diagnostic Radiology Oral Certifying Exam will include representative cases across seven categories* and will include basic diagnostic radiology procedures where appropriate. The candidate should anticipate discussing practical and clinically relevant physics and safety principles in the context of cases. The case discussions will assess the clinical, observation, communication, and professionalism skills required of an independently practicing diagnostic radiologist.
*Categories: Abdominal, Breast, Cardiothoracic, Musculoskeletal, Neuroradiology, Nuclear, and Pediatric.
April 17
First-year Resident Discusses the Exam
2023
November 14
Oral Exam Update
June 28
Addressing Questions About the New Exam
May 11
Town Hall Recording Available
April 13
Transition to New DR Oral Exam
Beginning in calendar year 2028 and first applying to DR residents completing training in June 2027 (entering their R1 year in July 2023), the certifying exam for DR will be the new DR Oral Exam. From that point on, a candidate’s first opportunity to take the new DR Oral Exam will be the calendar year following completion of their DR residency. All DR candidates taking the DR Certifying Exam after 2027 will be required to take the new DR Oral Exam regardless of when they completed residency training. We anticipate having two exam administrations per year.
The DR Qualifying (Core) Exam’s content, format, and timing will remain unchanged.
As part of continuing efforts to improve testing, particularly in light of technologic advancements implemented during the pandemic, the new DR Oral Exam will be an online exam taken in a location of the candidate’s choosing.
The new DR Oral Exam will include select critical findings as well as common and important diagnoses routinely encountered in general DR practice, focusing on examples that optimally assess observation skills, communication, judgment, and reasoning (application of knowledge learned during residency).
It is not meant to represent a comprehensive review of clinical content. The oral exam aims to assess higher-level skills that are needed to be an effective diagnostic radiologist and are valued by referring physicians and patients.
To mitigate subjectivity and potential bias inherent in an oral exam, examiners will use a standard set of cases, and detailed rubrics will be used to score each candidate. This is an improvement over the prior oral exam model and is facilitated by current technology, including software developed specifically for this purpose by the ABR and currently used for oral exams in the three other disciplines (interventional radiology, radiation oncology, and medical physics). As in the past, examiner panels will meet after each session to discuss candidate results to ensure fairness and consistency. A conditioned exam result will be possible. The panels will be balanced for geography, gender, and new vs experienced examiners.
For more detailed information about the new DR Oral Exam, please see our frequently asked questions (FAQs). We will be sharing additional information and guidance for exam preparation over the coming years.
-
+ New DR Oral Exam Frequently Asked Questions (FAQs)
In what ways will the new DR Oral Exam be the same as the prior DR oral exam?
The candidate will meet with one examiner at a time. Each session will use a standard set of cases for each exam category. The examiner will display case materials to the candidate and ask questions relative to the content being displayed.
How will this be different from the prior DR oral exam?
The new DR Oral Exam will be delivered remotely. The breadth of content will be similar to the prior oral exam. While the Qualifying (Core) Exam allows the candidate to demonstrate knowledge across the entire domain of diagnostic radiology, the new DR Oral Exam will focus on higher-level skills such as observation, synthesis, and management. Candidates will be evaluated on communication and their ability to incorporate clinical information into recommendations for management and additional imaging when appropriate.
What clinical categories will be reflected in the new DR Oral Exam?
We anticipate there will be seven individual exam sessions. The clinical categories are breast, chest (includes cardiac and thoracic), abdominal (includes gastrointestinal, genitourinary, and ultrasound), musculoskeletal, nuclear radiology, pediatrics, and neuroradiology. Diagnostic radiology procedures, professionalism, and safety will be incorporated into the clinical categories as appropriate.
How advanced will the cases be?
The content will include critical findings as well as common and important diagnoses routinely encountered in general practice.
What are the expected times for the exam for the first year and subsequent years?
We anticipate an exam administration in the winter/spring and another in the fall of each year. Each administration will likely consist of four or more one-day exam periods (each candidate will only need to set aside one day to take their exam). We’ll post exam dates on the ABR website approximately 18 months before the first exam.
What content should be expected in interventional radiology?
Because interventional radiology/diagnostic radiology (IR/DR) has a unique primary certificate and residency training pathway, performance of advanced interventional procedures (e.g., embolization, stent placement, thrombectomy, vertebroplasty, etc.) will not be covered in the new DR Oral Exam. However, knowledge of anticipated imaging findings and complications after interventional procedures will be part of the new DR Oral Exam.
In addition, DR procedures (e.g., image-guided biopsy, abscess drainages, arthrocentesis, lumbar puncture, etc.) will be covered, including indications, contraindications, techniques, anatomic considerations, and complications. Interventional radiology content will continue to be included on the computer-based Qualifying (Core) Exam.
Will there be physics or noninterpretive skills (NIS) content?
Professionalism and selected physics concepts that are directly related to safety will be incorporated as appropriate.
How will the ABR inform residency programs what will be covered so they can help prepare their residents?
The ABR will communicate with program directors regularly to ensure a smooth transition. In addition, exam preparation resources will be posted on the ABR website as exam development processes commence.
Will a study guide be provided?
Yes. A study guide will be posted on the ABR website well in advance of the first new DR Oral Exam administration.
What is the impact on an Early Specialization in Interventional Radiology (ESIR) resident?
This change will not impact ESIR residents in DR residency programs.
What is the impact on a DR resident who then matches into an Independent IR residency?
A DR resident matching into an Independent IR residency who opts to take the DR Certifying Exam after 2027 will take the new DR Oral Exam.
How will this impact a DR resident in the 16-month nuclear radiology training pathway?
DR residents in a 16-month nuclear radiology training pathway taking a DR Certifying Exam after 2027 will take the new DR Oral Exam.
Will this have any impact on the DR Qualifying (Core) Exam?
No. There will be no changes to the Qualifying (Core) Exam content or timing.
Will this have any effect on the timing of subspecialty exams?
No. This change will not impact subspecialty exam timing.
Will the criteria for who can sit for the new DR Oral Exam stay the same or change?
Eligibility criteria will not change. Candidates must successfully complete residency training, pass the Qualifying (Core) Exam, and pay applicable exam fees.
How many times can I take the new DR Oral Exam?
A candidate’s first oral exam opportunity will be the calendar year following the completion of residency training. Candidates are eligible to take the exam through the end of their board eligibility, which extends six full calendar years after the completion of residency training. We anticipate having two administrations of the new DR Oral Exam each year.
What can you share about the evaluation and scoring process?
We will share information about the evaluation and scoring process at least one year prior to the first exam administration.
How will conditioning work?
We anticipate that candidates failing two or fewer clinical categories will receive an exam result of “conditioned.” Candidates with a conditioned exam result are required to repeat only the clinical categories failed at the previous administration.
Will there be examiner panels that can “raise” a weak score if the candidate does well in all other clinical categories?
Yes. This has been a long-standing part of the oral exam process. Examiners review the performance of each candidate at the end of each day to confirm the final exam result. All examiners must agree on the overall result for each candidate.
What efforts will be made to mitigate bias and subjectivity in the assessment of the candidates?
For each session and category, the examiner will use a standard set of cases for all candidates. Detailed rubrics will be used to score each candidate. This is an improvement over the prior oral exam model and is facilitated by current technology, including software developed specifically for this purpose by the ABR.
As in the past, examiner panels will meet after each session to discuss candidate results to ensure fairness and consistency. The panels will be balanced for geography, gender, and new vs experienced examiners.
What if the candidate has a conflict of interest with the examiner?
Candidates and examiners are provided the opportunity to identify and report conflicts on the day of the exam prior to the first session. Conflicts are resolved before the start of the exam. If neither party identifies a conflict prior to the exam, but realizes that one exists as they greet one another, the exam period will be rescheduled with a different examiner.
What if the candidate has a complaint about an examiner?
Candidates should contact the ABR immediately following the completion of their exam if they have concerns about their exam administration.
What if I have failed or not taken the computer-based DR Certifying Exam by 2028? What exam will I take?
Beginning in 2028, all diagnostic radiology candidates will be required to take and pass the new DR Oral Exam to earn certification, regardless of when they completed residency training. We anticipate being able to provide two computer-based DR Certifying Exam administrations in 2027 to help with the transition.
What if I can’t take the new exam during the time it is offered?
We anticipate having two administrations each year to allow flexibility.
What happens if I don’t pass before my board eligibility expires?
This policy is unchanged. Candidates failing to successfully complete the initial certification process within the board eligibility period are no longer considered by the ABR to be board eligible and are no longer permitted to designate themselves as such for communications or credentialing purposes. Candidates wishing to regain board eligible status must complete an additional year of training.
What are the computer requirements, camera requirements, bandwidth requirements, etc. for the exam?
The technical requirements for candidates are not extensive. Details regarding the technical requirements will be provided well in advance of the first exam administration.
What are my options for selecting a remote exam location?
This will be an online exam taken in a location of the candidate’s choosing. Details regarding the location requirements will be provided well in advance of the first exam administration.
Will the candidate be able to adjust (magnify, contrast, etc.) the images in real time?
Yes. The exam software allows a candidate to modify an exam image in real time during the exam. By default, candidates have this capability at the start of every case.
Will the exam have scrollable stacks?
Yes. There will be scrolling stacks and other types of multimedia images. We’ll share more details on the type of imaging well in advance of the first exam.
What happens if technical issues interfere with the exam?
Technical issues can happen during any remote exam administration. For oral exams, the schedule allows additional time to complete periods that were unfinished due to technical issues on the part of the candidate or examiner. ABR staff do everything possible to ensure that each candidate can complete the exam on the scheduled date. More than 99% of candidates in ABR oral exams in medical physics, interventional radiology, and radiation oncology have completed their exam on the scheduled day.
Prior to every remote exam, candidates have an opportunity to test their computer system with the same software that is used in the exam. This allows for potential issues to be identified and addressed well ahead of exam day.
Are exam accommodations available under the Americans with Disabilities Act (ADA)?
Yes. The ABR offers ADA and lactating and expectant parent exam accommodations for all exams.
How will this affect the implementation of “mini fellowships” in the R4 year?
Training during the R4 year is determined by the residency program director and continues to be handled at their discretion.
What is the cost for the exam?
No change in cost is anticipated as a result of implementing the new oral exam. The exam fees are available here.
How will the exam be tested or piloted before the first administration to make sure it works well?
We anticipate using our existing oral exam software, which is in use for other ABR disciplines and has been thoroughly tested. We do not expect new features will be needed for this exam.
Will there be normals?
We have not yet determined in what way normals might be included.
When will the ABR begin recruiting examiners for the new DR Oral Exam?
We will begin accepting volunteer applications for DR oral examiners in July 2026. Visit the Diagnostic Radiology Volunteer Page for more information.
-
+ Evaluating Potential Solutions (October 2022)
In October 2022, a project planning group comprising members of the ABR Board began meeting weekly to evaluate potential solutions and to develop an exam proposal based on information learned during our call for stakeholder comments in May 2022. The planning group met regularly with a larger working group of Diagnostic Radiology Trustees and was supported by ABR staff and Jennings Healthcare Marketing.
Key stakeholder organizations representing the interests of residents, program directors, department chairs, private practice, and community radiology groups were contacted and asked to provide representatives who could participate in longitudinal feedback facilitated by Jennings. These representatives formed a single stakeholder group that convened on multiple occasions in November 2022 to discuss options for exam content, timing, and structure. Relying on this valuable input, the project planning group continued to meet through the end of 2022 and early 2023 to develop and refine options. The process culminated in a proposal for exam changes including a new oral certifying exam that was provisionally approved by the ABR Board of Governors at the February 2023 board meeting.
-
+ Call for Stakeholder Comments (May 2022)
In May 2022, the American Board of Radiology (ABR) began seeking stakeholder input and insights into the Diagnostic Radiology (DR) Initial Certification (IC) process. This review included both the Qualifying (Core) and Certifying Exams.
The comprehensive stakeholder input process was facilitated by our third-party partner, Jennings Healthcare Marketing. The process focused on gathering feedback and perspectives through multiple channels as the precursor to considering possible process modifications. Stakeholder input included three primary sources:
Targeted Stakeholder Organization Input. Given the large number of affiliated organizations in radiology, we focused on several key groups representing residents, program directors, department chairs, private practice and community radiology perspectives. Jennings helped facilitate multiple structured feedback sessions with these organizations’ executive leadership. In addition, the targeted organizations were encouraged to seek input from their constituency. Organizations were asked to submit a written synopsis of their feedback to the ABR as part of the process.
Other Stakeholder Organization Input. Recognizing the value of input from as many perspectives as possible, we also invited any interested organization to submit a written synopsis of their feedback to the ABR. We encouraged organizations to seek input from their constituency in their written comments. Recommended discussion points were provided for interested organizations.
Individual Stakeholder Input. We also provided the opportunity for individual input and feedback through the publicly available Diagnostic Radiology Initial Certification Feedback Survey, which was distributed through the ABR website, The Beam e-newsletter, and social media. The survey was available May 25-August 31, 2022.
We appreciate the engagement and thoughtful discussions we had with so many people during this time of focused stakeholder input, including the response to the individual stakeholder input survey.
Stakeholder Input Summary
Overview:
Most stakeholders share the viewpoint that the current DR IC process does not accurately measure what radiologists do. Stakeholders reported that:
- The current DR IC process needs improvement.
- Multiple-choice questions (MCQs) do not optimally assess the detection, interpretation, and communication skills of trainees.
- MCQs can adequately (though not optimally) assess knowledge; many reported that MCQs alone are not a sufficient measurement of knowledge, particularly in the DR Certifying Exam.
- The DR Certifying Exam occurs too long after graduation from residency.
What has been going well:
- Stakeholders noted the value of offering remote exams and recommended continuing this moving forward.
- Stakeholders praised the ABR’s increased transparency and solicitation of stakeholder input.
- Stakeholders believe image-rich exams are beneficial since they get closer to testing what they are intending to test.
- Stakeholders noted that test questions are vetted, though some reported that a few seem antiquated.
What has not been going well:- Some stakeholders reported that the exam is moving away from testing general knowledge toward testing more specialized knowledge, which is in line with the direction the profession is moving.
- Some stakeholders noted that this transition is biased in favor of larger programs.
- Others noted that this move toward specialization is less useful for smaller or more rural practices, where it is more helpful to have broad knowledge across the multiple subspeciality areas that constitute the broad domain of diagnostic radiology.
- Many stakeholders reported that the certification process is a good measure of the requisite knowledge required to be a competent practicing radiologist, but it is a poor measure of a radiologist’s clinical competence as it pertains to interpretation skills, communication skills, and professionalism.
- Stakeholders reported that preparation for a multiple-choice exam leads to trainees answering as many test questions as possible in lieu of taking cases as a study and learning mechanism, which results in fragmented learning, limited critical-thinking, and practice deficits.
For more information, please contact us at information@theabr.org or 520-790-2900.