14
05 '19
By ABR Trustee Donald J. Flemming, MD, and David J. Laszakovits, MBA
Delivering our exams at local commercial testing centers has been a topic of conversation for many years. We understand the discontent about added cost for travel and stress of being away from families and training programs. We take this very seriously and are working hard to find more manageable solutions.
To that end, we want to share a little about what we’re trying to accomplish, the unique features of the exams, and the obstacles we’ve faced attempting to secure a commercial testing vendor to deliver our diagnostic radiology (DR) initial certification exams.
At the most fundamental level, any high-stakes testing organization must achieve several basic exam delivery elements to ensure that our initial certification processes meet best practice standards. Perhaps the most important standard is that the exam be “proctored and secure.”
Exams used to evaluate professional performance, such as the SAT, MCAT, and USMLE, usually require the testing environment to be proctored and secured, which ensures the overall fidelity of the exam administration. Prominent commercial testing vendors can’t ensure basic elements we need (e.g., lighting and monitor calibration) to deliver exams.
In addition to this basic necessity, delivering an exam to assess the clinical competencies required to practice DR presents a number of other challenges. Because of the nature of practice, the most appropriate way to present DR exam material is through clinical vignettes that have a patient history, a question to be answered, multimedia content, and answer options. The multimedia content usually includes several high-resolution images or a video file.
These delivery requirements have proven to be insurmountable obstacles for the testing vendors that we’ve engaged over the years. The majority of these vendors’ clients deliver text-based question exams with little or no multimedia content.
For almost a decade, we’ve sought the services of numerous national commercial testing vendors for the delivery of our exams. We’ve been successful in securing a vendor for delivering our radiation oncology and medical physics exams solely because they are text-based with little or no multimedia content.
Unfortunately, we’ve not been able to do the same for our DR exams; however, that is not for lack of trying. We have approached two prominent commercial testing vendors to explore our goal of delivering the DR initial certification exams at testing centers. Neither was able to provide a proposal that matched our needs.
We are committed to making the initial certification process as effortless as possible. While our past efforts have not been successful, we will continue to pursue everyone’s goal of delivering DR exams in local commercial testing centers. Technology is constantly evolving, and perhaps local exam delivery will become more feasible in the future.
07
05 '19
In the next few weeks, we will be posting our annual report, a publication that covers notable achievements by our organization and diplomates in 2018. From exam results over the past decade to accomplishments by our volunteers and staff in serving diplomates from the field’s four disciplines, readers will find numerous items of interest regarding ABR activities.
Because we know our diplomates are busy, this year’s format is similar to last year’s more streamlined guide. Just like last year’s report, you won’t find long messages in this guide. We carefully considered what needed to be included and are presenting the information in a clear and succinct way.
We appreciate the assistance we received from several of our diplomates who contributed thoughts and material for the guide. We also are grateful for the cooperation we received from Tucson-based Radiology Ltd., which allowed us take photos of diplomates at work that we are featuring throughout the annual report.
The guide will be available on our website toward the end of the month or the beginning of June. We will send notification through our Twitter and Facebook pages, and on myABR when it’s posted, so please be on the lookout.
We are proud to have served our diplomates and their patients since 1934. Our boards of governors and trustees, and staff look forward to another successful year serving the profession.
30
04 '19
by Valerie P. Jackson, MD
Online Longitudinal Assessment launched for diagnostic radiology and diagnostic radiology subspecialty diplomates this past January. Four months after implementation of the system, ABR Executive Director Valerie Jackson took time to answer a few questions about OLA, the American Board of Medical Specialties’ Vision for the Future Commission, and ABR exams in general.
OLA has been active for four months. What are some of the emerging trends?
More than 16,000 DR diplomates and subspecialties, almost 90 percent of the total enrolled in MOC, have answered at least one question in OLA. More than 380,000 questions have been answered, and more than 46 percent of participants have answered at least 20 questions. At that pace, this group will complete its yearly requirement of 52 questions by July. Of course, diplomates may complete up to 104 questions per year if desired.
Every OLA participant has the opportunity to participate as an OLA “rater,” which helps to set the passing standard for each question. Almost one-third of participants have been involved in the rating process.
Diplomates may add comments to each OLA question, and to date we have received more than 10,000 comments. Each comment is reviewed so that we can improve our processes. Many comments have been very positive about the OLA experience; others have been extremely helpful in alerting us to potential problems with an item. Feedback is provided directly to volunteers who create the content for OLA.
We are very pleased with the level of engagement of our diplomates with OLA and look forward to rolling this out to interventional radiology, radiation oncology, and medical physics in January, 2020.
We have received more than 10,000 comments from diplomates using OLA. Upon which suggestions is the ABR looking to act?
Based on feedback from our diplomates, the ABR is working on a number of enhancements for OLA.
Online Longitudinal Assessment launched for diagnostic radiology and diagnostic radiology subspecialty diplomates this past January. Four months after implementation of the system, ABR Executive Director Valerie Jackson took time to answer a few questions about OLA, the American Board of Medical Specialties’ Vision for the Future Commission, and ABR exams in general.
OLA has been active for four months. What are some of the emerging trends?
More than 16,000 DR diplomates and subspecialties, almost 90 percent of the total enrolled in MOC, have answered at least one question in OLA. More than 380,000 questions have been answered, and more than 46 percent of participants have answered at least 20 questions. At that pace, this group will complete its yearly requirement of 52 questions by July. Of course, diplomates may complete up to 104 questions per year if desired.
Every OLA participant has the opportunity to participate as an OLA “rater,” which helps to set the passing standard for each question. Almost one-third of participants have been involved in the rating process.
Diplomates may add comments to each OLA question, and to date we have received more than 10,000 comments. Each comment is reviewed so that we can improve our processes. Many comments have been very positive about the OLA experience; others have been extremely helpful in alerting us to potential problems with an item. Feedback is provided directly to volunteers who create the content for OLA.
We are very pleased with the level of engagement of our diplomates with OLA and look forward to rolling this out to interventional radiology, radiation oncology, and medical physics in January, 2020.
We have received more than 10,000 comments from diplomates using OLA. Upon which suggestions is the ABR looking to act?
Based on feedback from our diplomates, the ABR is working on a number of enhancements for OLA.
- We are assessing the amount of time required to answer an individual OLA question. While the vast majority of diplomates successfully answer both one- and three-minute questions in less than one minute, there have been comments about wanting more time. The time limits were set up based upon our experience with our exams and the fact that OLA is meant to assess “walking-around knowledge” that diplomates should know “off the top of their head.” The board will be reviewing all data and comments about timing for the questions to decide if changes are necessary.
- The board is evaluating the feasibility of adding a clinical practice area for emergency radiology.
- We are developing a mechanism to count OLA participation as part of ABR’s Self-Assessment CME (SA-CME) requirement.
- Eventually, we plan to allow diplomates to customize their OLA notification experience, letting them change the frequency of notifications of new questions. However, this will require extensive software development, so in the meantime, we will continue to send weekly reminders. Each question has a four-week shelf life and there is no requirement to answer questions every week.
