OLA Review Process Improved Through Diplomate Comments
by N. Reed Dunnick, MD, ABR Associate Executive Director for Diagnostic Radiology
2021;14(1):7
Certification by an American Board of Medical Specialties (ABMS) Member Board is an important sign of quality. It is often used by patients to find a physician and may be required by healthcare providers. As a diplomate’s medical practice evolves, however, their remote board certification becomes less pertinent.
In 2019, the ABR began Online Longitudinal Assessment (OLA) for diagnostic radiology (DR), which allows diplomates to satisfy their knowledge assessment for Continuing Certification (Maintenance of Certification, MOC) by answering questions online throughout the year. Two items relevant to the diplomate’s selected practice profile are sent each week and are available to be answered for four weeks. At least 52 questions must be answered each year and as many as 10 can be declined. Almost 20,000 DR diplomates have participated in the program thus far.
In addition to giving diplomates flexibility in how and when they answer questions, OLA also allows them the opportunity to comment on each item, providing direct feedback that the ABR uses to improve content. The associate executive director (AED) of the pertinent discipline (diagnostic radiology, interventional radiology, radiation oncology, or medical physics) reviews each of the comments and often recommends a change in the item. Some items need simple fixes while others are referred back to committee for review.
In diagnostic radiology, 36,419 comments have been submitted on 4,135 items. Many are complaints, but quite a few are compliments. I especially like those that say, “I learned from this.” Diplomate comments on OLA items are valued by the ABR as they help focus attention on potentially problematic items. Diplomates have found errors in sidedness, grammar, spelling, and references. They have also questioned whether an item tests “walking-around knowledge” and whether two choices might be correct answers.
Many comments have reflected concerns about technical issues, such as an item not having enough images or the images taking too long to load. In our practices, we have a full set of images and as much time as needed to interpret an exam. OLA items are written by a diagnostic radiologist and reviewed by a subspecialty committee to ensure that the item can be answered correctly with the images and information provided. In response to your comments about wanting more time to consider an item, the time allowed has been increased from one to three minutes for many of them.
There were also quite a few diplomates who report that they were interrupted when trying to answer the item. While these interruptions may not be completely avoidable, it takes only a few minutes to open and answer two items.
Although it is not easy to review all the comments, it helps the ABR improve the OLA program. We thank the diplomates for taking the time to provide feedback.
