OLA Performance Feedback and Customization Preferences
by Geoffrey Ibbott, PhD, ABR Associate Executive Director for Medical Physics, and J. Anthony Seibert, PhD, ABR Governor
Online Longitudinal Assessment (OLA) continues to evolve, and as the end of the first year of participation approaches for medical physics diplomates, many participants will discover that they have received feedback in terms of a current cumulative score relative to the “passing standard” on their OLA dashboards.
For a given certificate, individual performance feedback occurs after 52 scorable questions have been answered over the course of the year. (Fifty-two questions are needed to meet the annual progress requirement.) A question is deemed “scorable” when at least 50 diplomates have responded to it, a minimum of 10 question ratings has been determined by diplomates who have volunteered to be question raters, and the question has acceptable psychometric statistics. Once the threshold number of scorable questions has been met, a score appears on the current cumulative score dashboard and indicates how the diplomate has performed relative to their unique passing standard. It is updated every week, and as more questions become scorable, the results will be plotted as a function of time. A snapshot of the dashboard running graph is shown below.
The running graph will include up to 200 of a diplomate’s most recent scorable questions. It is important to note that the current cumulative score does not affect the status of the Part 3 MOC requirement. The annual performance evaluation, however, which occurs on January 1 each year for the most recently answered 200 scorable questions, will eventually affect Part 3 status. If a diplomate answers the minimum requirement of 52 questions per year, a performance evaluation will be some time in the future (January 1, 2024, for those who started in January 2020). On the other hand, if a diplomate answers the maximum number of questions per year (104), the first evaluation would occur within a two-year period. Thus, there is flexibility in how one can engage in the evaluation process.
If a diplomate is below the passing standard on January 1 of the evaluation year, there is still time to raise the performance before the MOC annual review on March 2. If the diplomate still falls below their passing standard at that time, their Part 3 status will change to “Fail.” However, if at any time during the evaluation year the passing standard is exceeded by answering more questions, the status will immediately change from “Fail” to “Pass.”
For diplomates who do not wish to participate in continuous certification or who are unable to remediate a failing Part 3 OLA evaluation, an alternative is to take and pass a traditional 5-year exam. The traditional exam for medical physics is the discipline-specific Part 3 (Oral) Exam. Frequently asked questions regarding OLA performance and alternatives are readily available by clicking on the “Performance FAQ” button on the OLA dashboard.
Diplomates who would like to stop receiving emails from the ABR whenever new questions are available or only get emails when questions are expiring can set preferences for email notification by clicking on the “OLA Preferences” tab on the dashboard:
An additional option is to choose how the questions are presented on the browser, with a choice of a Dark Theme (black background and light lettering) or a Light Theme (white background and dark lettering).
Depending on the room environment or display characteristics, one theme or the other might be better suited for more easily visualizing the content of the questions presented.
The process and presentation of OLA continues to evolve and improve to best serve every ABR diplomate involved in demonstrating their continuous commitment to lifelong learning.