New Dashboard Provides Insight into Performance
2020;13(6):11
In October, diplomates participating in Online Longitudinal Assessment (OLA) may have noticed a new dashboard after logging in. This long-awaited addition has provided diplomates with a frequently updated status on their current OLA performance. The OLA performance dashboard displays diplomates’ current cumulative score as well as their annual performance evaluation.
Current Cumulative Score
The current cumulative score reflects how a participant is performing against their unique passing standard. This score becomes available after a participant has answered 52 scorable questions and is updated every Monday from that point forward. The current cumulative score does not impact MOC Part 3 status.
The passing standard for each OLA question is established individually by diplomates who volunteer as question raters. OLA is a criterion-referenced assessment and a participant’s score is not impacted by others’ performances.
Annual Performance Evaluation
The annual performance evaluation section shows how a participant is performing after they have answered 200 scorable questions. The annual performance evaluation does impact MOC Part 3 status.
The first evaluation occurs on January 1 after a participant has answered 200 total scorable questions. After that, performance evaluations take place annually and include the participant’s most recent 200 scorable questions. The first annual performance evaluations will take place January 1, 2022.
How are Diplomates Performing?
More than 95% of diplomates have a cumulative score that meets or exceeds their OLA passing standard. These are preliminary data, since many diplomates have been participating in OLA for less than a year. Since there are tens of thousands of OLA participants with different unique sets of questions, passing standards can vary widely. As of October 2020, the range of passing standards is 50% to 83%. Check out our blog for more details on how cumulative scores are established.
Additional OLA Statistics
Since OLA launched in January 2019, more than 30,000 diplomates have answered at least one OLA question, and more than 3 million questions have been answered overall. Diplomates don’t seem to be having issues with the question time limits: the average response time for one-minute questions is 26 seconds and the average response time for three-minute questions is 43 seconds.
Based on participant feedback, the overall question relevance rating is 4.08 out of 5. In addition, less than three percent of all questions have been declined. Given the wide diversity of practice patterns and subspecialization, these are especially promising statistics and a testament to the extraordinary efforts of the many volunteers who write questions.
