We have written about the importance of having Online Longitudinal Assessment (OLA) participants serve as question raters. One of OLA’s unique features is that diplomates set the passing standard for questions in their specialties.

Having as many participants as possible rating questions is crucial to OLA’s fairness and relevancy. These ratings help ensure we have the necessary data on each OLA question to validate that it is performing at an acceptable level to be considered a “scorable question.” A question is scorable when it has been answered by 50 or more diplomates, has been rated by 10 or more diplomates, and has acceptable psychometric statistics.

As of earlier this month, 36 percent of participants had rated questions. We appreciate everyone who has served as a rater and invite those who haven’t started to do so. Participation as a question rater is vital to OLA’s success as well as the overall fairness and relevance of the platform.