Scorable Questions Necessary to Determine OLA Performance

Scorable Questions Necessary to Determine OLA Performance

We have written about the importance of having Online Longitudinal Assessment (OLA) participants serve as question raters. One of OLA’s unique features is that diplomates set the passing standard for questions in their specialties.


Having as many participants as possible rating questions is crucial to OLA’s fairness and relevancy. These ratings help ensure we have the necessary data on each OLA question to validate that it is performing at an acceptable level to be considered a “scorable question.” A question is scorable when it has been answered by 50 or more diplomates, has been rated by 10 or more diplomates, and has acceptable psychometric statistics.


As of earlier this month, 36 percent of participants had rated questions. We appreciate everyone who has served as a rater and invite those who haven’t started to do so. Participation as a question rater is vital to OLA’s success as well as the overall fairness and relevance of the platform.

CategoryBlogs, News

Help spread the word
© 2023 American Board of Radiology    |   
Privacy & Legal    |   
Americans with Disabilities Act (ADA) Site Map
The American Board of Radiology does not and shall not discriminate based on race, color, religion (creed), gender, gender expression, age, national origin (ancestry), disability, marital status, sexual orientation, or military status in any of its activities or operations. These activities include but are not limited to hiring and firing of staff, selection of volunteers, conducting committee meetings, and administering exams. We are committed to an environment free from discrimination, sexual harassment, and other unlawful forms of harassment. To report any actions of discrimination, sexual harassment, or other unlawful harassment, please contact Karyn Howard, Managing Director, at 520-790-2900 extension 2171 or you can call our confidential hotline at 844-280-0005.
Version: 3.0.58
The American Board of Radiology