When our Online Longitudinal Assessment was introduced in January, one of its most important features was an ability to collect observations from you, the people who use the system.

After answering each OLA question, you are presented with a feedback screen. While completing the questions is optional, we hope that you take a brief moment to answer. The data from your responses is used to help create the passing standard for OLA by allowing you to provide feedback about the question that you just answered.

Confidence Rating

The feedback provided for the confidence rating question is solely for you. When you review your answered OLA questions in the My OLA History section, you can sort your data using the confidence rating. For example, if you wanted to see the questions that you answered correctly, but indicated you were not confident in your response, you can do that using the My OLA History filtering tools. Remember this is a tool to help aid in learning and continuous improvement.

Relevance Rating

The feedback that you provide for the relevance rating question is used to help inform the volunteer committee members who write and vet OLA questions. It will be used by the volunteers to make improvements to the question and inform them how to provide better OLA content. In some circumstances, relevance data may even help us identify problem questions that need to be removed from the OLA system. In the event this happens, the question would not be counted in your OLA performance.

OLA Passing Standard Rating

The feedback that you provide for this question is used to create the passing standard for the question. OLA uses a criterion-referenced passing standard, which is set based on the answer that you and other OLA participants provide on this question. For more information about the standard setting process, click on the “Refresh Training” link and check out Scoring and Performance Evaluation FAQs. If you haven’t completed the online training to become an OLA question rater, we’d also encourage you to do that. The training can be completed in approximately five minutes online.

Open-text Feedback

This area is your chance to provide whatever feedback you would like about the question you just answered. We appreciate any comments regarding the OLA question that you would like to provide. Comments submitted here go to the volunteer committee members writing the OLA questions as well as to ABR staff to help us improve OLA.

A continuous assessment tool, OLA was established as a better alternative to the 10-year MOC exam. While providing feedback on OLA questions is a quick and easy process, the positive effects it has on the system are important and long-lasting. We encourage you to help us and your peers by rating questions and providing feedback. It’s the best way to ensure that OLA remains fair and relevant for everyone.