Online Longitudinal Assessment (OLA)

Online Longitudinal Assessment (OLA)

Last verified on November 14, 2019
Online Longitudinal Assessment (OLA) is a convenient way to satisfy Part 3 of the Maintenance of Certification (MOC) program. It has started for all diagnostic radiology (DR) and diagnostic radiology subspecialty diplomates (neuroradiology, nuclear radiology, and pediatrics) and will begin for radiation oncology, medical physics, and interventional radiology diplomates in January, 2020.
We have made available several options to help diplomates as you start OLA:
Please whitelist the ABR domain (@theabr.org) in your email to receive important communication about this ongoing assessment. We are anticipating a high volume of questions during the initial weeks, so please be patient when you email (information@theabr.org) or contact us at (520) 790-2900.
We recommend using a desktop computer, laptop, or tablet when answering OLA questions. Because the assessment is image-driven, cell phones don’t provide optimal results.
 

Background

OLA is a progressive online assessment that will replace the previous proctored MOC Exam required every 10 years. OLA is aligned with the other three parts of MOC that remain the same—Part 1: state licensure; Part 2: CME and self-assessment; and Part 4: participation in practice quality improvement activities.  

Benefits of OLA

  • Flexibility in Participation – You choose when to answer the questions.
  • Immediate Feedback – You will know if you answer correctly or incorrectly, and what the correct answer is for each question. Brief discussion and reference information is also included.
  • No Travel Required – Questions are available for you to answer on your own time at work or at home with no travel required.
 

OLA Questions

The OLA questions are written and reviewed by your radiology peers who volunteer their time on a number of committees.  

OLA Process

OLA enables diplomates to demonstrate what they know. For diagnostic radiologists, questions will be tailored to a self-selected practice profile. The OLA dashboard will enable you to monitor your progress and performance. ABR board members and staff are committed to ensuring that OLA is reliable, reasonable, relevant, and meaningful. The ABR has a sustained responsibility to the public to certify that diplomates demonstrate the requisite knowledge, skill, and understanding of their disciplines to benefit patients.  

OLA FAQs

 
 

Getting Started

How long do I need to participate in OLA?

OLA is a continuous assessment in which diplomates involved in MOC will participate every year throughout their careers.

When do I begin using OLA?

Diagnostic radiology started January 7, 2019, and interventional radiology/diagnostic radiology, medical physics, and radiation oncology follow in January, 2020.
 

When does OLA begin for my specialty certificate?

Diagnostic radiology started January 7, 2019, and interventional radiology/diagnostic radiology, medical physics, and radiation oncology in January, 2020.
 

How does OLA fit within the MOC program?

OLA will fulfill the Part 3 requirement, replacing the 10-year MOC exam. Diplomates will need to continue to attest to meeting Part 1, 2, and 4 requirements each year between January 1 and March 1.
 

Will I get to choose the clinical practice areas included in my exam?

Yes, practice profile rules for diagnostic radiologists will not change. Diagnostic radiology diplomates without a subspecialty will be able to choose up to three practice areas in which to receive questions. Radiation oncology, medical physics, and interventional radiology/diagnostic radiology diplomates will not have the ability to select clinical practice areas.
 

If I’m a diagnostic radiologist who sets my practice profile as General Diagnostic Radiology, what kind of OLA questions can I expect?

You will receive questions that cover GI, GU, Interventional, MSK, Neuroradiology, Nuclear, Pediatrics, and Thoracic.

Will CME credit be given for participation in OLA?

The ABR is not accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide CME. However, those completing their OLA progress requirement each year will see a reduction in their self-assessment CME (SA-CME) obligations from 25 to 15 over a three-year period when making their attestations.
 

I recently passed an MOC exam. Will I be required to participate in OLA?

Diplomates must participate in OLA when it becomes available for their specialty unless they choose to take the five-year traditional exam.
 

How should I study?

The goal with all OLA content is that diplomates won’t have to study. All content is intended to be “walking-around knowledge,” or answers the average diplomate should know.
 

How will OLA affect initial certification?

This change will not impact the ABR’s initial certification exams for diagnostic radiology, interventional radiology/diagnostic radiology, radiation oncology, medical physics, or any subspecialties.
 

Answering OLA Questions

How many questions do I have to answer?

  • For a DR certificate only, or for a DR certificate and one subspecialty: a minimum of 52 question per year
  • For a DR certificate and two or more subspecialties: a minimum of 52 questions per year for each subspecialty certificate
  • For IR/DR only: a minimum of 52 questions per year
  • For IR/DR and any subspecialty certificate(s): a minimum of 52 questions for your IR/DR certificate, and 52 per year for any subspecialty certificate you are maintaining
  • For RO: a minimum of 52 questions per year
  • For MP: a minimum of 52 questions per year for each specialty certificate
 

How will I receive my questions?

You will be notified once a week via email that you have two question opportunities available. You may choose to answer the questions immediately or at a more convenient time. Each question opportunity will be available for four weeks. When you choose a question, you will have a limited amount of time to answer it.
 

Will I know if my answer is correct?

Yes, you will receive immediate feedback. Each question will have a brief explanation of the correct answer and at least one reference will be given.
 

What if my answer is incorrect?

You will receive a similar question on the same topic in the future.
 

Do I need to answer the included survey questions before I get my answer?

No. You can skip them from the top of the page. The “confidence” and “relevant” survey questions are designed to be a gauge and learning tool for OLA users. Users will have the ability to review their confidence and question relevance information in the history tab and compare it to their correct and incorrect answers.
 

What happens if I don’t answer a question and the timer runs out?

If you don’t select an answer and your timer runs out, the question will be counted as incorrect. If you selected an answer and your timer runs out, the system will use the answer you selected to score that question.
 

How often may I decline a question?

You will receive 10 declines per year for each set of 104 question opportunities you receive. These are intended to be used when a question arises that isn’t a part of your clinical work. Unused declines expire at the end of the calendar year.
 

Will I have 10 declines throughout the year if I do not use them?

You will start with 10 in January. Unused declines expire at the end of the calendar year.
 

How long will each question be available?

Each opportunity will be available for four weeks.
 

How many questions do I need to answer correctly?

Each question is rated individually, and each diplomate receives a different set of questions, so each passing standard will vary slightly. You will have a dashboard showing your performance against the passing standard well in advance of answering 200 questions, so you’ll know how you’re doing.
 

What do I do once I answer 52 questions per certificate maintained?

You have two options regarding OLA participation for the rest of the year.
Take a break: You may wait until next year to resume answering questions.
Keep going: You may choose to answer more than 52 questions a year. If you do, you will reach the 200-question OLA evaluation threshold faster and have your OLA performance evaluation conducted sooner. There is no penalty to continue participating.  

What if I don’t answer 52 questions during the year?

If you do not answer 52 questions a year, the remaining questions will be counted as incorrect. Please remember that the questions are based on “walking-around knowledge,” not something that requires studying.
 

Why does it take some time for a question to load?

Questions may take longer because OLA loads the entire set of images so that there is no delay once the timer starts. The load time is also dependent on your internet connection speed. For example, if there are three images to load, and you are using a public access Wi-Fi, you will likely experience a longer load time. If you are unable to wait for a question to load, you can select cancel and your question opportunity still will be available.
 

Why do some questions allow three minutes and other one minute to answer?

The timing depends on the complexity of the question. Question timing is determined by the volunteer committees, consisting of context experts who write, vet, and compile OLA questions.
 

I selected the “Answer Now” button on my dashboard page, but after the question loaded, I clicked “Cancel.” What happens? Does this count as an opportunity?

Selecting “Cancel” before the question appears on the screen does not count as a question opportunity. A question opportunity is taken only when the question is seen. At that point, diplomates should either answer the question or decline it. Choosing “Cancel” at that point counts as an incorrect response.
 

If I decline a question, does the system load a different question or bring me back to the dashboard?

Declining a question will bring the diplomate back to the dashboard and use a question opportunity.
 

How do I use the window/level item tool (looks like a sun or a gear)?

After you select the window/level tool, place your cursor over the image and left-click on the mouse. Drag up and down or left and right to adjust your image. You will see the change to brightness and contrast. To reset the image, select the “Refresh” icon.
 

Scoring and Performance Evaluation

How many questions do I have to answer correctly?

There is no set number of correct answers that ensures diplomates meet or exceed the passing standard. Each passing standard is unique to the practice profile chosen by individual diplomates.

When will I know how I’m doing?

ABR staff is working on a performance display for the OLA dashboard that will show diplomates how they are faring in relation to their passing standard. The feature will go through pilot testing and should be available by the end of the year.

When will my OLA performance be evaluated?

After you’ve answered 200 questions, your OLA performance will be evaluated at the MOC annual review every March 2, as it is currently with MOC Parts 1, 2, and 4.
 

How long will it take to answer 200 questions?

For diplomates beginning OLA in 2019 and answering 52 questions per year, the first evaluation will occur on March 2, 2023. For those answering more than 52 questions per year, the first evaluation may occur earlier.
 

How will I know if my OLA score is above the passing standard?

After you’ve answered 52 questions, your performance will appear on your OLA dashboard. This graph will show your performance as it relates to the passing standard for the questions you have answered.
 

What will happen if I’m below the passing standard after 52 questions?

Nothing. The first evaluation of your OLA performance does not occur until March 2 after you’ve answered 200 questions.

What will happen if I’m below the passing standard after 200 questions?

At the next annual review, your MOC status will change from “Meeting” to “Not Meeting” requirements. This change does not affect your certification and you have one year to remediate your OLA performance.

How do I remediate my OLA performance if I’m below the passing standard?

There are two ways:
  1. If you answer additional OLA questions and your performance rises above the passing standard at any time over the 12 months between annual reviews, your MOC status will immediately change from “Not Meeting” to “Meeting” the requirements for MOC. It’s important to remember that upgrades to your MOC status can occur at any time. Downgrades can only occur on March 2, at the MOC annual review.
  2. If you prefer, you can take and pass a traditional exam. Doing so will ensure your MOC Part 3 status will remain “Meeting” for five years. Also, after passing an exam, OLA will be re-set and any previous scoring deficit will be removed.

How is the passing standard set for OLA?

The passing standard for OLA is criterion referenced; OLA is not graded on a curve. The passing standard for each OLA question is established individually and the aggregate rating for your 200 questions defines your passing standard.

Who sets the passing standard for OLA questions?

The passing standard for each question is established by OLA participants who volunteer as question raters. This is an important characteristic of OLA; the radiology community will set the passing standard for each question.

Who can participate as a question rater?

OLA participants who watch a short question rating video and complete a four-question quiz may participate. After answering an OLA question, you will have the ability to rate the question, along with your colleagues, to establish the passing standard.
 

OLA and Subspecialty Certificates (CAQs)

When does OLA begin for my subspecialty certificate (CAQ)?

Diplomates with subspecialties in neuroradiology, nuclear radiology, and pediatric radiology begin when their specialty starts OLA. Diagnostic radiology starts in January, 2019, and interventional radiology/diagnostic radiology begin in January, 2020. For diplomates who have converted their diagnostic radiology certificate to IR/DR, OLA starts in January, 2020.
OLA is not available for a hospice and palliative or a pain medicine certificate. Those diplomates must take a traditional exam.

I hold a subspecialty certificate. How will my practice profile be set?

Since you are maintaining a subspecialty certificate, your practice profile will be set for that subspecialty and can’t be modified.
 

Technical Requirements

What devices, browsers, and operating systems are compatible with OLA?

We have optimized the OLA experience for desktop computers, laptops, and tablets. The minimum display resolution on these devices should be 1024×768. We recommend using the latest versions of Chrome, Firefox, Safari, Internet Explorer, or Edge. Preferred operating systems include Windows 10, iOS 10, and Android 6. OLA may work with other devices, browsers, and operating systems, but we will not develop software or test for performance on them.

What about mobile support?

OLA is designed to work with iPads and similar tablet devices. Support for smartphones and other small devices with resolution less than 1024×768 is not planned at this time. OLA may work on other unsupported devices, but we cannot guarantee an optimal experience.