ACCME PARS Update, Part 1 of 3: Reaccreditation Processes

ACCME PARS Update, Part 1 of 3: Reaccreditation Processes

The ACCME updated the reaccreditation process in early 2018. To discover the extend of the changes and provide guidance to readers, Erin Schwarz and Jacob Coverstone, editors representing the Almanac, spoke with Dion Richetti, vice president of accreditation and recognition, and Teri McCauley, manager of accreditation and recognition systems with the AACCME.

According to Richetti, even though things are different, little is changing. After speaking with the ACCME … we kind of agree.

Since announcing the changes, the ACCME has held webinars for providers preparing for reaccreditation, but we at the Almanac want to prepare providers at all stages of accreditation for the changes. (See a short summary of the changes at the bottom of this article.)

The following series is taken from a conversation that confronted assumptions, challenged processes and (we hope) clarifies ACCME reaccreditation going forward.

Part 1

Almanac: Thank you for making time today. We wanted to discuss the changes that are being made to the ACCME self-study/reaccreditation process.

ACCME: We really appreciate the opportunity to talk with you and share with your readers what we consider to be good news about enhancements to PARS.

Almanac: We want to clarify first, we’re also interested in changes to the initial application process. Our first question is in two parts:

1. What prompted the changes to the self-study outline, both for initial and reaccreditation?
2. What was the internal process for determining the changes that were needed, such as modifications to the questions in the accreditation outline?

ACCME: Those are great questions. I want to preface our comments with one thing — when you said “the process for accreditation and reaccreditation” — we’re not changing the process. The process is the same. The three data points remain the same: the self-study, performance in practice and the interview.

The only thing changing in terms of what the ACCME is doing is that we’re responding to the community’s request to put everything into a more efficient, secure online system of collecting the information. That’s the primary difference.

At the same time, to address your current question, obviously we’ve been using, since roughly 2009–2010, a format for the outline for the self-study, which over the years has changed many times. What’s been consistent for several years is that we’d asked providers to use two examples to discuss how they planned their CME activities.

One of the things we’ve heard is that this approach was somewhat limiting — that the two examples didn’t always provide surveyors with all of the information they needed.

For instance, providers might have five ways that they produce CME activities — and very often that wouldn’t necessarily be captured in all of the performance-in-practice files. Or it would be, in the performance-in-practice files, but wouldn’t be described in the self-study. 

So, the primary change to the questions in the outline for the self-study is to ask providers to describe how it is that they plan their CME activities, rather than limit it to two examples. That gives them the opportunity to provide full content to the surveyors.

We didn’t see it as a major change but more as a reaction to the needs of the surveyors and our Accreditation Review Committee (ARC) members. Our volunteers need to be able to understand the full context of a provider’s CME program.

Does that help?

Almanac: Yes, especially the explanation that the process itself is not actually changing and retains the same basic elements. That’s an important point, and one that the CME community should note.

The previous self-study process was in place for about seven years. But in that format, when the instructions said “give us two examples,” it included that, if a provider felt those examples were insufficient, that they could add a “Section B” to a criterion and include additional examples.

Were you finding that providers weren’t utilizing that opportunity?

ACCME: It was very rare that providers actually gave a more comprehensive response. One of the things that’s important to understand about accreditation is that most of the providers will give the information that’s requested. So, when a part of the process is presented as “optional,” it adds an element of subjectivity and typically providers err toward answering only those questions that are required.

Instead, we say, “tell us how you plan your activities,” and then it’s up to the provider to give the complete picture. And, obviously, the point of the accreditation interview is to clarify anything within the materials, and that’s always an option. It always has been.

Almanac: That inspires a couple of follow-up questions:

We understand that it’s a provider’s responsibility to submit enough information to help ACCME to decide. In the past, it hasn’t gone well when providers submitted vague responses; you like answers to be specific and clear. Submitting examples tended to help with that clarity.

Do you have recommendations for organizations to ensure that the information they provide is specific enough if providers are no longer being encouraged to provide examples? As you noted, providers will naturally follow the outline and tend not to do what’s optional.

ACCME: Asking for examples was a tactic that ACCME employed to help providers convey a picture of their planning process.

What we have now are specific requests for information. If you look at the current outline — “describe the process or processes used to identify the professional practice gaps of your learners and the educational needs that underlie the identified professional practice gaps” — that doesn’t limit you to two, and it says “describe the process or processes.”

Now again, the more that we would quantify things, the less flexible the outline becomes. So we felt that the new outline would reduce the burden for surveyors to try to find out where, within a self-study report, a provider tried to talk about how they address professional practice gaps. At the same time, we wanted to allow the provider to give a complete picture of all of the things that they’re going to show in the performance-in-practice files.

Almanac: Was there a deliberate decision to eliminating the ability for providers to include things like screenshots and graphics, or was that a limitation of this new, online platform?

ACCME: There was never a requirement for providers to give screenshots or graphics.

As you know, we have a criterion-referenced system. We’re trying to create a platform in which everyone has an equal opportunity to give the ACCME the information they need in order to address the criteria.

None of our questions asked for screenshots or graphics. We’re asking everyone to describe, using language, and to show, using evidence and description in the performance in practice, what they actually do.

We received a lot of feedback from surveyors stating that it was often difficult — because every self-study looked so different — to find what was most relevant. Ultimately, we’ve always asked for descriptions and evidence, and we’re still asking for descriptions and evidence. We believe this approach will simplify the process and create a more uniform approach that doesn’t offer advantages to individual providers who have the resources to create [enhanced] materials.

ACCME PARS Update, Part 2 of 3: Implementing System Updates
By Jacob Coverstone and Erin Schwarz

A continuing conversation with Dion Richetti, vice president of accreditation and recognition, and Teri McCauley, manager of accreditation and recognition systems with the AACCME.

Almanac: How is ACCME helping surveyors, members of the Accreditation Review Committee (ARC) and ACCME staff transition to the new submission process, and how will they be trained to interpret the results?

ACCME: “Interpret” might not be the best word because we don’t ask surveyors to interpret anything; we ask them to collect data.

Just as we transitioned the system several years ago to a PDF -based system, and then to a combination online/PDF system, we’re now going to a completely online system. Just as any new digital platform requires training and getting used to it, we anticipate that we will need to support all the providers and all volunteers in the use of these tools — but they’re simply tools.

So yes, providers, surveyors, ARC and staff are learning to use the new tools. And we’ve been training providers — demonstrating the enhancements to the accreditation process — starting with the November 2018 decision cohort. We’ve incorporated it into our ongoing self-study webinars, which take place three times a year. 

Almanac: Going forward, is it your expectation that you’ll keep hosting those training webinars for each cohort, or do you envision a point when you’ll ask providers to view something archived on the website?

We’re wondering about the ongoing or foreseeable future concerning transitioning folks to this new platform.

ACCME: That’s a great question, and I certainly wouldn’t make a prediction beyond what we’re planning now. What we’ve found is that this was a way for us to connect with each cohort as they started, and to make that material available on the web. We felt that this was a much more accessible way to give all providers access to the same information and a live opportunity to ask questions. We think that’s great, because when you may think that you don’t have a question, but when you hear someone else’s question, you may go “oh yeah, I wish I’d asked that!”

So, we like doing these webinars, and will continue to have a venue per cohort, but we’ll see what the future holds.

Almanac: It sounds like “standardization” is the theme we’re getting in all of this. Is that an accurate interpretation?

ACCME: Our goal has always been to give all providers the same opportunity to demonstrate their CME program through the accreditation process. I think what’s different, in terms of opportunity, is that we’ve created — with this webinar — equal opportunity for all providers to participate in the self-study training without having to travel to Chicago [for the self-study session] and to have it be as specific to that cohort as possible.

Almanac: You also mentioned before, just providing a platform that gives everyone an equal opportunity and standardizing responses … information becomes a little more consistent …. That’s why we noted that, thematically, standardization seems to be an overarching theme. 

ACCME: I think “simplification” has been the principle behind the ACCME’s evolution for as long as I [Dion] have been associated with the organization. And, we see this as further simplification.

The idea is to reduce burden and shipping costs for providers.

Almanac: I want to ask more general questions about simplification, and you have had some experience with simplification, so we’re wondering if — through simplifying the process — have you previously encountered any unintended consequences?

For example, the simplification process that led to the performance in practice following a structured abstract reduced the amount of space for the description of professional practice gaps.

Have you experienced an unintended consequence wherein a provider may not have had sufficient opportunity to provide information that would be crucial to the decision-making process? And, if not, have you about anything similar in the shift toward the new, structured way of asking self-study questions?

And lastly, if you later learn that providers aren’t able to paint a full picture of their activity planning through this new process, how might that be remedied?

ACCME: To answer your first question: no.

If you look at the compliance rates for the last seven to eight years, you’ll see that there has been an increase in compliance with Criterion 2 to the point where something like 95 percent of providers are found in compliance with C2, 3, 5 and 6, so we feel that the simplification of that process of performance in practice, if anything, has reduced the amount of extraneous information reviewed by surveyors.

Almanac: Thank you. We’re trying to tie a couple of these threads together to explore where unintended consequences may emerge. Previously, in the self-study process, there was more space to provide information on professional practice gaps.

For example, if you planned a large annual conference, planned off of a multi-gap structure, you may not have the ability in that performance-in-practice file to display all of the gaps that you have identified. Whether or not each individual gap needs to be displayed is a question that needs to be interpreted on your [the ACCME’s] end, but, as a provider, I’ve always understood it that we should provide a comprehensive perspective: “Here are the gaps we are supporting.”

With a larger-scale annual conference, we may have identified 30 different gaps, and we’ve created different sessions to address them, and we don’t have some sort of thematic link between the sessions … It seems that the current performance-in-practice structure would not allow for all of that information to be presented.

Up until now, the self-study provided a place for that exposition; it seems like the opportunity to present those examples are being removed from the self-study. In essence, providers may show evidence of adhering to a planning process, but that evidence may now be incomplete. Is there any concern about that?

ACCME: No, I don’t think we are concerned about that. As you know, the structured abstract has been in place for roughly four years. During that period, you’ll notice that there’s been no change. I think by the time we implemented the structured performance-in-practice abstract, providers had pretty well mastered the concept of developing CME activities that meet the professional practice gaps and underlying educational needs of their learners.

That had been well demonstrated through performance in practice, so the ACCME is not concerned that maintaining the format of the structured abstract in this digital platform is going to change anything further.

We have continued to say to providers: If you have a regularly scheduled series, and you plan that regularly scheduled series to meet many different, specific professional practice gaps and underlying educational needs, it’s perfectly acceptable to summarize those into an overall professional practice gap and underlying educational need.

Providers have come up with many different tactics to show, in a short 50-word/100-word paragraph that they are meeting that requirement. If ultimately our objective is to ensure providers are demonstrating compliance, we have had no change in the compliance with those criterion. In fact, if anything, it’s gone up.

I think the intended consequence of this change was to reduce the administrative burden on providers. Certainly providers can, and should, construct the education as far down to the level of the individual learner’s needs as they think is appropriate.

To demonstrate compliance with the accreditation requirements, that level of description is not what we’re asking for.

ACCME PARS Update, Part 3 of 3: Going Forward With PARS
By Jacob Coverstone and Erin Schwarz

A continuing conversation with Dion Richetti, vice president of accreditation and recognition, and Teri McCauley, manager of accreditation and recognition systems with the AACCME.

Almanac: Earlier, you mentioned that as the simplification process went into effect, as the structured abstract went into effect, compliance rates started going up.

Did anybody question or look at that change as potential for false positives?

Could it have been that we were no longer assessing enough, or thorough enough information to tell that a provider was operating in a noncompliant manner? Or, are you absolutely confident that as much information as is needed to make accurate findings is still being presented by providers throughout the process? That the trend of improved compliance findings after the collection process was simplified are all accurate.

ACCME: Providers, as you know, attest to only giving the ACCME accurate information, and we are a voluntary, self-regulated system. Our objective is to provide opportunities for providers to tell us what they’re doing well, and sometimes we see that they’re not always meeting the expectations.

There’s no indication that the data we’ve been receiving for the past 10 years under the current set of criteria fails to give us the information we receive to make accreditation decisions that are fair, accurate and objective.

Remember, the change that we have put in place for the most part are not changes to the information we collect. And, throughout the entire review process, and all the way through the ARC and up to the board meeting, we ask providers for additional materials. We have that opportunity to ask for more information if something is unclear.

Almanac: Are there character-limited fields in the new platform? Could there be an issue where there “won’t be enough space” or something similar in the webform? In the PDF, at times, there wasn’t enough space to sufficiently articulate gaps or other information. Are you prescribing a character limit?

ACCME: In terms of our word limits, nothing has changed from the abstract. The ACCME has taken the abstract and moved it into an online platform, and also we have simplified some of the tabular information into a format, which is the same, so that surveyors get a complete set of documents related to financial relationships and individuals in control of content. We spend the most time going back to providers for a complete and thorough list of all individuals in control of content and the relevant financial relationships and the nature of those relationships.

We do our very best to give providers every opportunity to provide all of the information. If you follow the structured abstract, you should be able to, as a provider, give the ACCME and its volunteers the information we need in a way that will be the most efficient and have the best results.

Almanac: You said something interesting. You said that providers have used a variety of different “tactics,” and that’s what providers are hungry to learn. We often refer providers to the ACCME’s online compendium of compliance and noncompliance. Does the ACCME plan to update the compendium? Do you feel like everything up there remains applicable, or can we look forward to getting new examples?

ACCME: The answer is “yes” and “yes.”

The ACCME no longer publishes a static compendium document. On our website, what we have instead is a dynamic interface where anyone can search for examples across all 38 criteria. [Editors’ note: You can view the ACCME compendium here: http://www.accme.org/examples-compliance-and-noncompliance]

We have recently redesigned our website and as part of the redesign process we updated and streamlined the examples.

Almanac: As a response to the standardization of how information will be received by ACCME, are you expecting that new compendium examples will be broader reaching — such that examples may be more applicable to a broader number of organizations — because the information will now be input in a similar manner? There won’t be as much variance in the submission process, or do you think it’ll kind of stay the same since compendium examples are based off of surveyor reported information?

ACCME: I want to clarify. I think the term “standardization” isn’t the ACCME’s objective. In other words, providers don’t have a standard way that they do CME. What the ACCME wants to do is simplify the accreditation process. We want providers to use — and we think our system allows for — a liberation of format. We’re not asking for everybody to do things in a cookie-cutter way in their education, and we think that the Commendation Criteria, with the menu of choices, helps providers to be unique and specific in what they do. So our examples are only driven by the creativity of the providers who submit their examples.

We don’t anticipate any change in the ACCME’s approach about putting that information out. We want the resource to be as applicable to as many providers as possible, but we realize that there are different provider types, and we’ve always encouraged providers to read the examples and try to see themselves within it but not necessarily to copy those examples — but to take from those examples what’s applicable to them as a unique provider of CME.

I hope that’s helpful, I’m not sure we’ve answered your question.

Almanac: Thank you, I think that addresses what we were trying to ask. Kind of a side question — and one that we think everyone wants to know the answer to — is why is PARS so slow now?

ACCME: The initial download when providers first interface with the enhancements, which has been available since the fall of last year, does take a little bit of time to open the application. I’m not sure if we’ve had other complaints about PARS being slow, other than that initial opening when a provider logged in and created a new password to the dashboard.

Almanac: Any time I go to PARS now, I’m faced with a screen that says “preparing application,” and it takes several minutes to actually get to the next page. 

ACCME: We are working on this issue as it arises, and we encourage providers to contact us if they experience this issue. We’re always looking to hear from our providers and, with the online system, we are sending our normal evaluation that the ACCME conducts after each interview. We’re happy to hear providers’ suggestions for changes and improvements to the system.

Almanac: If a provider is working on their self-study in PARS, and they’re not saving their work regularly, and the system times out, will any work entered since the provider last hit “save” be lost, or would their progress be automatically saved at the point of timing out?

ACCME: The information is saved at that timeout. It autosaves it.

We’re now beginning to work with the fourth cohort and the feedback has been primarily positive. We understand that there have been technical issues, and it’s important that we continue to hear about those issues, and nobody should every be shy about contacting the ACCME, particularly about technical difficulties and using our system.

 Almanac: Well, thank you. This has been a wonderful opportunity to speak with you about an issue of great curiosity and importance to our readers.

Summary of Changes to ACCME Accreditation Process

• All documentation will be submitted online, through the PARS system, including a) self-study narrative and b) performance-in-practice files.
• The self-study outline has been modified to eliminate the “two example” requirement for Criteria 2-6 and focuses questions on provider’s process or processes for each Criteria (click here to see the outline for the July in November 2019 cohorts).
• The performance-in-practice files require similar information as was previously requested in the structured abstract (click here to see the version for the July and November 2019 cohorts).
• The ACCME requests that providers complete a template Excel file, rather than submitting individual completed disclosure forms or other documentation (see screenshot below).

ACCME’s “Performance-in-Practice Individuals in Control of Content” Excel Template

ACCME’s Instructions for Providers

Page 1 of Self-Study Report Outline