Three radiologists and one emergency medicine physician shared tips, caveats, and lessons learned in the implementation of appropriate use criteria (AUC) during a timely session at the 2020 virtual meeting of the Radiological Society of North America, “Decision Support and the Implications for Federal Regulation (PAMA).”

In what could be a final reprieve for the program that requires a CDS mechanism be used in the ambulatory and ED settings for MR, CT, and nuclear medicine in a minimum of eight priority clinical areas, CMS recently granted ordering clinicians one more education year in 2021 prior to the implementation of penalties for failure to consult AUC before ordering advanced imaging.

Ali Raja, MD, MBA, MPH, Massachusetts General Hospital, and Pamela Johnson, MD, Johns Hopkins Health System gave the first two of four talks, all from physicians associated with four academic health systems that are qualified provider-led entities (QPLEs). They offered important context for PAMA, a key lesson learned from the specialty’s failed CMS Medical Imaging Demonstration in 2011, and insights into the opportunities the CMS program provides to develop and implement evidence-based care. Coverage of the other two talks—from Keith David Hentel, MD, Weill Cornell Health System, and Jon Mongon, MD, PhD, University of California, San Francisco—will appear in the March issue of Hub.

In addition to practicing emergency medicine, Ali Raja, MD, MBA, MPH, serves as executive director of the department of emergency medicine at Mass General; senior faculty at Mass General’s Center for Evidence-Based Imaging; and executive director of Harvard Medical School Library of Evidence, a repository of clinical decision support for imaging. Dr. Raja began his talk by providing the context for the passage of PAMA Act (passed in 2014) and then shared lessons learned from the specialty’s failed Medicare Imaging Demonstration Project, which ended in 2013. 

PAMA began out of widespread concern among policy makers and payers for the potentially inappropriate use of imaging based on three underlying concerns:

·      escalating use of high-cost imaging and the belief that some was unnecessary.

·      payor focus on reducing both the use and the payment per unit of service; and

·      concern about patient safety, radiation exposure, and also the patient experience in getting care that was needed.

“Most importantly, while all of these concerns were playing out, one thing that was obvious and inarguable was the fact that there was a large amount of variation in the use of imaging,” said Dr. Raja. “Any time there is variation, there is a concern that some of that variation points to unwarranted use of imaging.”

Dr. Raja shared findings from the team at Partners Health and published in 2015 that looked at variation in the use of CT and MR imaging in a large federal Medicare database, linking areas of high reimbursement with higher-than-average use; and likewise, identifying areas with lower reimbursement with lower use rates. “That variation led to the thought that whether this is right or wrong, it is different across the country and we can all work to improve that,” he said.

One of tools the team looked at to combat variation was clinical decision support (CDS), as there was an existing body of literature that demonstrated the effectiveness of CDS in managing the inappropriate use of imaging. The team at BWH Center for Evidence Based Imaging has used CDS to decrease the use of CT for suspected pulmonary embolism by 20% and increase yield by almost 70% over two years. They also decreased the use of MRI for lower back pain and CT for traumatic brain injury.

The team at MGH also looked at adherence to evidence-based guidelines, including ACEP guidelines for Head CT for minor trauma, NQF guidelines for Chest CT in the ED for PE, and ACP guidelines for lumbro-sacra MRI for ambulatory patients. It was able to show that the use of CDS increased the amount of imaging that adhered to evidence-based guidelines.

Lessons from the MID

If Dr. Raja had just one takeaway, it would be this: If CDS is implemented broadly and not in a targeted fashion, you risk duplicating the experience of the CMS Medicare Imaging Demonstration (MID) Project, nine years ago, in which 99% of alerts were ignored. BWH—along with Geisinger, University of Pennsylvania, and the Weill-Cornell Medical College—was one of five convenerships that implemented CDS broadly, conceived as a potential alternative to pre-authorization based on professional society guidelines. It focused on 11 targeted high-cost outpatient imaging procedures for Medicare fee-for-service patients, utilizing MID guidelines.

Physicians could ignore guidelines if they wished, and physicians received comparative data so they could see their performance compared to peers. Out of 83,000 orders during the demonstration period, only 8% in the control group and about 6% in the intervention group had actionable results.

“Unfortunately, when those 6% had alerts go off, 99% of alerts were ignored; 1% had exams modified; 0.03% had exams cancelled,” noted Dr. Raja. “This really showed that broad implementation of non-evidence based CDS doesn’t actually change anything, and, in fact, 99% of those alerts being ignored meant that they were just useless facts popping up and impeding imaging flow. There was no change in the use of high-cost imaging both in the data that we found and in an independent review by RAND that was submitted to Congress in 2014.”

Dr. Raja cited a Health Affairs blog post that identified information technology, especially EHRs, as a major contributor to physician burnout because they contribute interruptions, distractions, and inefficiencies. “If we are going to implement CDS, and I am a firm believer in doing so, we need to do it in an effective and evidence-based manner that doesn’t make physician burnout worse.”

AUC in a Capitated Reimbursement System

Pamela Johnson, MD, VP of Care Transformation, Johns Hopkins Health System (JHHS), and Vice Chair of Quality and Safety, Department of Radiology, discussed the Johns Hopkins experience as a qualified provider-led entity (QPLE) in a capitated reimbursement system, the challenges in developing AUC and measuring its effectiveness in a large academic and community practice health system, as well as opportunities for scholarly activity and cross-institutional collaboration to harmonize rules.

Johns Hopkins spent three years in the state of Maryland’s global budget revenue model, which saved CMS almost $600 million, before advancing to a total cost of care model. Now JHHS will become accountable for the total cost of care for those patients assigned to its system, projected to save $1 billion between 2018 and 2023.

“Our work is tightly aligned to this reimbursement model, and our strategic plan is also closely aligned to the appropriate use criteria program,” said Dr. Johnson. “We are committed to improving the quality and affordability of health care, aiming for precision in everything we do, working like one organization with one set of rules and making Johns Hopkins easy. High value radiology has been a priority in our department for a number of years beginning with appropriate order, exam performance and interpretation, and then making evidence-based recommendations to guide optimal patient management.”

Harmonization with Existing Initiatives

As the AUC mandate was nearing its kickoff date, the JHHS emergency department had championed adding evidence-based guidelines in the EMR to improve the efficiency and effectiveness of care being delivered and 70 guidelines were developed, a combined effort of faculty, residents, PAs, and pharmacists that included best practices through literature review and evidence-based guidance. This effort reduced unnecessary hospital admissions and optimized the use of labs and imaging consults, medications, and length of stays.

“They were doing some really fantastic work and our Strategic Priority Executive Committee funded advancement of this work to the inpatient and outpatient settings across the health system,” Dr. Johnson said. “We wanted to harmonize our appropriate use criteria program with this existing work.”

For example, using a chest pain guideline, the department reduced one-day length of stay over several years.  As a QPLE and in accordance with the federal regulations that allow a targeted approach in developing AUC, the JHHS AUC team took the approach of developing a narrow foundation so that they could design strong-evidence AUC in a narrow area and then build on that in a data-driven approach for areas where they identified over-use in the institution.

“We pulled together the clinical teams of specialists that practice in the ED and a separate team that practices in the ambulatory setting, and they were the champions of this work in collaboration with radiology,” Dr. Johnson reported. “We had representatives from all of the specialties that relate to the eight priority clinical areas. They worked together so that we had multispecialty input but also consensus across the health systems for whatever we were designing because we have academic hospitals and community hospitals, and we needed to make sure that all of our diagnostic decision-making is harmonized. These were rolled out across all regional hospital EDs and a large community practice and addressed all of eight priority clinical areas.”

Likewise, at same time the CMS program was rolling out, JHHS was rolling out an MSK service line that was developing an integrated service line approach for lower back pain in the ambulatory setting; Johnson’s team wanted to integrate the AUC for imaging that they were developing. The outcome was a comprehensive guideline that includes a lot of decision-making beyond imaging and beginning when the patient presents in the ambulatory setting, but the radiology decision-making is in accordance with the CMS requirements for a QPLE.

Requirements of a QPLE

As a QPLE, the team invests a great deal of time in performing broad literature reviews to select high-quality evidence, which it incorporates in evidence tables that are graded, an activity with value for educating residents to critique literature and trials and to understand appropriate use. “We are trying to keep this as an academic initiative where the literature review and evaluation are a critical part of defining best practice even beyond imaging using evidence-based guidelines,” said Dr. Johnson.

JHHS also is a member of the High Value Practice Academic Alliance, also a QPLE, which provides further opportunities to develop evidence for the AUC program. A recent meta-analysis by the Alliance conducted to evaluate for occult hip fracture in elderly patients was published in Radiology. 

QPLEs are required to post evidence on their web sites along with the evidence tables. The mandate also requires that providers get feedback on their appropriateness as determined by clinical information entered at time of order.

As a participant in the Maryland Total Cost of Care model, the team is evaluating more than the appropriateness of the orders. “We are looking at ordering effectiveness,” Dr. Johnson said. “We also are looking at how our evidence-based guidelines, which incorporate imaging, labs, and other decision-making, how they affect other important measures that contribute to patient outcomes and total cost of care, including our overall diagnostic rate, the number of patients admitted, procedures in hospital, cost, and charge reduction. Then from there, we want to design even more robust outcome dashboards that look at downstream testing, downstream treatments, procedures, and patient outcomes. This is our approach to performance improvement across the care continuum and longitudinally for these patients to reduce total cost of care, improve their health, and improve their outcomes.”

Hurdles and Next Steps

JHHS is working on addressing a number of hurdles and planning next steps for the initiative. “We are working on handling imaging orders placed outside the EMR through an Epic web-based platform that can be accessed by other EMRs,” she reported. The department is also working on a structured impressions initiative to enable more robust data mining for the purpose of providing more useful feedback reports to clinicians.

“Our Emergency residents have been getting ordering effectiveness reports for two measures— pulmonary embolism and head CT—that collate their data to tell them how many they ordered and more importantly what percentage they ordered had a critical finding compared to the average for their peer group (for instance 4th year resident is compared to other 4th year residents),” Dr. Johnson explained. “This is important data. We are trying to develop benchmarks to know what the optimal positive exam rate is, and there is more work to be done on that.

"In the ambulatory setting it is not as easy to measure ordering effectiveness, so we are developing more structured reports. For example, cardiac radiologists incorporate the CAD Reporting & Data System into their impression, to facilitate determination of what percentage of reports had actionable findings and give that feedback to the ordering providers."

“We are developing PE-Rads in the High Value Academic Practice Alliance for the same reasons, because we know that an isolated sub-segment PE is very different from a saddle embolism with right heart strain,” Dr. Johnson continued. “Ordering effectiveness needs to take into account these different levels of disease severity.”

Interestingly, the Maryland GBR Model is outside the required payment settings defined in the law. “This could have been a real challenge for us in driving engagement, but everyone agreed we are going to do the right thing for patients and proceed regardless if we are in some ways exempt because of the global budget model in the ED setting,” said Dr. Johnson.

In conclusion, Dr. Johnson shared that developing AUC as a QPLE has been an exciting project for JHHS radiology, one that has been very much guided by the Hopkins mission and Maryland Total Cost of Care Model. “We look forward to evaluating more downstream outcomes and resource utilization for ongoing performance improvement using a data-driven approach that enables us to continually refine our interventions,” she concluded.