When Disaster Strikes: Measuring Health Care Quality in Crisis-Affected Communities

In an interview with Maria DeYoreo, PhD, senior statistician at the RAND Corporation and co-director of the RAND Center for Causal Inference, she shares the inspiration behind her study published in the February 2025 issue of The American Journal of Managed Care® titled, "Do Localized Disasters Impact Clinical Measures of Health Care Quality?"

DeYoreo also discusses the key concerns for performance measurement programs when assessing provider quality in disaster-affected areas and how CMS has addressed these in the Medicare Advantage Star Ratings program.

This transcript has been lightly edited for clarity; captions were auto-generated.

Transcript

What are the key concerns for performance measurement programs when assessing provider quality in disaster-affected areas? How has CMS addressed these concerns in the Medicare Advantage Star Ratings program?

Well, the primary concern is that health plans or contracts that operate in areas that were impacted by localized disasters, such as fires, floods, hurricanes, or earthquakes, may see lower performance scores. For example, they may have lower rates of breast cancer screening or receive worse ratings of patient experience measures as a consequence of the disaster. It might be unfair to compare them to contracts that were not affected by a disaster.

CMS has addressed these concerns with something called the disaster policy in the star ratings, which allows contracts that had a meaningful or substantial percent of enrollees residing in disaster-affected areas to receive the higher of their current year–measure star rating or their prior year–measure star rating and scores. The purpose of this policy is to avoid unfairly penalizing disaster-affected contracts.

What was the objective of your study? Why did you decide to investigate this?

We wanted to determine the effect of localized disasters on health plans' quality performance scores using a variety of clinical and patient experience measures. We investigated this because we wanted to know whether disasters meaningfully impact performance in a negative way and which measures are affected by disasters and which aren't.

Ultimately, this has implications for performance measurement programs in terms of whether quality of care measure performance data can continue to be used for public reporting or payment purposes.

How did you define and measure "disaster exposure" for Medicare Advantage contracts?

We defined disaster exposure as the percentage of MA [Medicare Advantage] contract enrollees or beneficiaries who resided in disaster-affected areas during the performance year or the performance period.

The disaster-affected areas were FEMA [Federal Emergency Management Agency]–designated individual assistance counties where the HHS secretary declared a public health emergency.