A new analysis by researchers at Brown University shows the association of Medicare Advantage star ratings with racial, ethnic and socioeconomic disparities in quality of care.
PROVIDENCE, R.I. [Brown University] — A five-star rating is supposed to be a gold standard and a reliable measurement of quality. Yet when it comes to Medicare Advantage health care plans, highest-star ratings don’t always indicate top-level performance for all beneficiaries, a study by Brown University researchers found.
The new analysis, published in JAMA Health Forum, shows that five-star ratings, the scale’s highest, are only modestly associated with quality of health care experience for racial/ethnic minorities and socioeconomically disadvantaged enrollees in the plans.
“It’s clear from this study that the way the current star ratings are constructed, they’re not capturing the full experience of all beneficiaries,” said David Meyers, lead study author and a Brown assistant professor of health services, policy and practice.
Meyers said that studies in other health care contexts have found that rating systems don’t tend to address racial and socioeconomic inequities within those systems without an explicit plan to do so.
“And in this case, the fact that Medicare Advantage plans can earn higher-star ratings overall even if they’re not adequately serving minority beneficiaries doesn’t provide much of an incentive to address health equity,” he said.
Since 2008, the U.S. Centers for Medicare and Medicaid Services (CMS) has used a five-star rating system to measure the performance of Medicare Advantage plans. Not only can potential enrollees use these star ratings to make decisions about plans, but highly rated plans stand to reap large financial benefits.
“Payment bonuses for four- and five-star plans can top $6 billion annually, so there’s enormous financial incentive for these plans to perform well in terms of rating,” Meyers said.
More than one-third of Medicare beneficiaries were enrolled in Medicare Advantage plans in 2019. Yet there hasn’t been much research done to validate what the star ratings mean and if they do an adequate job of capturing the experiences of an enrollee who might be in a plan, Meyers said. In addition, these plans enroll higher proportions of racial/ethnic minorities and people with lower income and education than the traditional Medicare program.
Meyers and his team wanted to find out how well the star rating system works for different populations of enrollees. Their study addresses the association between a Medicare Advantage plan’s overall star rating and what the star rating would have been if calculated for enrollees who are racial/ethnic minorities or who have lower socioeconomic status, as well as whether plans with higher star ratings have lower disparities in care.
The Medicare Advantage star ratings are generated using data from all enrollees in a plan, and CMS assigns bonus payments without segmenting results by race, ethnicity, or socioeconomic status, Meyers said.
For the study, the researchers analyzed the experiences of 1,578,564 Medicare Advantage enrollees in 2015 and 2016, taking into consideration racial/ethnic and socioeconomic factors. They compiled data from five different sources that allowed them to recreate about 75% of the measures that go into the star rating calculation. From that individual data, Meyers said, they were able to reconstruct CMS’s star ratings, but broken down by race, ethnicity and socioeconomic status.
“What we essentially did was to go through and say, ‘If we were to rate a plan based only on the experience of Black beneficiaries — or Hispanic beneficiaries, or low socioeconomic beneficiaries — what star rating would the plan get?’” Meyers said.
The researchers then compared the different star ratings by group to see how they related to each other. The study had several key findings.
First, the researchers observed only a modest correlation of simulated star ratings when calculated for low vs. high socioeconomic status enrollees, and between racial/ethnic minority and white enrollees in the same plan. Second, plans with higher ratings have larger racial/ethnic disparities than did those with lower ratings. Third, contracts with lower concentrations of low socioeconomic status and Black/Hispanic individuals had larger disparities and worse quality for these individuals. By contrast, plans with higher enrollments of low socioeconomic status and Black/Hispanic individuals did a better job for these beneficiaries.
Of the most significant of these findings, Meyers said, is the fact that as the official star rating went up, the disparities increased among beneficiaries of different racial, ethnic and socioeconomic groups.
“It was within the five-star plans where we saw the largest disparity in outcomes between high socioeconomic status and low socioeconomic status beneficiaries, between white and Black beneficiaries, between white and Hispanic beneficiaries,” he said.
While the study did not examine the reasons for these differences, Meyers said they may have to do with structural barriers such as reduced access to providers as well as structural racism — similar barriers to care that lead to health care inequities in general.
The main concern revealed by the findings, Meyers said, is that because the plans can earn high ratings — and high financial bonuses based on those ratings — despite providing inequitable care, that means that the plans aren’t being adequately incentivized to address disparities. He said he hopes these results are motivating for plan managers.
“These results call for more attention by plans to ensure that they’re doing everything they can to address these disparities in outcomes,” he said.
For CMS and policymakers, the study results are a sign that the current Medicare Advantage star ratings system is inadequate, the researchers wrote in the study: “If aggregate contract star ratings hide clinically important differences in quality between advantaged and disadvantaged plan members, then quality measures that directly assesses equity may be needed.” The researchers suggest that stratified ratings be made public so that beneficiaries can make better decisions about the best plan for them — currently, there’s no way to see how a particular group, such as Black beneficiaries, rates a plan.
The researchers also suggest that CMS includes a measure of equity in the ratings calculation.
“Right now there are about 30 different measures that are factored into star ratings,” Meyers said. “Adding a measure that judged a plan on how well they addressed disparities could create more of an incentive for plans to try to ensure that they’re providing high-quality care to all of their beneficiaries, and that no group falls through the cracks.”
Brown researchers who contributed to this study include Vincent Mor, Momotazur Rahman, Amal Trivedi and Ira B. Wilson. This work was supported by grant R36HS02705101 from the Agency on Healthcare Research and Quality.