On Dec. 21, CMS released a long-awaited update of its Overall Star Ratings. 3,692 hospitals received ratings in the latest update, which was postponed from July while the agency revised the underlying methodology.

The methodology changes have certainly impacted hospitals' performance: The December update contains a much broader distribution of hospital performance across the range of star ratings (see below) than the old methodology would have yielded.

Individual hospitals might find that their performance has changed considerably since the last update in April 2017: 49% (1,716) of the hospitals that received ratings in the April and December 2017 updates saw their star ratings shift—and 3% (111 hospitals) saw shifts of 2 stars or greater.

Overall Star Ratings: CMS's attempt to make quality data more accessible to patients

CMS's Overall Star Ratings are a composite quality rating based on 57 hospital inpatient and outpatient quality measures clustered into seven measure groups. Hospitals receive 1 to 5 stars based on their performance.

CMS developed the star ratings so that consumers would have a single aggregate measure through which to assess hospital quality. However, the ratings have drawn criticism from a variety of stakeholders, including providers, advocacy groups, and researchers. Some of these stakeholders have found fault with CMS's measure-level assessment of individual hospitals, weighting of measure groups in the calculation of an Overall Star Rating, and statistical methods for grouping hospitals into a particular star rating.

New methodology moves more hospitals away from middle rating

In response to these criticisms, CMS postponed its scheduled July 2017 update of Overall Star Ratings and decided to alter several main aspects of the statistical processes through which it groups hospitals into ratings categories. If you're interested in more detail, you can access CMS's FAQ about the updates here.

The upshot of these changes, according to CMS, is that star ratings are now more meaningful because they more accurately reflect important differences in quality performance. And indeed, as you can see in the chart above, the new methodology has reduced the number of hospitals assigned middling performance (three stars).

Stakeholders see further room for improvement

While many stakeholders have expressed approval of CMS's changes, others have indicated that the new methodology fails to address many of the ratings' most serious flaws. For example, detractors point out that the methodology favors low-volume hospitals that do not meet reporting thresholds for many of the star ratings' underlying quality metrics. Conversely, hospital reporting more quality measures (many teaching hospitals and larger hospitals) receive lower star ratings.

In a September 2017 letter to CMS, the American Hospital Association also raised a broader question about the ratings' usefulness: "We continue to have significant concerns about the conceptual underpinnings of star ratings… the ratings often do not reflect the aspects of care most relevant to a particular patient's needs." For example, a cancer patient could potentially benefit from more information about oncology-specific processes and outcomes, while outcomes for patients presenting with heart attack symptoms are less relevant. CMS has not indicated whether it intends to further refine its star rating methodology.

Despite controversy, star ratings being touted as distinguishing factor by high-rated facilities

It is unclear to what extent Overall Star Ratings are being used by patients and patient advisory groups (for example, neither Consumer Reports nor U.S. News and World Report have incorporated these ratings into their hospital rankings). However, hospitals awarded four and five stars are publicizing their high performance on their websites and in local news sources across the nation.