U.S. News & World Reports' "Best Hospitals" rankings are touted as the "global authority in hospital rankings," but their methodology leaves out some key factors, Judith Garber and Shannon Brownlee write for Health Affairs Blog.

Popularity contest for specialized medicine

According to Garber, a health policy and communications fellow at the Lown Institute, and Brownlee, SVP of the Lown Institute, U.S. News "places much more weight on hospitals' performance in specialties or complex medical procedures than on care for chronically ill patients, the population that makes up the bulk of hospitalizations." In fact, of the 448 possible points a hospital can receive for its "Honor Roll" score, 340 come from specialty scores, which only rank outcomes for "challenging or critical" procedures, Garber and Brownlee write.

Garber and Brownlee argue that while this approach makes the list useful for the "few patients who can actually shop around for a hospital for a complex procedure or problem, the overall rankings could be misleading for the majority [of] consumers, most of whom will be hospitalized not for specialty procedures but rather for exacerbations of such chronic illnesses as heart failure and diabetes."

Further, Garber and Brownlee point out that "more than 25 percent of each specialty score comes from expert opinion, measured by a survey of physicians," a process that some stakeholders say "turns the ranking into a popularity contest." For instance, according to Garber and Brownlee, research in 2010 found that the U.S. News rankings were nearly identical to rankings based solely on reputation, while a 2017 study found that "reputation had a larger impact on hospitals' scores than more objective measures."

Cost and waste

Garber and Brownlee also argue that U.S. News overlooks cost of care as a factor in its rankings, never recognizing "hospitals' efforts to cut unnecessary services, improve efficiency, or accept alternative payment models." Doing so, the authors contend, "could incentivize hospitals to improve efficiency and rein in the prices they charge, both of which could bring down the cost of health care for everyone."

In fact, they argue the ranking system encourages low-value care. They explain that while the rankings allocate a small amount of credit for hospitals that avoid preventable complications, the rankings do not award points for avoiding low-value care or ensuring a patient is well-informed on health decisions. Further, while the rankings reward hospitals for having highly specialized technology, such as a positron emission tomography, they do not have any sort of measure for whether these technologies are being used effectively or efficiently. Similarly, according to Garber and Brownlee, the rankings reward hospitals for preforming "the most complex and risky procedures without harming the patient but does not address the question of whether hospitals should perform these procedures" in the first place.

Social mission

According to the authors, aside from one measure regarding translation services, the rankings also do not assess whether a hospital fulfills its "social mission." They write, "There is no examination of how much a hospital gives back to its community through funding local not-for-profits, no information on whether it has a free clinic and how many community members are served, and no statistics on the proportion of women or people of color on staff."

"Hospital rankings have the potential to change hospital practices, for better or for worse," Garber and Brownlee write. And as we shift toward value-based care, "we need a ranking system that helps hospitals move toward that goal," Garber and Brownlee continue, "one that doesn't rely on reputation among physicians, includes multiple risk-adjusted outcomes, and takes cost of care, social mission, and high-value care into account" (Garber/Brownlee, Health Affairs Blog, 9/26).

Get the cheat sheets: How hospital quality ratings programs work

Download our one page sheets for summaries on the methodology and metric categories used in five hospital quality rating programs:

• US News and World Report Best Hospital Rankings
• CMS Star Ratings
• Leapfrog Group Hospital Safety Grade
• Truven Analytics’ Top 15 Health Systems
• Truven Analytics’ Top 100 Hospitals