Rating Systems Seldom Agree on Hospital Quality

Patients are urged to become savvy health consumers, and a plethora of rating systems for hospitals and physicians are available to help them. However, none of these systems are standardized, and, as it turns out, even the best known do not yield reproducible results.

Researchers analyzed the 2012–2013 results of four nationwide hospital rating systems:

• U.S. News and World Report's “Best Hospitals”

• HealthGrades' “America's 100 Best Hospitals”

• Leapfrog Group's Hospital Safety Score

• Consumer Reports' Health Safety Score

Each of the four systems had different criteria for hospital eligibility and quality. All told, 844 hospitals were rated as unusually good (“high performers”) by at least one system; only 85 of these high-performing hospitals were rated as high performers by two systems, and only 3 were rated high-performing by three systems. No hospital was rated as a high performer by all four systems; of those ranked high performers by at least two rating systems, Leapfrog and HealthGrades agreed on 55%, whereas Consumer Reports and U.S. News did not agree on any.

Similar patterns applied to low-performing hospitals: No hospital was ranked as a “low performer” by all three of the systems that rated poor performance. Of the 15 hospitals rated as low performers by two of the three systems, Leapfrog and HealthGrades agreed on 7, and U.S. News and Consumer Reports agreed on only 3.

Within individual scoring systems, the researchers found that certain characteristics were overrepresented among high- and low-performing hospitals. Leapfrog appeared to favor private over public hospitals, Consumer Reports favored small hospitals and those in the Midwest, and U.S. News favored major teaching hospitals and those treating low percentages of Medicare patients.