I

t’s hospital ratings season in America, that time of year when marketing executives kick it into high gear to trumpet — and spin — the way their hospitals are graded by outside organizations.

This year, their workload has been especially heavy. In addition to annual rankings released Tuesday by U.S. News & World Report, the federal Centers for Medicare and Medicaid Services (CMS) just released its star-rating system for hospitals nationwide. While the ratings offer some valuable information to consumers, here’s why you shouldn’t put too much stock in the results.

No matter how objective the metrics, ratings are inherently subjective

The federal government and private organizations use a vast array of hospital data to break down their performance and boil it into easy-to-understand ratings for consumers. U.S. News & World Report formulates its rankings by relying on a mix of physician surveys and data reported to trade groups and the federal government. CMS ratings are based solely on data hospitals are required to report to the agency.

advertisement

The information itself may be objective, but which data points are used — and how they are weighted — can lead to contradictory conclusions. Consider the following:

Of US News & World Report’s top five hospitals,  only one  — Mayo Clinic — received a top, five-star rating from CMS. The rest — Cleveland Clinic, Massachusetts General Hospital, UCLA Medical Center, and Johns Hopkins — were not even in CMS’ top 100. (Johns Hopkins did not receive a rating from CMS because of incomplete data).

J.B. Silvers, a professor of health care finance at Case Western Reserve University, said the information, while boiled down into simple lists and stars, can be difficult for consumers to unpack, especially when hospitals are working behind the scenes to figure out ways to better their scores every year. “Some hospitals are better at working the numbers than others,” he said. “My guess is the safety net hospitals aren’t as good at it as richer hospitals.”

Newsletters

Sign up to our Daily Recap newsletter

Please enter a valid email address.

One bad score, and your rating is shot

Lobbyists and some hospital executives argue that the CMS star-rating system penalizes providers that treat greater numbers of low-income patients. That’s because the CMS system hurts hospitals with higher rates of readmission, a factor heavily influenced by a patient’s economic status.

Critics have noted that teaching hospitals and safety net hospitals — both of which tend to deal more with sick, low-income patients — fared poorly in CMS ratings.

“On its surface that says, ‘Hey, you’ve got a bias,’” said Silvers. “Big teaching hospitals are taking care of the most complex cases. Hardly any of them are in the five-star category. Most of them are in the one-star category.”     

What’s not in the ratings matters, too

Ratings systems capture huge amounts of information on everything from whether doctors are washing their hands to mortality rates for heart attack patients. But plenty of information is not collected that might help provide a more nuanced — and accurate — assessment of performance. One hospital quality specialist noted, for example, that the severity of a cancer patient’s disease is not typically recorded by hospitals in diagnostic information and doesn’t get factored into outcome data and ratings. This can unfairly penalize hospitals that treat large numbers of patients with advanced cancers.

“Stage 4 cancer patients are more likely to die than stage 1 cancer patients,” said Harold Miller, chief executive of the Center for Health Care Quality and Payment Reform.  “But if you don’t know that information — and you’re measuring mortality rates — then you are going to get a bad answer.”       

Gaming the system

Hospitals routinely launch initiatives to improve scores in particular ratings categories. They can also change how they parse and report certain information to improve their results. In some cases, those adjustments can mean positive improvements in the care provided to patients. In others, they can skew the results or cause hospitals to avoid treating some patients, altogether.

For example, a recent study by the American College of Surgeons found that organ transplant centers are removing the sickest patients from their waiting lists because of concerns that a negative outcome could harm their ratings and lead to federal scrutiny.

“You may decide not to take patients who are at risk of bad outcomes if somebody’s going to start rating you on your outcomes without adjusting properly for that,” Miller said. “Those are the patients that need the hospitals the most, and they are the ones that are going to potentially get disadvantaged by this.”

The marketing agenda

Hospital ratings — particularly the results published by U.S. News & World Report — are often used for hospital advertising campaigns. Most institutions ranked in the top 10 will hype their ratings success on billboards, banners, and brochures that get blasted across the regions they serve.

But the glossy mass media messages offer little valuable information to consumers trying to figure out where to get treatment for a broken leg or have gall stones removed because common procedures are seldom evaluated. “It could be the hospital is bad on some things, and really good on other things,” Miller noted. “But if you’re the patient, how do you know? You might be going to the hospital for something it’s really good at, but you can’t always tell that from the ratings.”

Leave a Comment

Please enter your name.
Please enter a comment.

  • Concerning the reduction in size of Kidney Transplant lists at different major transplant centers, I have been monitoring those numbers over the last 6 to 7 years.

    I have a fair amount of experience on these issues:

    1. I lost my kidneys, thru no fault of my own, when my kidneys were accidentally destroyed by my surgeon during a “Laporascopic Stomach Banding procedure;

    2. As a result I was on kidney dialysis for almost 5 years — a procedure that is universally described by people on dialysis as “Better Than Dying — BUT JUST BEARLY BETTER — and about 10% of dialysis patients decide death is better, and refuse further dialysis and die a totally painless death over the course of 1 to 2 weeks;

    3. After almost 5 years on dialysis, I obtained a kidney transplant, and it was an amazing transformation. It was LIKE BEING REBORN. In the first 8 hours, I pee’d out 2 liters (2.2 quarts) of the darkest, smelliest urine you could ever imagine. And within 12 hour I was my old self again — my mind woke up — my sense of humor came roaring back — my coloring returned to normal after almost 5 years — and I was again HAPPY TO BE ALIVE — which was often missing during my time on dialysis !!! I HAD BEEN REBORN !!!!!

    But I digress…Back to major Transplant Centers knocking approved kidney transplant recipients off their Waiting List in order to IMPROVE THEIR HOSPITAL RATINGS..

    For much of that time I have studied Transplsnt Centers, several Transplant Centers had Wait Lists of between 3,000 and 6,000 patients waiting to get a kidney transplant, although they were only transplanting kidneys into 100 to 300 recipients per year.

    Although they would lose some patients to “death” or “transfer to other transplant center,” the number of new patients seeking to be added to their Wait Lists, more than made up for the “Losses” the their “Wait Lists.”

    Then suddenly the size of all these massive “Wait Lists” suddenly dropped to 1,000 to 2,000 potential Recipients on their Wait Lists.

    I have been trying to figure out what force of nature or humankind had cause this sudden and massive drop in the Wait List size of these major Transplant Centers.

    Now I finally know !!!

    These poor Potential Transplant Patients, who had been waiting for years while on dialysis for years for their chance to finally obtain a LIFE-SAVING kidney transplant, suddenly had their HOPES OF GETTING their new, life-saving kidney transplant were suddenly SMASHED, and with total disregard for what it would mean to the hopeful recipient, disqualified” as a Potential Kidney Transplant Recipient so the these major Transplant Centers could “phony up” the number they had to report — by “REMOVING THE SICKEST PATIENTS,” [who had already been fully tested, certified, and approved as “healthy and safe” to receive a kidney transplant by their Transplant Center], FROM THEIR WAITING LIST — JUST SO THAT THESE TRANSPLANT CENTERS COULD HOPE TO UP THEIR “RATING SCORES” TO HELP THEM “BETTER MARKET THEIR HOSPITAL…”

    Whatever happened to the Hippocratic Oath???

  • Finally, someone is covering this subject. Business has really fluffed up the image of some of our countries leading hospitals; if some took closer looks at the malpractice rate or number of litigation cases annually, staff turn over (particularly the temporary/contract hires), or even (compulsory but anonymous) patience opinion polls regarding treatment, I’m sure their mouths would drop to the floor.

  • Do you remember how the housing bubble and lack of regulatory oversight triggered a global financial collapse during the Bush administration? Central to the problem was how RATING AGENCIES like Moody’s and Standard & Poors gave top AAA ratings to even the riskiest loan pools. Relying on untrustworthy ratings is what doomed AIG, and the same can happen in healthcare if ratings are not to be trusted.

    These days consumers are more likely to use the Internet and Google to search for the procedure they plan to have — “Best knee replacement surgeon in Texas” — and then compare several different rating systems that may even include Yelp and Angie’s List. While not necessarily the best way to search, it’s pretty reliable because of how Google ranks hits. And while it’s still possible to game that system too, it tends to be self-healing, again because of how Google adjusts its ranking algorithms to favor content that people find most valuable.

Sign up for our Morning Rounds newsletter

Your daily dose of news in health and medicine.

X