The Science Journalism Tracker, from the Knight science writing program at MIT, offers helpful analysis. Yesterday, it took on a New York Times story on the Dartmouth Atlas. The Times report cited take-downs of the program’s mapping of health care costs and practice pattern variations.
The maps “appeared to show where the waste in the system could be found. Beige meant hospitals and regions that offered good, efficient care; chocolate meant bad and inefficient.
The maps made reform seem relatively easy to many in Congress, some of whom demanded the administration simply trim the money Medicare pays to hospitals and doctors in the brown zones. The administration promised to seriously consider doing just that.
But while the research compiled in the Dartmouth Atlas of Health Care has been widely interpreted as showing the country’s best and worst care, the Dartmouth researchers themselves acknowledged in interviews that in fact it mainly shows the varying costs of care in the government’s Medicare program. Measures of the quality of care are not part of the formula.
For all anyone knows, patients could be dying in far greater numbers in hospitals in the beige regions than hospitals in the brown ones, and Dartmouth’s maps would not pick up that difference. As any shopper knows, cheaper does not always mean better.
Here’s what the science trackers had to say:
Their bias is evident from the start. They describe the study in the lede as coming from “a once obscure research group” at Dartmouth. You see how clever that is? Without actually pointing to any problems with the study, they suggest that there’s something fishy about it. How important could this study be if it comes from an obscure research group?
Actually, Elliott Fisher, one of the authors of the Dartmouth atlas who is quoted lower in the story, is one of the nation’s leading health-care economists. I’m sure he was obscure once (wasn’t everybody?), but he’s not anymore, and he hasn’t been for a long time.
The once obscure Abelson and Harris then note that the study has been “widely interpreted” as showing the country’s best and worst care, but that all it does is show the variations in cost. Here’s the problem with that observation: Showing the difference in costs is precisely what the study’s authors say it does. Fisher says so in the story!
Fisher does, however, admit that both he and others have miscast the findings at times.
No doubt in the truth that cheaper is not always better, as the Times story says. But it is also true that more-is-better has been the mantra of both patients and provider for years. The result: massive waste and poor care. While cost may be a crude measure, it is an important red flag. Studies like these need to control for as much as possible and recognize the limits of the findings.
Gooz News call is an “important debate.”