It’s not all that often that mortality statistics make the news (yes, once a year when the CDC releases annual data, but that’s not the same thing.) And it’s really unusual when the news is about the reporting surrounding the mortality rate itself.
Those who accuse CNN and other mainstream media outlets of “fake news” will probably revel in a recent decision by a federal judge in Atlanta, Georgia. While Judge Orinda Evans didn’t all out declare that CNN was peddling in falsehoods, she did take aim at the network in an initial judgment in favor of a former hospital CEO who sued CNN accusing them of purposely skewing statistics to reflect poorly on a West Palm Beach hospital. Judge Evans didn’t mince words in her 18-page order allowing the case to move forward, and dismissing CNN’s attempt to get it thrown out of court.
Davide Carbone, former CEO of St. Mary’s Medical Center in West Palm Beach, filed a defamation lawsuit against CNN after they aired what he claims were a “series of false and defamatory news reports” regarding the infant mortality rate at the hospital. CNN’s report said the mortality rate was three times the national average. However, Mr. Carbone contends that CNN “intentionally” manipulated statistics to bolster their report. He also claims that CNN purposely ignored information that would look favorable to the hospital in order to sensationalize the story.
Hmmm, let us look into this.
WHAT THE JUDGE WROTE
Let’s take a look at the court doc:
Here are a few things I will pull out with respect to the case – note: these are screenshots of the document, as I don’t have the time to do OCR and fix the mishmash that results.
So first, if I want to look up the CNN articles, they’re starting June 2015.
Truth is an absolute defense is on point – you will see that CNN is claiming the stats they reported are correct!
And what’s interesting, the plaintiff agrees that the raw data are correct. But that the comparisons being made are misleading.
So the stats – 6/48 = 1/8 = 12.5% are true from a raw basis.
Where’s the beef?
RISK-ADJUSTED MORTALITY RATES
But here’s the deal – if one is doing open-heart surgery on babies… that’s inherently risky as a procedure, and some of those babies are going to be in a much worse way than others.
Cry alert: This recent post on the death of a 7-month-old after heart surgery gives you an idea — I will excerpt one part:
Everyone was in shock. We had the best team of pediatric heart surgeons, cardiologists, NICU and PICU nurses that you could assemble in America. Rebecca had been recovering. Her echocardiograms had all been good. The pacing wires had been firing. Everything should have worked. It was like the A Team of cardiology teams was on her side. They simply don’t lose people, certainly not kids like Rebecca. But as the head surgeon later told us, “One minute she was fine, the next she was in arrest and would not come back.”
There are different kinds of heart surgeries babies may need, with some being relatively minor, and some like Rebecca’s, being more extensive. She had had open heart surgery, and open heart surgery is serious at any age, but particularly difficult for babies.
That’s why you need to look at what kind of patients/surgeries are being done if you want to compare mortality rates or even comparing how good non-fatal results are.
I don’t know the details of that particular hospital, but the problem is if all their pediatric heart surgeries are on babies in bad condition, needing open heart surgeries — you would expect them to have higher mortality rates, just as a result.
You don’t want to penalize doctors and surgeons for caring for a high risk pool of patients — getting no care at all is obviously worse for such patients. These babies who have high mortality rates in open heart surgery probably have a close to 100% 1-year mortality rates without surgery. There are likely less dire conditions where they can do closed heart surgery, and the mortality that results is not nearly so high.
So the way you make fair comparisons is to do some kind of adjustment for risk — this is done with regular mortality rates all the time. For example, this data set of county-level mortality rates by cause over time has age-adjusted mortality rates so you can compare trends.
If they didn’t do the age-adjustment of mortality rates, you might be mainly comparing that one county in 1980 had a lot of old folks in it and one county in 2010 had a bunch on youngsters. Old people have relatively high rates of dying of pneumonia; youngsters have relatively high rates of dying violent deaths (suicide, homicide, car accidents). You don’t want to bias the data so that you’re looking at age trends instead of actual mortality trends.
Now, when you collapse high-dimensional info like death distribution over various ages, you lose some insight from the data, but it does make it easier to make comparisons on other dimensions, like noting if a specified population not reduced by risk-adjustment is showing disparities in mortality. Then one can dig more into the age-related aspect. That’s what has happened in looking at the alcohol/drug overdose mortality spikes among white people, later focused down to middle-aged white people without college degrees. But that’s for another time.
CNN KNEW ABOUT RISK-ADJUSTED MORTALITY RATES
Risk-adjusted rates for mortality after surgery is more complicated, and I checked out the quality measures at the Society of Thoracic Surgeons. There sure are a lot of them there.
Let me pull a relevant one:
Congenital Heart Surgery Measures
Operative Mortality Stratified by the Five STS-EACTS Mortality Categories
NQF # 0733
Date of Endorsement
Operative mortality stratified by the five STS-EACTS Mortality Levels, a multi-institutional validated complexity stratification tool
Number of patients who undergo pediatric and congenital open heart surgery and die during either of the following two time intervals: 1.) Prior to hospital discharge 2.) Within 30 days of the date of surgery
Number of index cardiac operations in each level of complexity stratification using the five STS-EACTS Mortality Levels, a multi-institutional validated complexity stratification tool
Any operation that is not a pediatric or congenital Cardiac Operation. Cardiac operations are defined as operations that are of operation types of “CPB” or “No CPB Cardiovascular” (CPB is cardiopulmonary bypass.) .
Any operation that is a pediatric or congenital open heart surgery (operation types of “CPB” or “No CPB Cardiovascular”) that cannot be classified into a level of complexity by the five STS-EACTS Mortality Levels.
1.Jacobs JP, Mavroudis C, Jacobs ML, Maruszewski B, Tchervenkov CI, Lacour-Gayet FG, Clarke DR, Yeh T, Walters HL 3rd, Kurosawa H, Stellin G, Ebels T, Elliott MJ. What is Operative Mortality? Defining Death in a Surgical Registry Database: A Report from the STS Congenital Database Task Force and the Joint EACTS-STS Congenital Database Committee. The Annals of Thoracic Surgery, 81(5):1937-41, May 2006.
The adjustment being used for the STS rate is far more complicated than simply taking the number of deaths and dividing by the number of operations.
Oh, the numerator is the same, but the denominator is adjusted. CNN did absolutely no adjustments, and compared the raw 12.5% with an adjusted number. And they knew it.
From the court doc:
FWIW, the plaintiff has not yet won his case, and CNN and the plaintiff could end up settling out of court. This ruling simply allows him to continue with his lawsuit.
But it will be interesting if the reporting standard becomes that media may find themselves on the receiving end of defamation/libel lawsuits if they deliberately use misleading statistics.
Not that I think it would do much — when the point is to gin up a big story out of nothing, because you have nothing substantive to report on… well I don’t think that’s going to change anything.
Also, actuaries have their own problems with communications.
Mortality Monday: How Many People Will Die in the U.S. in 2017?
Mortality Monday (with a little Trumpery): Supreme Court Probabilities Take 2
Mortality Monday: Suspicious Russian Deaths