Author: admn

It’s “Turtles all the way down: ” an amazing bargain on Terry Pratchett’s DISKWORLD novels.

Not my usual topic of course but Humble Bundle has the most amazing bargain until the end of January on 39(!) Diskworld Novels:

https://www.humblebundle.com/books/terry-pratchetts-discworld-harpercollins-books

the novels fall into what you might describe as the comic fantasy genre and if you like that genre, this is an amazing bargain on some of the best books in this genre.

So what is Diskworld? “it’s a flat planet balanced on the backs of four elephants which in turn stand on the back of a giant turtle.” The books cleverly parody many traditional tropes in fantasy and, how can you not like a series where a hero named “Cohen the Barbarian” pops up from time to time?

Finally, it’s important to always remember that, as this well known story tells us, it’s “turtles all the way down”.

“A well-known scientist (some say it was Bertrand Russell) once gave a public lecture on astronomy. He described how the earth orbits around the sun and how the sun, in turn, orbits around the center of a vast collection of stars called our galaxy. At the end of the lecture, a little old lady at the back of the room got up and said: “What you have told us is rubbish. The world is really a flat plate supported on the back of a giant tortoise.” The scientist gave a superior smile before replying, “What is the tortoise standing on?” “You’re very clever, young man, very clever,” said the old lady. “But it’s turtles all the way down!”

The number needed to treat: A better way to explain efficacy

In my previous blog entry, I tried to explain why absolute risk reductions was the right number to look at. The trouble is absolute reduction is expressed as a percentage and people generally hate percentages and usually need to do some further mental gymnastics to process the information. So statisticians in the 1980’s came up with another way to look at absolute risk reduction, it’s called the number needed to treat (usually abbreviated NNT).  NNT is actually pretty easy to understand. For example, suppose  a treatment cures everyone treated, well then the NNT is 1. One person, one treatment, one cure. Similarly, if when you treat two people, one  is cured, the NNT is 2. If you need to treat 3 people to cure one, the NNT is 3 and so on. 

But amusingly enough, all you need to do to get the NNT is basically flip over the absolute risk reduction percentage. For example, if the absolute risk reduction is 1%, that means 100 people needed to be treated to cure one. This means   the NNT is 100 but that is exactly:

1/1%

Of course, since fractional people are kind of weird, NNT is always rounded up to the next integer. So for example if the absolute risk reduction is 6%, the number needed to treat would mathematically be 1/6% or 16.6666, but we say the NNT in this case is 17.  

Yes, NNT gets more complicated if a treatment could conceivably harm some people and help others, but in most cases, just flipping over the absolute risk reduction and rounding up gives you the  NNT.

While I suggested in my previous blog that pharmaceutical companies should be required to give absolute risk reduction whenever they give relative risk reduction in an advertisement, requiring them to add the NNT in the same font etc. might even be a better idea!

The Semaglutide (Wegovy) Clinical Trial: Or How headline numbers mess with both Doctors and Patients minds

The recent SELECT study broadcast that using Wegovy size (2.4mg/weekly) dosages of Semaglutide for three years would reduce the risk of cardiovascular men events among some pretty seriously ill people by 20%. Unfortunately, when you dig deeper, what you discover is that what the study really showed is that if you treated 1,000 of these people for three years, you would reduce the number of cardiovascular events by 15 events. Moreover, at current list prices, this reduction of 15 events would cost almost 50 million dollars. How could a true headline number be so different from the actual reality?

Well, it all started with a press release way back on August 8, 2023 that yes heralded “a 20% reduction in cardiovascular events in non-diabetic overweight people with pre-existing cardiovascular or peripheral artery disease. Heck 68% of them already had a heart attack and they were 62 years old on average. These were not heathy people. And so the press release made it seem like using Semaglutide at a Wegovy-like dose of 2.4mg injected each week would be a game changer. Of course, us math types were eagerly awaiting the underlying numbers because press releases too often confuse the issue. They do this by not telling you all of the numbers and only giving you the numbers that make the trial look as good as possible.

When the actual trial numbers just came out (https://www.nejm.org/doi/full/10.1056/NEJMoa2307563), as many of us expected, this was a wonderful example of why “headline numbers” from clinical trials in a press release need to be viewed with suspicion. The problem is headline numbers are always about “relative risk reduction.” And relative risk reduction, while occasionally a useful statistic, is almost always a pretty small piece of the puzzle. So, let me first explain what relative risk reduction is and why it doesn’t tell you anything about how few events you prevent?

To understand why, let me give you an exaggerated example. Imagine a drug company tells you that our (very expensive) wonder drug reduces the risk of death by 50% in a fairly common disease. Sounds great, no? Then you dig a little further and discover if 10,000 people have the disease only two die if left untreated, with the wonder drug only one dies. Yep 50% reduction, no lies here. And then you dig a little deeper and find out the drug is likely to put you in the hospital, make 10% of the people who use it deaf, ruin the kidneys on 10% more etc., etc. Now you may be thinking is all that worth it for saving one life? That is a hard question, but in any case, congratulations you have just discovered “absolute risk reduction.” That’s the reduction in the actual number of events. And, as you have also just figured out, absolute risk reduction is a much better number to focus on than relative risk reduction.

More precisely, relative risk reduction measures how a treatment works in the treated group versus the group that got the placebo. It’s a pure percentage. Pure percentages like this are not tied to actual numbers of events: they are derived from the events and hide the number of events basically.

Let’s try another example: suppose your trial has 2000 patients, 1,000 got the treatment and 1,000 got a placebo. In the treated group, you had 8 events and the untreated group 10 events. That’s a 20% relative risk reduction, since 2/10 is 20%. But absolute risk reduction looks at the actual cases relative to the size of the groups. In other words, it takes into account the fact that most people in a trial don’t have any “events” at all. In our example, it’s an absolute risk reduction of only 2 individuals and as a percentage, that’s really small:

10/1000 – 8/1000 = 2/1000 =.2%

In other words: a 20% relative versus .2% absolute risk reduction – which is 1/100 of the relative risk reduction and wouldn’t make such a great headline number.

Obviously, I and many other math types think we would all benefit if drug companies were forbidden from broadcasting relative risk reduction without an equal emphasis on absolute reduction in their press releases or advertisements. Headline number with a high relative risk reduction uses the well know phenomenon of “anchoring” (https://en.wikipedia.org/wiki/Anchoring_effect) to mess with people’s minds!

O.K. what about the SELECT trial? It was a big trial of 17,604  patients, 45 or older and they all had preexisting cardiovascular disease. They also were overweight, with a body-mass index of 27 or greater. They may have been pre-diabetic but they were not yet diabetic. The trial lasted a little over 3 years. First off, the trial used what is called a “composite” endpoint: death from cardiovascular causes, a nonfatal heart attack, or a nonfatal stroke. Math types would automatically tell themselves: trials to detect combined events are easier to get significant results out of than trials for individual events. Testing for individual events need bigger and longer trials. For example, in the SELECT trial while deaths were reduced, they weren’t reduced enough, as we math types would say, to be ”statistically significant.” Also noteworthy, was that 17% of the participants in the Semaglutide groups dropped out of the study, roughly twice as many as in the placebo group. This was presumably because of the well known side effects to these drugs.

The results were as follows: there were 8803 in the the Semaglutide group. A cardiovascular “event” happened in 569 of the 8803 patients (6.5%). The placebo group had 8801 people, 701 of the 8801 patients had an event (8.0%).  This means the absolute risk reduction was about 1.5%. And yes the relative risk reduction was about 20%. But please note: the relative risk reduction was 16 times the absolute risk reduction!

Although I am not a doctor, I think it is wrong to call this a game changer: A 1.5% lowering in risk is small after all. But yes, it is obviously significant because this was a high risk group after all. Equally obviously, many cardiologists are excited because this is a pill that changes the risk for some pretty sick people, but I have to ask: how much of this excitement is due to the anchoring phenomenon of highlighting a 20% relative risk reduction? After all, as the accompanying editorial to the paper in the New England Journal of Medicine titled:” “SELECTing Treatments for Cardiovascular Disease— Obesity in the Spotlight” made it clear, we have no way of knowing if the effect of this drug comes from people losing 10% of their body weight. And of course the connection between weight loss and lowering the risk of cardiovascular events is well known: cardiologists have been telling patients like this to try to lose weight for basically forever. We simply don’t yet know if there are effects on cardiovascular health from Semaglutide over and above the weight loss it causes. And we do know that Semaglutide often leads to muscle loss and lower bone density.

But more to the point, while one would certainly like to have these drugs available to these patients, if the price of these drugs doesn’t come down, you can make a good argument their cost will literally break Medicare. Why? Well, roughly 35% of Medicare patients are overweight or obese. Roughly 75% of people over 65 have coronary artery disease. (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6616540/#:~:text=The%20American%20Heart%20Association%20(AHA,age%20of%2080%20%5B3%5D.)

Even if you assume these numbers are independent of each other, which is very unlikely, because overweight people are more likely to have cardiovascular disease, this means at least 26% of the roughly 31 million Medicare recipients (and likely more) might benefit from these drugs. We have an eligible population for these drugs therefore of more than 8 million people. At current list prices we would be spending 8mil*1350*12 or about 130 billion dollars a year. Total Medicare spending is about 725 billion a year. So, Medicare spending would go up by more than 17% overnight and stay at that level for a long time to come.

(Note added: My Slate article has more on the economics of just how much reducing Obesity saves using the NHANES data and data from a seminal PLOS article: https://slate.com/technology/2023/07/ozempic-costs-a-lot-it-doesnt-have-to.html)

Another slate article on Sensitivity and Specificity

https://slate.com/technology/2022/01/rapid-testing-covid-math-false-negatives-sensitivity.html

But they cut my draft down dramatically. I wanted to add a discussion of what is called “positive predictive value” (PPV) i.e. the answer to the question “if you test positive, do you have the disease?”

If a disease is relatively rare (alas not Omicron) even a positive result with a very specific test can be very misleading and unfortunately can confuse your doctor (https://www.nejm.org/doi/pdf/10.1056/NEJM197811022991808).

Here’s an example of what can go wrong. Suppose you have a test that is 99% specific.  If you read my Slate article you now know this means it only has 1% false positives. That is a pretty good test in the real world. But suppose the disease is also really rare, say only 1 in 1,000 people have it. Then it turns out that a positive test, even though the test is pretty darn good, isn’t telling you as much as you think! Why? Well, suppose you test 1,000 people. Since we are assuming the disease prevalence is only 1 in a 1000, you had only one person with the disease in your group of a 1,000. Now our test is 99% specific so there are 1% false positives. So among our 1,000 people, you will have about 10 false positives (.01*1000=10 although technically it is .01*9,999 =9.99). The 10 false positives dwarf the 1 true positive and the odds of you not having the disease are 10 to 1

(A clear treatment of PPV in the context of Covid here: https://www.someweekendreading.blog/weekend-editrix-exposed/)

A very, very very fast spreading disease, severe outcomes take longer to develop – what happens to reported rates of hospitalization?

As a precaution against what will surely be misreported by the press, here is a thought experiment with some numbers.

The UK is reporting a doubling time of 2 days. Suppose there are 100 Omicron cases now,  then in 2 weeks we would have 12,800 people testing positive on a given day if everyone was tested. (Seven doublings means multiply by 2*2*2*2*2*2*2 = 128 = 2^7 . ) Suppose, as may very be true, that Omicron is substantially less virulent than Delta. Let’s assume a hospitalization rate of say 1% instead of Delta’s 2.3% which will happen 2 weeks after a positive test. (Hospitalizations lag positive tests by about 2 weeks for all versions of Covid so far). Then, from our initial 100 cases, there would only be one hospitalization in the two weeks that follow because 1% of a 100 is 1!

But what does this mean? Should we say that Omicron doesn’t seem to result in any significant hospitalizations? Of course not, the signal is hidden in the noise. But I betcha that we can expect some of the more innumerate press to say that Omicron has “a vanishingly small chance of being hospitalized”. After all, 1/12,800 is pretty darn small – it just isn’t what is going on.

The Base Rate Fallacy: X% of new Covid cases are among the vaccinated is a BS statement

Suppose you see a headline that says something like: “50% of our 100,000 new Covid cases were among the vaccinated”? Should you be concerned that the vaccine isn’t working anymore? The answer is: absolutely not – well, not without a lot more information. This statement is an example of using numbers to confuse rather than illuminate. And the best way to understand that this is almost certainly a totally meaningless statistic,  perhaps even rising to the level of complete BS,  is to use a technique I’ve explained before – think about what a statement would mean at extremes. 

So here is an extreme situation to use to think about this statement. Imagine someone is publishing this “statistic” about a place where, say, 99% of a 20,000,000 population were vaccinated and yes they had 100,000 new cases they were reporting on. The “statistic” in our headline is saying that, of the 100,000 new cases, 50,000 of them were in the vaccinated population (50% of 100,000 cases), and so 50,000 were in unvaccinated people.

So, first off, we can calculate the total number of unvaccinated people is 200,000 (1% of 20mil) and 19,800,000 people were vaccinated (our 99% vaccinated rate => .99*20mil vaccinated people). Then, the odds of getting sick if you are vaccinated is:

50,000/19,800,000 or about .0025 =1/4%

I.e. really low. But if you are unvaccinated the odds are:

50,000/200,000 = 25% 

or 100X greater and really really high. 

Thus, for this hypothetical example, you would know that someone is deliberately trying to confuse you or is simply unaware of the effect choosing the wrong size for the bottom of the fraction (the denominator) has on percentages!  

If the denominator you chose in a calculation is the wrong one, you have fallen victim to what is called “the base rate fallacy.” In this case, the “statistic” used the total number of cases of covid (100,000) as the denominator, not the total number of vaccinated people (19,800,000). You simply can not divide by 200,000 to find out the odds of getting the disease if you are vaccinated, because your “population” size of vaccinated people is 19,800,000 not 200,000.  And, when you divide by 200,000 when you are supposed to divide by 19,800,000  – well you saw the result above, you are off by about 100 fold! .

Base rate fallacies come up all the time in thinking about medical statistics. They are, for example, at the root of the “paradox of the false positive.” which I talked about before (https://garycornell.com/2020/05/28/testing-4-i-tested-positive-do-i-really-have-the-disease/). Recall that having a positive test result for a disease isn’t enough data to make a decision – you need to know how rare the disease is in a population.  

To sum up:

Any statement about “odds” or “probability” is meaningful only when you know they have used the right size of the sample to divide by. Denominators matter!

When will we get to herd immunity?

I haven’t written about the pandemic in a while because, well, we have vaccines that work pretty damn well – even against the incredibly contagious delta variant. People just need to get vaccinated. I could do a post every day that just repeats that 500 times I suppose.

But I was talking to someone and they asked just how bad the delta variant could be for the United States. First off, what is absolutely clear is that:

Delta is so contagious that, until we get to herd immunity, if you don’t have some sort of immunity or don’t take strong precautions, i.e. N95 masks, social distancing, you will catch it. 

So the most important question is when we will reach herd immunity? That’s actually not an easy question to answer and what answer you get depends on the model you use for herd immunity. And all the models depend on questions we don’t yet have complete answers to,  for example: how rare is it that a vaccinated person gets reinfected and if they do get reinfected, how likely will it be that they can transmit it? Similarly, if a person already had a version of Covid and isn’t vaccinated, how likely are they to get reinfected and then transmit delta?  And you can also ask: how likely is a child under 12 who catches delta to transmit the virus etc. The point is, the number of groups you can use and how they transmit delta in your model can grow, and then the model becomes very complicated. At that point, large-scale computer simulations are often the best way to get an answer for your model. 

But there is some reason to believe that the naive model for calculating herd immunity I discussed here https://garycornell.com/2020/10/27/herd-immunity-1/ using an R0 of roughly 7 (https://www.thelancet.com/pdfs/journals/lanres/PIIS2213-2600(21)00328-3.pdf) will work pretty well. One can see for example how well it matches up with the Institute for Health Metrics projections which are based on very sophisticated computer simulations. 

Using the model I described in my blog and an R0 of 7, we need that 85.71% (1-1/7) of the population to be immune – to not be transmitting the virus to other people – before herd immunity kicks in.  So when will 85.71% of the population not be transmitting the virus? 

I want to explain how one might get a handle on this number in the rest of this blog. I’m going to make the following simplifying assumption:

  • I will assume that herd immunity happens when 85.71% of the population over 12 is vaccinated or has had a version of Covid.

This assumes vaccinated people and people who have had Covid are not contributing significantly to the transmission of delta and the transmission from children under 12 also isn’t significant to blocking herd immunity.  If these assumptions are false and people in these groups do contribute to transmission significantly, it will make herd immunity happen much later, but based on what I have read so far, current thinking seems to be that this is unlikely.

There are about 329 million people in the United States and about 280 million of them are over the age of 12.

So since 85.71% of the 280 million people over 12 is about 240 million (.8571*240mil=239,988.000), we have to find out when 240 million people are not transmitting it to the remaining 40 million people over 12.  

According to the CDC, as I write this, about 185 million people 12 and older have received at least one shot and about 161 million are fully vaccinated. I’m going to assume therefore that we can take 185 million people out of the equation. Let call these people category “A”.  Category A lets us remove a lot of people from our 240 million goal – if only it were more. 

Our goal shrinks to:

240mil – A = 240mill- 185mil = 55 million

So we are down to a goal of 55 million more people being or becoming immune before we get to herd immunity.

This 55 million people goal is made up of two groups in our model. Those who have already had Covid and those who are vulnerable and will get it in the months to come. 

To analyze this number, we need to first figure out how many people have gotten Covid and aren’t vaccinated. Let’s call the number of people that are 12 and older, aren’t vaccinated, but have been infected by Covid, B.  This means the number of people who will get sick going forward before we get to our  goal of herd immunity is:

55mill – B

 Let’s call this number “V” for vulnerable.

V = 55mill – B

Now we get to the joys of modeling. I have searched for good information on how many people are in group B (have gotten Covid but aren’t vaccinated), but have come up short. There just doesn’t seem to be any good numbers on the size of group B. 

But all is not lost: there are good estimates on the total number of people who have been infected by Covid, we just don’t know how to distribute them between groups A (vaccinated) and B (unvaccinated). 

The best estimates I have seen are that between two and three times the number of people who have tested positive (roughly 33mil) actually have had COVID. This means it is reasonable to assume between 67 and 100 million people in the United States have had a version of Covid. Let’s be as optimistic as possible and assume that 100 million people over the age of 12 have had some version of Covid. 

But we still don’t know how to split these 100 million people between groups A and B is. We have to do this because if they are in group A we have already removed them from the equation, we don’t want to count them twice! How do we proceed?  Here’s what we are assuming:

  • The “odds” of having been infected with Covid if you are over 12 is 

100mil/280mill = .357 

So, of the 185 million people in our vaccinated group A, we will assume 35.7% of them have already had Covid:

.357*A = .357*185mil  = about 66 million people in group A have had Covid

(Yes, I know that people in group A probably took better precautions, or got vaccinated before they could catch Covid, so their infection rates are lower than group B’s, but you can change this number to take this into account if you want.)

The rest of these 100 million people are exactly the people who have had Covid but aren’t vaccinated i.e. group B. So 

B = 100mil – 66 million

This means that, with our assumptions, group B has about 34 million people!

So now let’s calculate V – the people who will get sick from delta before we get to herd immunity with our assumptions.  In our model, since V is equal to:

V = 238mil – A –  B  

or

V = 238mil – 185mil – 34mill  

so

V = 55mil – 34mil = 21mil

Our model predicts 21 million more people over 12 will get delta before we get to herd immunity! 

Let’s check our simple model against the very sophisticated Institute for Health Metrics (IHME) model which goes until November 2021 (https://covid19.healthdata.org/global). Their model predicts there will be another 50,000 deaths by November 1st and since the death rate is roughly 1/10 of the hospitalization rate, their projections imply 500,000 or so hospitalizations (https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2778237) by November 1. 

Now compare this to the result of our analysis. What we got was 21 million more people who have no immunity will get sick from delta before we get to herd immunity. This implies that there will be roughly 610,000 hospitalizations from delta (best knowledge is that 3% of those infected are hospitalized)  and 61,000 more deaths before herd immunity kicks in (using the current knowledge that mortality is 1% of hospitalized cases). This number is consistent with the IHME numbers and probably means our model, simple though it may be, is realistic. Also, if you believe the IHME model and this analysis, we probably won’t get to herd immunity until the beginning of 2022. And, alas if our analysis is right:

hospitals in areas with low vaccination rates i.e. where most of the people in group V live, will not just be overwhelmed by sick patients, they will break completely under the burden. 

Feel free to make your own assumptions in this model and change the values for the variables accordingly. But I believe this isn’t a bad model and it gives a good picture, how &^%$ things are going to get in the United States because of the number of people over 12 who are unvaccinated.