I was just about to write a blog about how insane this claim is. If anything, the numbers show that the AstraZeneca vaccine lowers the risk of blood clots but the great Weekend Editor beat me to it. So, even if you have to skip the great statistical stuff in his blog, just read his great post, please!
Author: admn
If we need even more reasons to go to one dose for the next few months!
https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(21)00432-3/fulltext
Pretty convincing evidence waiting 90 days between 1st and 2nd doses…
Move to a single dose now!
I get so mad when the people in charge don’t seem to do the obvious logical reasoning from the facts. But it is actually often even worse that that. Too often, even if they do know what logic requires, they won’t follow through on the conclusions that those facts and logic implied. For the latest case in point, consider the following:
- We now have some really good evidence that a single dose of mRNA vaccines convey really good immunity (https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(21)00448-7/fulltext) (even if it may be short-lived unless people get a booster shot).
- The more people get some protection, the fewer the number of mutations escape into the wild. (Common sense but always worth quoting Fauci: “Viruses don’t mutate unless they replicate, And if you can suppress that by a very good vaccine campaign, then you could actually avoid this deleterious effect that you might get from the mutations.”)1
- We will have a lot more doses in a few more months. Actually, we will have more than enough mRNA doses to vaccinate everyone in the United States (and then some) by the end of the summer or early fall with massive increases coming starting in May. (And that doesn’t count new vaccines coming down the pike like Novavax’s!)
Ergo, why the ^%&$ aren’t we moving to a single dose of the mRNA vaccines now, save giving the second dose for when we have no supply constraints and, of course, completely stopping the idiotic reserving of any second doses if that is still being done?
Reserve the J&J Vaccine for people who are more likely to engage in high risk activities
We now have a one-dose vaccine that we are confident is reasonably effective on younger people while, as usual, being somewhat less confident that it is effective on older people. Moreover, not only does the J&J vaccine require only one dose, it has no fancy requirements for transporting it. Even more, reasoning by analogy with the similar Oxford/Astra Zenica vaccine, it may reduce transmission significantly. So what should we do with it?
Here’s a modest proposal and before you dismiss it as crazy, note that it is actually backed by some really interesting mathematical models of disease transmission and some really good empirical evidence on contacts between groups as well as data on how Covid-19 is transmitted by different age groups.
Use the J&J vaccine to vaccinate people more likely to engage in riskier behavior or less likely to get a second vaccine and start doing this immediately once the J&J vaccine is approved. More generally, reserve it for people under 50.
Here’s one way to implement this idea: every day pick a city with a reasonably large airport. Show up at the airport with a “swat” team and vaccinate everyone who passes through it with the J&J vaccine. Do it for the bus and train terminal in the same city on the same day if there is one. Extend the idea by showing up in front of bars and restaurants if they are open in that city and offer to vaccinate everyone in that bar or restaurant. Do this by mobilizing the national guard and the commissioned corps of the public health service – think of it as analogous to a military mission to “secure” a city1.
Picking the city at random helps a little to prevent people from gaming the system I suppose. But that isn’t really the point. We shouldn’t care if people game the system. There are actually two points to keep in mind. The first point is that since the J&J vaccine doesn’t require any fancy storage capabilities, it’s certainly practical. The needed logistics are well within the capabilities of the national guard and the public health service.
The second point is the key though – it’s because it is the best way to break the back of the pandemic by greatly reducing transmission rates. Why? Because some really good models of disease transmission predict that lowering the infection rates among people who are more likely to transmit the disease is the best way to break the back of a pandemic! If you think about it for a second, you probably don’t need any fancy mathematics: this clearly works by lowering transmission rates quickly. So, yes, I really am advocating giving the people likely to engage in risky – even stupid – behavior the J&J vaccine and not giving it to people who might be at higher risk. Save the Moderna and Pfizer vaccines for high-risk people of course but don’t give them the J&J vaccine even if it takes longer to vaccinate the high risk population as a result of this choice.
So why is this a really good idea from the point of view of turning the pandemic around in the quickest possible way? Well, it is certainly reasonable to conjecture these kinds of people are less likely to show up for the second dose of the Moderna or Pfizer vaccine, but that isn’t actually the reason to act quickly to vaccinate such people with the one-dose J&J vaccine. The real reason to vaccinate them goes back to mathematical models that were developed around 12 years ago. One of the best was done by Jan Medlock and Alison P. Galvani and was published in Science (Science 25 Sep 2009: Vol. 325, Issue 5948, pp. 1705-1708 DOI: 10.1126/science.1175570) but a far better treatment of their ideas may be found in Medlock’s powerpoint presentation here:http://people.oregonstate.edu/~medlockj/other/flu.pdf. (You do need some knowledge of differential equations though.) And, in case you are wondering if this mathematical treatment leads to a result that is way too theoretical and not backed by “real” evidence, Mossong et al (https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0050074) showed: “ a consistent pattern of contact frequency by age, with a gradual rise in the number of contacts in children, a peak among 10- to 19-y-olds, followed by a fall to a lower plateau in adults until the age of 50 and a sharp decrease after that age. And, while I suppose one can argue that Mossong et al is too old to trust fully, a recent paper (https://science.sciencemag.org/content/sci/early/2021/02/01/science.abe8372.full.pdf) showed that 65% of Covid-19 infections came from people between the ages of 20-49 and concluded that “Targeting interventions – including transmission-blocking vaccines – to adults aged 20-49 is an important consideration in halting resurgent epidemics and preventing COVID-19-attributable deaths.”
So let’s start by using the, easily administered, one dose J&J vaccine on the people most likely to spread the disease and more generally people who are less than 50 while reserving the Moderna and Pfizer vaccine for people at higher risk! (If we ever run through those people, we can use the J&J vaccine for people who already have been infected by Covid of course: https://www.medrxiv.org/content/10.1101/2021.02.05.21251182v1.)
Bad %^$# happens a lot i.e. why a “vaccine side effect” probably isn’t one
As I write this more than the equivalent of a 9/11 catastrophe happened yesterday (12/10). It’s horrible and it’s going to get worse. The head of the CDC predicts this level of deaths for the next 60-90 days and even the conservative model used by the IHME says it is likely we will have more than 500,000 deaths in the United States by April 1.
And yet, there is light at the end of this dark, dark tunnel: we are about to roll out a massive vaccination effort based on what can only be described as one of the greatest triumphs of modern science. We have two vaccines that are based on a new technique that will be applicable to many viruses, not just SAR-Covid 19. If widely adopted, these (and other vaccines that are coming soon) will stop the horror. Granted not fast enough, but it will happen and could (should?) be completed by the end of the 3rd or very early in the 4th quarter of 2021.
Unless people don’t get it.
The problem is that surveys show many people will be reluctant to get the vaccine. Yes, it seems that the vaccine will probably make you feel awfully crappy for 48 hours after the second shot, but that isn’t the only problem. People worry about really bad things happening because of the vaccine. But the problem is that it is hard for people to understand that random ^%$& happens a lot. They say: “Oh my friend’s father got this vaccine and had a stroke two days later.” Or, I just saw on the news that some healthy 35 year old had a stroke a week after getting the vaccine.”
They confuse correlation with causation. Why? Well unfortunately people of all ages get strokes and if you are vaccinating millions of people some of them will get strokes within a few days of getting the vaccine. How many? We can actually calculate roughly how many! From https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3250269/: “Among adults ages 35 to 44, the incidence of stroke is 30 to 120 of 100,000 per year, and for those ages 65 to 74, the incidence is 670 to 970 of 100,000 per year over 75 years”
So, if we use these numbers and take the midpoint for people 35-44, about 75 people per 100,000 who are 35-44 will have a stroke in a year. There are about 45 million people 35-44 in the United States. That means there will be about 75*(45million/100,000) = 33,750 strokes in people between 35-44 in the United States or almost 100 a day. These strokes have nothing to do with a person getting a vaccine. And the rate of strokes in people over 75 is about 10 times higher. Because there are about 35 million people over 75 in the United States, we would expect about (820*35million/100,000) or about 800 strokes a day that have nothing to do with a vaccine.
So please, please keep in mind when hearing anecdotes about side effects from these miracle vaccines that bad ^&%$ happens randomly a lot.
I want to end by showing you a table for background rates on a lot of bad ^%&$ you might see as being “caused” by these vaccines-even terrible ones like death. Every time you hear an anecdote about some bad side effect after a Covid 19 vaccine please think about this table (taken from:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2861912/) . For example, within one week after vaccinating 10,000,000 people, you will likely have around 98 people keel over and die for no apparent reason and if all of them were pregnant women, almost 27,800 miscarriages.
Predicted numbers of coincident, temporally associated events after a single dose of a hypothetical vaccine, based upon background incidence rates
Number of coincident events since a vaccine dose | Baseline rate used for estimate | ||||
Within 1 day | Within 7 days | Within 6 weeks | |||
Guillain-Barré syndrome (per 10 million vaccinated people) | 0·51 | 3·58 | 21·50 | 1·87 per 100 000 person-years (all ages; UK Health Protection Agency data) | |
Optic neuritis (per 10 million female vaccinees) | 2·05 | 1440 | 86·30 | 7·5 per 100 000 person-years in US females (table 2)16 | |
Spontaneous abortions (per 1 million vaccinated pregnant women) | 397 | 2780 | 16 684 | Based on data from the UK (12% of pregnancies)34 | |
Sudden death within 1h of onset of any symptoms (per 10 million vaccinated people) | 014 | 0·98 | 575 | Based upon UK background rate of 0·5 per 100 000 person-years (table 2)28 |
Statistics in the Pfizer Data – how good do they show the vaccine to be?
Both the UK and the FDA have released enough information so that one can make a good bet on how the Pfizer vaccine worked. (See https://www.fda.gov/media/144245/download for example). It makes for fascinating and informative reading. I am not competent to comment on the medical aspects described there other than to say that when I first started reading about possible vaccines many months ago, I never found any virologist who predicted we would have a vaccine that was more than than 70% effective. To have a vaccine that is likely about 95% effective for people 18-64 is nothing short of a medical miracle: we really lucked out.
However, when you look at the key statistical table (“Table 8: Subgroup Analyses of Second Primary Endpoint: First COVID-19 Occurrence From 7 Days After Dose 2, by Subgroup) things get murkier. More precisely, what one sees is exactly what I thought would happen, the signal becomes really bad for people over 65 and completely useless for people over 75. Here’s an excerpt of that table, and then I will try to explain what is going on:
What I need to explain is how to think about what that “95% CI” in the last column means and why it is so important. “CI” stands for confidence interval and is the key when a statistician looks at data and tries to tease out signal from noise. The ideas behind a confidence interval are simple, although how to define it precisely and then calculate it, is a bit tricky.
In a nutshell when we pull a single number from a bunch of measurements – whether it is the average weight of what’s in a bunch of boxes of cereal or how effective a vaccine is – we know that number isn’t going to be perfect. So what we want and, well, should do is not focus on that single number but give a range around that number and then ask when, say, the odds on average are that 19/20 times that we are within that range i.e. what happens if we do the experiment repeatedly. When we do this with a range you get what statisticians call a “95% confidence interval1”. The more data you have, the tighter you can make your confidence interval!
So now let’s look at some individual lines from the table above and tease out just what the signal is. The major line is for people 18-64 and we had enough cases to say that our 94.6% efficacy average for the vaccine has a 95% confidence interval runs from 89.1 to 97.7. So what the biostatisicians who analyzed the data are telling us is that, roughly speaking, if we bet that this vaccine is between 89.1% and 97.7% effective for this group, this is an awfully good way to bet and we will win 95% of the time. These are astonishingly good numbers and we all have a lot to be thankful for. (Although having data by age deciles would have been better, they don’t have enough data to do that even in this bigger group I suspect.)
But then we have the next two lines and they unfortunately, confirm what I wrote about here (https://garycornell.com/2020/10/22/we-are-unlikely-to-have-a-vaccine-that-is-proven-effective-for-seniors-for-a-long-time-unless-dramatic-action-is-taken-now/). For people 65 to 74, while the average number (92.9%) looks great, the confidence interval is not. It says that what we can say, roughly speaking, that a bet that the efficacy is between 53.2 to 99.8 is a good bet. Or, I would say you really don’t have a great way to bet. This kind of confidence interval says that didn’t have enough cases in this group to really say much at all and so the confidence range is too large to be really useful.
And when we get to people over 75, what they describe isn’t a confidence interval, it’s a joke. A confidence interval of -12.1 to 100 is a lot like saying they threw a bunch of darts at a dart board at random and did everything from hit bystanders (i.e. the vaccine made things worse) to perfect protection. Who would make any bets on what is going on in this situation?They simply didn’t have enough cases to say anything meaningful and so what they say is just totally useless.
But I don’t want to end on a depressing note! My friends who think about these questions feel pretty strongly that while the vaccine will likely be less effective in people over 65 than it is in younger people, the dropoff won’t be great enough to make a big difference. For example, if it is 20-25% less effective in these age groups (which they think is the worst case scenario), you still get a vaccine that is roughly between 70% and 75% effective – which is still pretty darn good.
Still I wish they had enrolled enough people >65 to have a better signal!
Obviously great great news but we aren’t home free yet
Efficacy in the 18-55 year old group much higher than expected! That augers well for seniors even without efficacy data. But we are a long way from home free. Issues to keep in mind:
- Only 25 million people (50 million doses/2) worldwide can be treated in the first few months. Only 500 million people worldwide in the first year (1 billion doses/2).
- It’s an extremely difficult vaccine to transport: https://garycornell.com/2020/08/30/back-of-envelope-calculation-the-number-and-the-costs-of-freezers-needed-for-the-pfizer-vaccine/
- Duration of immunity completely up in the air as is the effect on preventing serious cases and deaths – and again no real info on seniors for the reasons I ‘ve written about at length. But no reason not to be hopeful.
- Long term safety day won’t be available for a while – probably mid 2021. First responders will be participants in one of the largest safety aka Phase 4 trials ever. But again there is no reason not to be hopeful.
But it couldn’t be better news, everything I have read indicated that people were hoping for a 70% efficacy signal at best, Pfizer got a 90% signal!
Why did the polls get it so wrong?
Another election, another massive screw up by the “best” pollsters in the business. Yes, excuses will be made about how margins of error are only 1/20 and while rare 1/20 events do happen occasionally (it’s roughly like getting two pair in a five card poker hand on your initial deal), they really don’t happen twice in a row.
So any attempts to ascribe this to “margins of error” is complete BS. Clearly these polls are not (and didn’t in 2016) use random samples. The magic didn’t happen because the groundwork wasn’t there. No random samples, no magic.
Here’s another way of thinking about this. If they did use random samples in each election, you could also safely assume the polling result errors in each election are independent events. Which means by the “multiplication of probabilities” result I have talked about before, the odds of two screwups in a row would be >= 1/400 and that just isn’t believable.
So something clearly is wrong in their methodology: they aren’t getting random samples. You can come up with conjectures as to why. For example, a common method to get random samples in polls is to do random dialing of phone numbers. If a certain group is always more likely to hang up on you than another group, randomness ain’t happening. You can attempt to weigh your results to take this into effect, and pollsters certainly claimed they would do that after the 2016 debacle, but whatever they added to their secret sauce in 2020 just made the dish taste worse, not better – that is for sure.
My conclusion after this year’s debacle: absent some methodological innovation, political polling isn’t worth paying much attention to going forward and so neither is the effort put into it – by a lot of really smart people.
Just what is a “margin of error” anyway-Sampling 1
You have been probably seeing a lot of polls lately. They all end by saying something like “we sampled 1,000 people and our margin of error is 3.8% or 4.5%” or some other weird percentage. I thought I would take some time to explain where this number comes from and what it means. I want to start by saying that the technical term isn’t “margin of error” but rather “margin of sampling error.” And the keywords are “sampling error” And, although it seems not directly connected to the pandemic, “sampling error” is a fundamental concept in statistics that must come up in dealing with trying to find Covid 19 prevalence for example, so I thought I would take some time to explain it. This post won’t get too much into the math but eventually math will rear its head when discussing sampling error so I will have some future posts that are a little more math centric.
Anyway, statisticians like to talk about a “population” – that’s what you are trying to understand by taking a “sample.” We can’t test everybody in the United States for the antibodies to the virus that causes Covid, so we test a “sample”. From the results of that sample we try to estimate the “true” result – the actual number of people that have caught the disease. For example, suppose we find that 10% of our sample test positive and we “jump” to the conclusion that, heck, probably 10% of the whole population is positive. Are we really jumping?
The answer depends on how the sample was taken! But if it was taken “randomly” – and I will have to have a post on what just that means, it’s actually a tricky concept, the answer is “probably no, we are really not jumping to conclusions” and this is true even if the sample seems so small compared to the actual population size. And yes, it seems magic that a sample size of a 1000 or so allows one to make reasoned judgements about populations in the 100’s of millions i.e. that you can poll 1,000 people and make reasoned judgements on how the 210 million adult population of the United States feels or is.
But it is true. A fundamental result – perhaps the fundamental result in statistics says that the results from relatively small-sized random samples come pretty close to the true result for the whole population under some pretty general and very reasonable assumptions. And this is true no matter how large the population you are sampling. And yes I’ll repeat it: it does seem like magic that it is the sample size rather than say the size of the sample relative to the size of the population is what matters.
In fact, if you take a “random” sample of about 1000 people from the adult population of the United States (about 210 million people), the odds of being off by more than 3% in either direction is roughly speaking 1/20. Go to about 2400 people and then 19/20 times you are within about 1% of the correct answer. All this means is that if you had the time and money to increase the sample size to what still seems ridiculously small relative to the population size, you can make the chance of you being wrong also ridiculously small. So I hope you can see why sampling can be so powerful in determining the hidden occurrence of Covid 19 infections for example and that polling, if done properly can work.
OK as a mathematician I need to say this: mathematics isn’t magic, it just seems that way sometimes. And for what it is worth, if I had to pick a single result in all of mathematics, that any mathematician can understand relatively easily why it is true and yet still have trouble believing it, it is this result.
But I need to reiterate that when looking at the results of any survey: (a)you need to be sure they did a random, unbiased sample, and (b)even if they did that, you need to keep in mind that almost all reported sampling results use a 1/20 chance of being off by more than their “margin of error.” Finally (c)it’s worth keeping in mind that if they did if they did a random unbiased sample, that there is only a very very small chance of them being off by twice their margin of error.
(Technical note: These calculations were done if you are looking at result of a more or less equally split population. The numbers needed would change slightly if you were doing a sample where you had a more extreme split such as (75-25%). But, roughly speaking, the error is proportional to the square root of the sample – and the population doesn’t figure into it!)
So stay tuned for more posts that go deeper into the magic and mystery of how sampling works!
Herd Immunity 1
Herd immunity is all in the news but in many cases the news media is not really describing the concept clearly. While technically the concept of herd immunity derives from mathematical models of disease transmission, when you drill down, it is just generalizing common-sense thinking about disease transmission.
Let’s start by imagining only one person still has a disease and everyone else is immune. Then, after you cure that person, assuming there is no animal reservoir (i.e. animals that can harbor the disease), the disease is over and done with. That’s what happened with smallpox on that glorious day when, on Oct. 26, 1977, in Merka, Somalia the last natural case of Smallpox was diagnosed. As a mathematician would say: “this is the boundary case for herd immunity, there’s simply no one left to transmit disease to”.
More generally, I want you to imagine the following situation: you have a bunch of people who are immune and a bunch of people who are infected. Now also imagine there is a large group of people who are neither infected nor immune, they are the “susceptibles”. OK, an infected person comes into contact with a group of people. Obviously, he or she encounters a certain number of immune people and a certain number of people who are susceptible. Suppose that person infects only one person and subsequently becomes cured. This gives us a steady state (potentially for a very for a long time). Every infected person passes the disease onto one person and gets cured themselves. The disease stays at whatever state it is in – that’s why they call it a “steady state”. Yes, it still infects the susceptible population, but slowly, one person at a time, until there are no more susceptible people. Then and only then does it die out.
But diseases usually don’t work like that. More likely our one infected person infects more than one person: say two people and then, of course, those two people each infect two more people for a total of four. The next round each of those four infect 8 etc. Then you have exponential growth and disaster ensues quickly. The disease will grow uncontrollably and then you’re down the rabbit hole I talked about this in the parable of the Lily Pond (https://garycornell.com/2020/04/20/the-parable-of-the-lily-pond-or-why-you-should-shelter-in-place/) .
Since mathematicians and statisticians have no trouble thinking about “fractional people”, I want you to join us and imagine that one person can only infect half a person. In this fantasy world of fractional people, that half a person will only infect a quarter of a person, the quarter person only an eighth of a person and so on. In this case the disease dies out.
In a nutshell, this is exactly what herd immunity is about: it’s about throwing infected people into a pool of both immune people and susceptible people, having them interact and then thinking about when the disease can no longer spread. More precisely, what the mathematical models try to do is predict when you will be in a situation where the number of immune people is so large that an infected person won’t find one or more susceptible people – even if they succeeded in finding “.99” of a person to infect before they can no longer spread the disease. Once the number of people an infected person infects on average is less than one person, “herd immunity” happens and the disease eventually dies out.
OK now you have seen what it’s all about, let’s try a more realistic situation especially for Covid-19. Imagine a situation where a person will infect at most three people on average. Whether they do or not will obviously depend on the number of susceptible people they encounter.
Let’s walk through a couple of scenarios:
Scenario 1: One infected person encounters two susceptible people, both get the disease, and each of these two infect two out of three more people (i.e. they infect four people in the second level). All hell breaks loose if this scenario continues for too long.
Scenario 2: One infected person always encounters one person who is susceptible and two who are immune. They infect that one person, and that person infects one person and so on. We are in a steady state for )potentially) a long time.
Scenario 3: The infected person doesn’t encounter any susceptible people, he/she gets cured and the disease transmission stops.
Looking over these scenarios, it is clear that scenario 2 is the “tipping point”. In fact, if you allow me my fractional people then, then if more than two out of three people are immune – even “2.01/3” people immune”, then the infected person will infect less than 1 person and the disease will (eventually) die out. Similarly, if there were say less than 2 people, say “1.99/3 people” who were immune (2.01 people who are susceptible) , the infected person would infect more than 1 person and the disease will grow.
Well, we’ve just proved a special case of the most important basic theorem in disease transmission, it’s called the “Threshold Theorem” i.e. the threshold for herd immunity. While the model that led to this theorem is not necessarily the right one in all situations, nonetheless, we’ve essentially proved that if you have a disease where people can infect 3 people on average (R0 is 3 as epidemiologists would say), then herd immunity occurs when more than 2 out of 3 (>⅔, 66.67%) of the people are immune.
Next, imagine a scenario where each person can infect 4 people (R0=4). Can you see why the tipping point occurs if 3 out of every 4 people (75% = ¾) are immune? With everyone infecting 5 people (R0 =5) the tipping point occurs when 4/5ths (80%) of the people are immune etc.
And now I hope you can see that how the general threshold theorem would be proved and here is its statement:
The threshold theorem: In the basic model of herd immunity, herd immunity occurs when more than 1-(1/R0) percent of the people are immune.
Many epidemiologists believe that the threshold theorem with R0 = 3 – i.e. our first situation actually describes COVID-19 transmission pretty well. So in this case herd immunity to COVID requires roughly ⅔ (66.67%) of the population to be immune.
That’s it: you now know more about herd immunity than your average talking head and probably more than your average doctor who doesn’t have an MPh! But I obviously have to end by cautioning you that many epidemiologists want to use different values of R0 in the threshold theorem instead of 3 for Covid-19. For example, an R0 of 2.5 would give (1- 1/2.5 = 60%) an R0 of 4 would give 75%. And, more importantly, other epidemiologists want to use much more sophisticated models of disease transmission and throw out the basic threshold theorem completely when it comes to Covid 19. But no epidemiologist has a model that predicts anything like 20 or 25% as the level for herd immunity as far as I know and we have some facts on the ground that make, say, a radiologist who believes that, an inhabitant of the crazy place in the science landscape.