However, when you look at the key statistical table (\u201cTable 8: Subgroup Analyses of Second Primary Endpoint: First COVID-19 Occurrence From 7 Days After Dose 2, by Subgroup) things get murkier. More precisely, what one sees is exactly what I thought would happen, the signal becomes really bad for people over 65 and completely useless for people over 75. Here\u2019s an excerpt of that table, and then I will try to explain what is going on:<\/p>\n\n\n\n What I need to explain is how to think about what that \u201c95% CI\u201d in the last column means and why it is so important. \u201cCI\u201d stands for confidence interval <\/em>and is the key when a statistician looks at data and tries to tease out signal from noise. The ideas behind a confidence interval are simple, although how to define it precisely and then calculate it, is a bit tricky. <\/p>\n\n\n\n In a nutshell when we pull a single number from a bunch of measurements – whether it is the average weight of what’s in a bunch of boxes of cereal or how effective a vaccine is – we know that number isn\u2019t going to be perfect. So what we want and, well, should<\/em> do is not focus on that single number but give a range around that number and then ask when, say, the odds on average are that 19\/20 times that we are within that range i.e. what happens if we do the experiment repeatedly<\/em>. When we do this with a range you get what statisticians call a\u00a0 \u201c95% confidence interval<\/span>1<\/sup><\/a><\/span>\u201d. The more data you have, the tighter you can make your confidence interval!<\/p>\n\n\n\n So now let’s look at some individual lines from the table above and tease out just what the signal is. The major line is for people 18-64 and we had enough cases to say that our 94.6% efficacy average for the vaccine has a 95% confidence interval runs from 89.1 to 97.7. So what the biostatisicians who analyzed the data are telling us is that, roughly speaking, if we bet that this vaccine is between 89.1% and 97.7% effective for this group, this is an awfully good way to bet and we will win 95% of the time. These are astonishingly good numbers and we all have a lot to be thankful for. (Although having data by age deciles would have been better, they don\u2019t have enough data to do that even in this bigger group I suspect.)<\/p>\n\n\n\n But then we have the next two lines and they unfortunately, confirm what I wrote about here (https:\/\/garycornell.com\/2020\/10\/22\/we-are-unlikely-to-have-a-vaccine-that-is-proven-effective-for-seniors-for-a-long-time-unless-dramatic-action-is-taken-now\/<\/a>). For people 65 to 74, while the average number (92.9%) looks great, the confidence interval is not. It says that what we can say, roughly speaking, that a bet that the efficacy is between 53.2 to 99.8 is a good bet. Or, I would say you really don\u2019t have a great way to bet. This kind of confidence interval says that didn\u2019t have enough cases in this group to really say much at all and so the confidence range is too large to be really useful. <\/p>\n\n\n\n And when we get to people over 75, what they describe isn\u2019t a confidence interval, it\u2019s a joke. A confidence interval of -12.1 to 100 is a lot like saying they threw a bunch of darts at a dart board at random and did everything from hit bystanders (i.e. the vaccine made things worse) to perfect protection. Who would make any bets on what is going on in this situation?They simply didn\u2019t have enough cases to say anything meaningful and so what they say is just totally useless.<\/p>\n\n\n\n But I don\u2019t want to end on a depressing note! My friends who think about these questions feel pretty strongly that while the vaccine will likely be less effective in people over 65 than it is in younger people, the dropoff won\u2019t be great enough to make a big difference. For example, if it is 20-25% less effective in these age groups (which they think is the worst case scenario), you still get a vaccine that is roughly between 70% and 75% effective – which is still pretty darn good. <\/p>\n\n\n\n Still I wish they had enrolled enough people >65 to have a better signal!<\/p>\n","protected":false},"excerpt":{"rendered":" Both the UK and the FDA have released enough information so that one can make a good bet on how the Pfizer vaccine worked. (See https:\/\/www.fda.gov\/media\/144245\/download for example). It makes for fascinating and informative reading. I am not competent to comment on the medical aspects described there other than to say that when I first … Continue reading “Statistics in the Pfizer Data – how good do they show the vaccine to be?”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":""},"categories":[6],"tags":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/posts\/595"}],"collection":[{"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/comments?post=595"}],"version-history":[{"count":24,"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/posts\/595\/revisions"}],"predecessor-version":[{"id":621,"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/posts\/595\/revisions\/621"}],"wp:attachment":[{"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/media?parent=595"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/categories?post=595"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/tags?post=595"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"\"\/](\"https:\/\/lh6.googleusercontent.com\/oiO6gYom1UZyrOhgpFx2iq8ike979u3805JHiVygP-Efh1Yaz2ttyPcgWKlT1AqHDM4v46th3EPIkOvRLyXA0fNUloPL-mL9eOFmSAzfbNOHyCZSQ0DyzMhcFUtQuZ520R5Qd2lj\")