Suppose you see a headline that says something like: \u201c50% of our 100,000 new Covid cases were among the vaccinated\u201d? Should you be concerned that the vaccine isn\u2019t working anymore? The answer is: absolutely not – well, not without a lot more information. This statement is an example of using numbers to confuse rather than illuminate. And the best way to understand that this is almost certainly a totally meaningless statistic,\u00a0 perhaps even rising to the level of complete BS,\u00a0 is to use a technique I\u2019ve explained before – think about what a statement would mean at extremes.\u00a0<\/p>\n\n\n\n

So here is an extreme situation to use to think about this statement. Imagine someone is publishing this \u201cstatistic\u201d about a place where, say, 99% of a 20,000,000 population were vaccinated and yes they had 100,000 new cases they were reporting on. The \u201cstatistic\u201d in our headline is saying that, of the 100,000 new cases, 50,000 of them were in the vaccinated population (50% of 100,000 cases), and so 50,000 were in unvaccinated people.<\/p>\n\n\n\n

So, first off, we can calculate the total number of unvaccinated people is 200,000 (1% of 20mil) and 19,800,000 people were vaccinated (our 99% vaccinated rate => .99*20mil vaccinated people). Then, the odds of getting sick if you are vaccinated is:<\/p>\n\n\n\n

50,000\/19,800,000 or about .0025 =1\/4%<\/p>\n\n\n\n

I.e. really low. But if you are unvaccinated the odds are:<\/p>\n\n\n\n

50,000\/200,000 = 25% <\/p>\n\n\n\n

or 100X greater and really really high. <\/p>\n\n\n\n

Thus, for this hypothetical example, you would know that someone is deliberately trying to confuse you or is simply unaware of the effect choosing the wrong size for the bottom of the fraction (the denominator) has on percentages!  <\/p>\n\n\n\n

If the denominator you chose in a calculation is the wrong one, you have fallen victim to what is called \u201cthe base rate fallacy<\/em>.\u201d In this case, the \u201cstatistic\u201d used the total number of cases of covid (100,000) as the denominator, not the total number of vaccinated people (19,800,000). You simply can not divide by 200,000 to find out the odds of getting the disease if you are vaccinated, because your \u201cpopulation\u201d size of vaccinated people is 19,800,000 not 200,000.\u00a0 And, when you divide by 200,000 when you are supposed to divide by 19,800,000\u00a0 – well you saw the result above, you are off by about 100 fold!\u00a0.<\/p>\n\n\n\n

Base rate fallacies come up all the time in thinking about medical statistics. They are, for example, at the root of the \u201cparadox of the false positive.\u201d which I talked about before (https:\/\/garycornell.com\/2020\/05\/28\/testing-4-i-tested-positive-do-i-really-have-the-disease\/). Recall that having a positive test result for a disease isn\u2019t enough data to make a decision – you need to know how rare the disease is in a population.  <\/p>\n\n\n\n

To sum up:<\/p>\n\n\n\n

Any statement about \u201codds\u201d or \u201cprobability\u201d is meaningful only when you know they have used the right size of the sample to divide by. <\/em>Denominators matter!<\/em><\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"

Suppose you see a headline that says something like: \u201c50% of our 100,000 new Covid cases were among the vaccinated\u201d? Should you be concerned that the vaccine isn\u2019t working anymore? The answer is: absolutely not – well, not without a lot more information. This statement is an example of using numbers to confuse rather than … Continue reading “The Base Rate Fallacy: X% of new Covid cases are among the vaccinated is a BS statement”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[6],"tags":[],"class_list":["post-682","post","type-post","status-publish","format-standard","hentry","category-the-pandemic"],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/posts\/682","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/comments?post=682"}],"version-history":[{"count":1,"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/posts\/682\/revisions"}],"predecessor-version":[{"id":683,"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/posts\/682\/revisions\/683"}],"wp:attachment":[{"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/media?parent=682"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/categories?post=682"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/garycornell.com\/wp-json\/wp\/v2\/tags?post=682"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}