Toby Rogers knows statistics but not vaccines

• Read original article here

RESPECTFUL INSOLENCE
"A statement of fact cannot be insolent." The miscellaneous ramblings of a surgeon/scientist on medicine, quackery, science, and pseudoscience (and anything else that interests him).
Menu
Antivaccine nonsense Bad science Medicine Skepticism/critical thinking
Toby Rogers knows statistics but not vaccines
Toby Rogers is an economist. He knows multivariate statistics. He doesn’t know epidemiology and pharmacovigilance. Does that stop him from fear mongering about vaccines? You know the answer!
Post author
No Comments on Toby Rogers knows statistics but not vaccines
Toby Rogers is an economist. He thinks he knows autism, vaccines, and epidemiology, "thinks" being the operative word.
It’s odd that I’ve written so little about Toby Rogers , given what I now realize to be his longtime antivaccine prolificacy. Maybe it’s because he’s based in Australia and until recently didn’t have a lot of impact here in the US. Maybe it’s some other reason. I don’t know for sure, but I do know that the one and only time I’ve mentioned him on this blog was last year, when he watched the Advisory Committee on Immunization Practices ( ACIP ) November meeting regarding whether an emergency use authorization (EUA) should be granted for Pfizer ‘s COVID-19 vaccine for children 5-11 years old. As you can probably guess, Toby Rogers was not thrilled, characterizing ACIP meeting as “not a scientific review” but rather “banal bureaucrats announcing plans for a Blitzkrieg and the bought white coats were cheering them on.” I barely mentioned him in my post , didn’t look into who he was, and promptly forgot about him.
Months rolled by, and I totally forgot about Rogers; that is, at least until I somehow had what has to be one of the silliest arguments against the CDC and vaccine safety studies that I’ve ever encountered in my two decades of writing about such things. It turns out that Toby Rogers has a Substack (because of course he does), where he exults :
As censorship has increased on Twitter, FB, and IG, the conversation has moved to Substack. I post my long-form essays here. My intention is to always have a free option alongside opportunities to financially support my work. The beauty of Substack is that all posts go directly to your email inbox. The revolution has begun and I have a lot of say about it.
Here’s a hint. Whenever someone has a Substack whose existence he justifies by invoking “censorship” on other social media, it’s about a 99% likelihood that his Substack will be filled with contrarian nonsense, and so it is with Rogers, arguably the most risible of which is a post from yesterday entitled The CDC’s failure to use multivariate analysis shows the total depravity of the vaccine program . Let’s just say that accusing the CDC of a failure of which it’s not guilty and then accusing it of “total depravity” tell me all I need to know about Toby Rogers.
And yet I still went past the title to read the post. First, he cites Mathew Crawford as having done a “brilliant series of articles” arguing that vaccine efficacy “may be zero.” I might have to do a deeper dive into those articles at some point, but let’s just say that Mathew Crawford is a statistician who appears to misapply statistics to medical questions. For instance, early in the pandemic, he was a big supporter of hydroxychloroquine , even after it had become clear that the drug didn’t work against COVID-19 . Of more interest to me is his bragging right at the outset:
As many of you know, I got a master of public policy degree from UC Berkeley in 2012. What you may not know is that UC Berkeley is usually the top rated quantitative public policy program in the country. So what that means in this case is that it is heavily focused on econometrics.
Econometrics is beautiful. It usually starts with a large data set for a population. Then one uses sophisticated statistical software (STATA, SPSS) to analyze the data. Econometrics involves massive equations that are looking for the effect of a particular variable, while controlling for a wide range of additional variables.
So Toby Rogers knows statistics. Great. Statistics, however, is a tool to be used to set up and analyze the results of research in specific fields in which specific hypotheses are tested and research questions addressed. Does it mean that he understands medical and epidemiological research design? I think you know the answer to that question. It’s also apparent from the introduction to Rogers’ article that he’s making one massive appeal to authority—his authority. In this article, he misapplies that authority in one area, econometrics, to another area, vaccine safety. The only similarity between the two seems to be that statistics are used in both.
It’s interesting to see where Rogers is coming from before launching into his application of his statistics background to epidemiology and pharmacovigilance:
What’s fascinating about econometrics is that if one really builds the model correctly, the largest effect size that one will ever see for a single variable is about 0.3. This means that X intervention explains 30% of the outcome — the rest of the variables explain the rest. We live in a multivariate world. 
Even with all of that complexity, there are still those (such as the great economist  Steve Keen ) who argue that econometrics, with its 15, 50, or even 100 variables is still completely inadequate and that if one really wants to understand how the world works, one must utilize the tools of physics (and supercomputers) to build models with millions of variables and account for things like chaos theory (this approach is called  econophysics ).
I can see physicians and biomedical scientists out there facepalming away, and well you should! Just because in econometrics the largest effect size one will likely see is about one third does not imply that this is true in medicine. While it’s true that many, if not most, effects in medicine detected through epidemiological study are less than 30%, there are a number of examples that are much more striking than that. The effect that smoking has on lung cancer risk, for example, is famously around a ten-fold elevation of risk. Are the epidemiological studies dating back 70+ years that found this effect therefore too simple because the effect is so much more than a 30% increase in risk?
It’s almost as though Rogers approached vaccine safety science from personal incredulity; just because he could not believe that vaccines can have huge effects in preventing disease (e.g., the MMR vaccine series being 95% effective at preventing measles) with an incredible record of safety must mean that there’s something wrong with the science behind vaccines and vaccine safety. Also, he comes from a world without much in the way of constraints. He can add as many variables as he wants to his econometrics models, almost without limit. Medicine doesn’t work like that.
I’ll revisit that last point before this is over, but first let’s take a look at Toby Rogers’ “reasoning,” such as it is. It boils down to, well, I’ll tell you what it boils down to after letting you read it first without my spin on it:
So that was my background when, in 2015, in the midst of a Ph.D. program in political economy, I started researching autism and decided to read a vaccine safety study for the first time. There are about 20 studies that the CDC points to as showing that there is no relationship between vaccines and autism. I assumed that I would not be able to read or understand these studies at all. Given what I knew about the complexity of econometrics, and knowing that the human body, biology, chemistry, and the immune system are even more complex than economics, I assumed that I would be looking at equations involving calculus, that used advanced statistical software to analyze hundreds or perhaps thousands of variables that impact health and disease.
So vaccine safety studies are inadequate because the statistics used in the epidemiology studies looking for adverse events are insufficiently complex compared to econometrics models? I note that Rogers doesn’t actually list the 20 studies or so that the CDC relies on for showing that there is no relationship between vaccines and autism. However, even without such a list, I’ve written about a few of them myself, which makes me laugh at the next passage:
The reality is quite different. Vaccine safety studies tend to be bivariate — they only look at two variables — the vaccine (independent variable) and whether someone suffered an adverse event (dependent variable).
I did a little searching on PubMed and easily found studies using multivariate methodology (e.g., this one from 2004 ) addressing the question of whether vaccines are a risk factor for autism. Looking at the general question of vaccine safety studies, searching PubMed, it’s not hard to find a number of studies using multivariate methodology. No doubt Rogers will dismiss them as not “multivariate enough,” but his argument at its core is silly, as someone with actual knowledge of how epidemiology is done pointed out:
Of the many ridiculous things I've read about vaccines, this is one of the worst. The entire argument is "I know about statistics and this is bad" without ever ONCE even examining numbers or the actual stats https://t.co/id6QqEfhRn
— Health Nerd (@GidMK) August 9, 2022
Also:
— Jean-Paul R. Soucy ???????? ???????? (@JPSoucy) August 9, 2022
He appears to be correct, too. A bivariate analysis does look at only two variables, and arguably there is no vaccine safety study that does that. It is true that such studies do look at whatever condition whose risk factor due to vaccines the investigators want to analyze as a dependent variable and vaccine uptake as the independent variable. (In fact, bivariate analyses are the simplest special case of multivariate analyses in which multiple relations between multiple variables are examined simultaneously.) All vaccine safety studies, though, look at lots of variables, including not just vaccination status with the vaccine(s) in question but anything that might confound the analysis, such as age, timing of vaccination, socioeconomic status, race/ethnicity, and a lot more.
It can get quite complicated:
did someone really bring up this argument?
— JH / Dr Strange Masked (@jhan2qt) August 10, 2022
In the above Tweet, Gideon is, of course, noting a very simple adage in biostatistics: Too many variables are actually often a bad thing in a study. Indeed, in smaller, preliminary studies (say, under 50 subjects) there’s an old adage that if the number of outcomes you’re looking for starts to approach the number of subjects in the study, you have a big problem and need to narrow your net.
Joking aside, the reason for not needing that many variables is, of course, that if you are looking at risk factors for a respecified diagnosis (e.g., autism) for which you hypothesize that a vaccine is a risk factor then you need to control for as many confounding factors as you can that also affect the risk of that outcome. It’s basic epidemiology that confounders matter. Indeed, confounders matter even in simpler designs, and controlling for them can be hideously difficult, regardless of the specific study design.
Of course, antivaxxers frequently like to weaponize uncorrected data; i.e., data that haven’t been adjusted for relevant confounders. We’ve seen this so many times in the past. For instance, one prominent epidemiological CDC study looking at whether the thimerosal used as a preservative in some childhood vaccines was a risk factor for autism started out with unadjusted results that showed a high odds ratio for autism risk based on exposure to thimerosal-containing vaccines, but when appropriate adjustments were made to the data the apparent increased risk of autism went away. Naturally, antivaxxers portrayed adjusting for risk factors as a “coverup” designed to hide the link between mercury-containing thimerosal and autism; indeed, that became the basis of Robert F. Kennedy, Jr.’s Simpsonwood conspiracy theory in 2005 . A similar thing happened with data from an MMR study, in which the unadjusted data showed a correlation between autism risk and MMR, but the adjusted data did not. The result? The CDC whistleblower conspiracy theory immortalized by Andrew Wakefield and Del Bigtree in VAXXED: From Cover-up to Catastrophe, a film so over-the-top in trying to make its claims that, as I wrote at the time, it would have made Leni Riefenstahl cringe.
None of this stops Toby Rogers from ranting:
Here’s the point I want to make: The CDC’s failure to use proper statistical tools (multivariate analysis and beyond) is a threat to national security. But the reason why the CDC relies upon crude bivariate analysis (that is categorically rejected by all fields of scholarly endeavor except vaccine safety studies) is because rigging these studies is the only way to hide the fact that these shots do not work and cause catastrophic harms. If the CDC ever used proper statistical methods, the national vaccine program would come to a screeching halt because the entire program is based on fraud.
This just embarrassing:
When I read a vaccine safety study for the first time, a sickening panic swept over me — “no, no, no, this cannot possibly be. THIS is what the CDC is relying upon!? THIS is what the CDC is using to claim that vaccines are safe!?” Far from being too advanced, these studies are so crude they would fail any Statistics 101 class in any college in America. Tears streamed down my face.
Tears! Did you hear me, tears! Toby Rogers wept (just like Jesus) because he was so upset at the CDC’s chicanery! It’s all fraud to him:
So I read another, and another, and another “vaccine safety” study (eventually reading all 20). And they were all the same. These studies are preposterous because they cannot possibly answer the question they are trying to tackle because their methods are too basic. Vaccine safety studies contain no biology, no chemistry, and vanishingly little statistics. These people are doing arithmetic, badly. In fact, they resemble a card trick more than anything having to do with science. I realized that the CDC has no idea what it is doing — the entire field of pediatrics and public health is a Potemkin Village, a Ponzi scheme, the worst intellectual fraud that I have ever seen.
So a political economist thinks that his statistical methods are far superior because…complexity. As I like to stay, statistical methods are tools, and you need to pick the right tool for the right job. I can’t comment on whether the super-complex models that Rogers brags about in his chosen field of econometrics are the right tools for the right job. I’m insufficiently knowledgeable in the field; so, imbued with what Rogers says about the supposed “humility” of the econometrics models that he touts, I won’t hazard a guess on that front.
I do, however, know biostatistics reasonably well for a non-statistician. I also know epidemiology reasonably well for a non-epidemiologist. Because of that, I know that in epidemiological studies, more variables are not necessarily better. What Rogers seems to be upset about is that, instead of thinking up hundreds of variables to look at, vaccine scientists looking for a link between vaccines and autism (for example) look only at potential confounders known to affect autism risk and, as controls, maybe a few known not to be. Why would they look at others? Another factor is that biomedical science exists in the real world, not in silicon, where one can add whatever variables and factors one wishes. Epidemiology datasets usually don’t have hundreds of variables associated with each case that can be mined.
Indeed, I have experience with one such dataset, a quality improvement database that I helped manage for a few years. Everyone always wanted to add more variables to it, but the consequence was that over the years it had become quite unwieldy and the number of variables that needed to be entered were becoming extremely burdensome to the hospitals who were part of our quality consortium. Toby Rogers is living in a world where he can construct models based on as many variables as as he pleases. Medical science doesn’t work like that because, even though computing power is still increasing rapidly while decreasing in cost, in the real world datasets are not infinitely expandable and infinitely updatable.
The other thing that Rogers apparently doesn’t understand is that adding a bunch of unnecessary variables to a vaccine safety study in the name of doing a “multivariate” analysis won’t just risk falsely decreasing the apparent vaccine efficacy (which he seems to be OK with) but will also ultimately dilute out any safety signals that he wants to find for vaccine injuries and adverse outcomes. Sure, the more variables you compare, the more likely you are to find associations between them, which is no doubt what Rogers, like all antivaxxers, likes, but proper statistical correction almost always eliminates their statistical significance.
Toby Rogers also goes straight to an antivax fallacy:
Vaccine safety studies almost always look at children who have received the full vaccine schedule as compared with children who have received the full schedule +1 more vaccine. And on that basis, they decide whether the 1 additional vaccine is safe. There is no unvaccinated control group. And they do not control for hundreds of other variables that influence health and disease.
Where have I heard this particular argument before about there being “no true unvaccinated control group”? I wonder…
Where have I heard this one before, other than pretty much everywhere in antivaccine conspiracy theories? Indeed, I remember writing about this one going back at least 15 years. It’s an article of faith among antivaxxers that the reason that neither the CDC nor any other reputable scientific organization has found a link between vaccines and autism is because there was never a true “unvaccinated control group.” Indeed, we even call this antivax trope a demand for a true “vaxxed/unvaxxed study,” and antivax doctors and scientists (not to mention quacks like homeopaths ) have been serving up crappy studies based on this premise for years now.
Hilariously, Rogers laments how no one but him seems able to see the errors that he considers so obvious about “bivariate analysis” and characterizes as “categorically rejected by all fields of scholarly endeavor except vaccine safety studies”:
But here’s what I cannot figure out — why do scholars in these other fields, who actually know their stuff, fail to call the CDC out on their chicanery? Even worse, why do scholars in these other fields — who could spot the errors in these “vaccine safety studies” in about 10 minutes if they did any due diligence — poison their own kids and themselves? We live in the dumbest era in human history — we have the tools to know and do better and yet mainstream society still participates in self-inflicted genocide because they just want to fit in.
Obviously, Rogers is so brilliant that only he can see the gaping flaw in every vaccine safety study ever done or cited by the CDC! Only him and no one else, because he’s so brilliant with his multivariate models involving hundreds of economic variables! It never occurs to Rogers that maybe—just maybe—the reason that scholars in other fields don’t call the CDC out for using incorrect statistical models is not because they are cowardly, paid off, or “just want to fit in.” Maybe—just maybe—the reason that these scholars don’t call out the CDC for its vaccine safety studies (or call out the FDA and reputable non-CDC and non-FDA scientists) is because these studies use the right statistical tools for the job, testing hypotheses involving vaccine safety involving whether vaccines work as intended and/or are associated with increased risks of autism or various diseases and adverse outcomes and we don’t need Toby Rogers’ super-fancy big brain econometrics-like analyses of vaccine safety data to test these hypotheses.
I agree that we might live in the dumbest era in human history, but not for the reasons that Toby Rogers thinks that we do. From my perspective, articles like the one Rogers wrote are one indication that we might be living in the dumbest era in human history.
Share this: