Introduction

A saline-placebo controlled trial is widely assumed to provide a more accurate assessment of a vaccine’s safety and effectiveness than trials using an active comparator or antibody measurements alone. That assumption is reasonable as far as it goes. A saline-placebo trial is the most rigorous design available in vaccine research and genuinely rare in flu vaccine development. The majority of flu vaccines were approved on antibody levels alone, or compared against another vaccine rather than an inert control.

But more rigorous is not the same as sufficient. What follows is a close reading of a saline-placebo controlled trial, examining what it actually showed, what it did not and could not show, and where its key finding no longer applies.

The trial “Efficacy, immunogenicity, and safety of a quadrivalent inactivated influenza vaccine in children aged 6–35 months: A multi-season randomized placebo-controlled trial in the Northern and Southern Hemispheres” by Pepin et al. (2019), tested VaxigripTetra, a Sanofi quadrivalent influenza vaccine, in healthy children aged 6 to 35 months. VaxigripTetra is not marketed in the United States; it is approved in the European Union, South Korea, Taiwan, and other markets across Asia and Latin America, including Israel. The full study and its regulatory record are publicly available.

About the trial

• The manufacturer, Sanofi, was responsible for every stage of the trial: it funded it, designed it, ran the statistics in-house, paid the medical writer, decided to publish, and approved the final version.
• It primarily aimed to determine whether its quadrivalent vaccine, already approved for those 3 years and older, would be efficacious for children aged 6 to 35 months.¹ “The primary objective was to demonstrate efficacy against influenza illness caused by any strain or vaccine-similar strains [abstract].”
• It included 3 vaccines: the quadrivalent vaccine being tested (2 A strains and 2 B strains – B/Yamagata and B/Victoria lineages), a trivalent vaccine containing the standard licensed trivalent vaccine with the recommended Yamagata strain, and a trivalent vaccine made specifically for the trial as a comparator for the Victoria strain. Although the Victoria comparator was taken as a clean baseline, that assumption was never tested.
• The trial, using a saline placebo, ran from March 2014 to July 2016 over four influenza seasons.
• The investigators declared that “A major strength of this study was its representativeness. It was a large study conducted over a wide geographical area in both hemispheres.” However, per-country enrollment, which appears only in the registry, reveals that about half the 5,805 children (2,999) in the study were recruited in the Philippines. A child growing up there has a different nutritional baseline, different infectious disease exposure, different immune history, and different genetic background than a child in Israel, the United States, or Europe. All of these factors affect how a vaccine performs and how the body responds to it. A trial conducted predominantly in one population tells you how the vaccine performed there. It does not tell you how it is likely to perform in yours.

Regarding efficacy

• Efficacy was assessed for the IIV4 (four-strain inactivated influenza vaccine) and placebo arms. Immunogenicity (antibody titers) was assessed for all children, including those in the two IIV3 (three-strain) comparator arms. However, the paper states that there is no established correlate of protection for children of this age (no antibody level that reliably predicts whether a child is protected). Raising antibodies is not proof of protection, and trial efficacy (performance under controlled study conditions with healthy, screened children) is not the same as real-world effectiveness.
• The authors concluded that the vaccine halved the incidence of mild flu among trial participants for any strain type: 50.98% and 68.4% for vaccine-similar strains. However, this applies only to relative risks, not absolute risks. The table below shows what the numbers look like in absolute terms:

* NNT = Number Needed to Treat: the number of children who must be vaccinated to prevent one case of mild confirmed flu. Both figures apply to mild illness only, not hospitalization or serious disease. The matched-strain NNT of 48 is higher than the any-strain NNT of 20, which may seem counterintuitive. You would expect the vaccine to perform best against the strains it was specifically designed to match — and in relative terms it does (68.4% vs 51%). But the absolute benefit is smaller because matched-strain flu was rarer in this population during these seasons: the placebo attack rate for matched strains was only 3.05%, compared to 9.84% for any strain. Fewer cases to prevent means a smaller absolute reduction, regardless of how effective the vaccine is at preventing them. This is precisely why relative and absolute figures tell different stories, and why the relative headline can be misleading.

• Overall efficacy was shown, but against the B/Victoria strain, it came up short on both measures: efficacy was not statistically demonstrated, and the antibody response was lower than the comparator. The paper itself acknowledges this:

“Efficacy of IIV4 against the B/Victoria strain will have to be established in further studies.”

Yet the conclusion claims additional protection from the extra B strain regardless:

“By including a second B-lineage strain, IIV4 should provide additional protection beyond IIV3, irrespective of which B lineage circulates during a given season or region.”²

• Furthermore, the abstract’s conclusion uses the word “effective” rather than “efficacious,” a meaningful distinction: a controlled trial measures efficacy under study conditions; effectiveness is a real-world claim that requires observational evidence this trial did not generate.

Regarding safety

• Safety and reactogenicity (reactions to the injection): The investigators concluded that both arms were similar except for injection-site reactions:

“… except for a higher proportion of participants reporting solicited injection-site reactions in the IIV4 group (39.9% [95% CI, 37.5–42.4%]) than in the placebo group (31.9% [95% CI, 29.6–34.2%]), proportions reporting solicited reactions and adverse events were similar for the IIV4, IIV3, and placebo groups.”

However, consider the following:

• Solicited and unsolicited reactions: “Solicited reactions were collected by parents and legal guardians, and unsolicited adverse events were recorded by investigators.” This means that for unsolicited adverse events (the category meant to catch anything unexpected), a parent first had to notice something, decide it was serious enough to contact the trial site, and an investigator then had to decide that it was worth recording. It does not provide a tabulation of how many events occurred but of how many events were actually recorded.
• Monitoring windows: The monitoring windows were short: solicited reactions for 7 days, unsolicited events for 28 days, and serious adverse events for 6 months. No long-term follow-up was planned, as stated explicitly in the trial registry. Post-marketing safety surveillance, if any, would be passive pharmacovigilance (adverse event reports submitted by practitioners after the fact), not active follow-up of trial participants.
• Deaths: Both arms enrolled similar numbers of healthy children, yet there were 5 deaths in the vaccine arm per the registry submission and only one in the placebo arm. The published paper reported only four, not five. Four of the vaccine-arm deaths were attributed to infectious causes: aspiration pneumonia, gastroenteritis, meningitis, and septic shock; the fifth was a poisoning. The single placebo death was listed as “sudden death.” Sanofi judged none of the deaths vaccine-related but did not explain the unequal distribution.³
• Hospitalization: The trial could show no benefit for severe disease: hospitalizations were equal, three in each arm, far too few to detect a difference either way. Hospitalization is the outcome that matters most for a parent weighing this decision, and the trial was never sized to address it.
• Adverse Events of Special Interest (AESIs): The paper listed eight AESIs to be monitored, including Guillain-Barré syndrome (GBS), anaphylaxis, and encephalitis.⁴ It reports only a total count: 29 vaccine recipients and 31 placebo recipients experienced at least one AESI, with no breakdown by category. It cannot be determined from the paper⁵ or the registry whether any cases of GBS, anaphylaxis, or encephalitis actually occurred, or whether any of the other listed AESIs ever occurred. Two events that do appear in the registry are absent from the paper entirely: one case of Bell’s Palsy and one case of Kawasaki’s disease, both in the vaccine arm, both zero in the placebo.

Trial stopped midway

• The trial was stopped at the first interim analysis, a pre-planned check of the data partway through the trial, once efficacy was demonstrated, at 5,806 children enrolled instead of the planned 8,536 (about 68% of the intended population). Stopping a trial early at its most favorable result is a known source of inflated efficacy estimates. Early stopping also cut the safety database by roughly 2,700 children. Rare adverse events that would have appeared in that population were never captured.

What this analysis shows

A saline placebo is a legitimate standard to ask for. Acknowledging that this trial used one is not a concession that ends the inquiry. It is the beginning of it. The vaccine reduced mild, confirmed flu by roughly half in healthy screened children during those specific seasons, against strains circulating at the time. The strain it succeeded against most clearly is now extinct. The strain it failed against is the only B lineage left. Safety monitoring ran for days to months, with no long-term follow-up, with eight specific adverse events of concern aggregated into a single undifferentiated number, and with the safety data from the trial’s largest cohort never published (see note 4). There is a wide gap between what was tested and what “safe and effective” is taken to mean.

One broader question this analysis raises is what peer review actually catches. The B/Victoria overstatement, claiming protection irrespective of which B lineage circulates in a paper that failed to demonstrate it for B/Victoria, passed peer review. The discrepancy between the registry’s five vaccine-arm deaths and the paper’s four, passed peer review. The AESI totals reported without per-category breakdown, the absence of the Asian cohort’s supplementary data, and the use of “effective” where the result supports only “efficacious,” all of it passed peer review.

Peer review did not catch these things because it is not designed to catch them. Peer reviewers do not necessarily see the raw trial data.⁶ They check whether the methods are described coherently and the statistics are internally consistent. Reviewers are not required to cross-reference the registry, verify the death count, or flag a conclusion that is not consistent with the data. That work falls to whoever reads the paper carefully enough to do it. The registry was publicly available for nine months before the paper was even submitted.⁷