A reliable assessment of COVID-19 vaccine protection duration

As vaccine effectiveness wanes over time, it is generally estimated using a standard Cox or Poisson model that assumes constant vaccine effectiveness over time. However, this is less precise over short time periods, and it indicates the effectiveness of vaccines fairly slowly.

Researchers from the University of North Carolina, University of Washington, and the US Food and Drug Administration propose fitting a Cox model with two time indexes, the event times measured from the start of the study in calendar time and the log hazard for the vaccine effect.

The research can be found on the medRxiv* preprint server.

Study: Reliably Assessing Duration of Protection for COVID-19 Vaccines. Image Credit: LookerStudio / Shutterstock

The Study

The researchers simulated a clinical trial mimicking the enrolment pattern of a BNT162b2 study and the trend of severe acute respiratory syndrome coronavirus 2 (SARS‑CoV‑2) infection occurring in the United States while the trial was undergoing.

They assumed the corresponding vaccine efficiency (VE) on the hazard rate (VEHR), which typically represents an exponential deterioration of vaccine effect by time since vaccination, of the hypothetical vaccine decreases from a peak of 95% at 7 days post-second dose to 70% at six months post-vaccination. The means of the VE averaged over time (VEConst) over 1,000 replicates are 94.4%, 89.9% and 81.6% over 0-2, 2-4 and 4-6 months, respectively.

The underestimation of the true level of waning by VEConst is made even more apparent as vaccinations typically coincide with an early peak in the incidence of infections, and then this incidence rate wanes for months afterward. This leads to a high percentage of exposures during the first part of each two-month interval when the true VE is higher than VEConst suggests.

A second trial was simulated in which the enrolment period was shifted to six months later, ensuring that the period with the most substantial vaccine effects coincided with the lowest point in exposure rates. The means of VEConst over 1000 replicates in this trial were 94.7%, 89.5%, and 78.8%, once again over 0-2, 2-4 and 4-6 months, respectively. VEConst is much closer to true VE in this second trial, but in both cases is significantly less close to the truth than the VEHR curve.

It is possible that neutralizing antibodies that confer short-term protection could wane log-linearly, which would lead to the waning of VE over several months. However, post-vaccination VE could be maintained at a plateau for a long time due to cell-mediated or memory immune responses remaining near enough constant over time. To investigate this, the scientists simulated another two trials in which the VE was allowed to reach a plateau at 5 and 3.5 months post full vaccination. In the first, five month trial, VE is overestimated by VEConst and underestimated by VEHR. In the second trial, both estimates provide reasonable approximations of true VE. However, VEHR allows more rapid detection of non-linear changes in VE over time, which are only detectable with VEConst over a more extended period of time.

Estimation of vaccine efficacy against symptomatic COVID-19 based on 6 months of follow-up in four simulated clinical trials. In the first two trials, the true VEHR (“truth”) decreases (linearly in the log hazard ratio) from a peak of 95% at full vaccination that lasts one month to 70% at 6 months after full vaccination. In the trial depicted in panel A, most participants received dose 2 at a calendar time coinciding with a peak in infection rates, whereas in the trial depicted in panel B, most participants received dose 2 at a time of low infection rates. In the trials depicted in panels C and D, the true VEHR plateaus at 5 and 3.5 months, respectively. In each trial, VEConst is obtained over 0–2 months, 2–4 months and 4–6 months post full vaccination, and VEHR is estimated under the Cox model in which the log hazard ratio is a piecewise linear function of time since vaccination, with change points at 0, 2 and 4 months post full vaccination. For each trial, the mean and standard deviation of each estimate over 1000 replicates are shown.

Phase three trials only provide efficacy information six months post-dose 2 due to crossover of placebo recipients to the vaccine arm. For more information on the long-term effectiveness of vaccines, observational studies are more useful and tend to enable estimation of VE against severe disease and against different strains, even in various subpopulations. VEHR provides similar advantages over VEConst in the assessment of the level waning in VE in an observational setting. Reduction in VE over calendar time, or since vaccination, can be caused by a decline of immunity, the emergence of new variants, or additional factors. Comparing VE at calendar time allows better assessment of VE waning due to declining immunity, and taking different calendar times allows better evaluation of waning due to new variants.

Conclusion

The authors highlight that their newly proposed approach, based on estimating VEHR, improves sensitivity for evaluating the true duration of VE using data from both observational studies and phase three clinical trials as it allows VE to vary continuously by time post-vaccination as well as adjusting for changes in disease incidence over calendar time. They point out that this approach was used effectively in a study in VE in North Carolina and argue that analyses of observational data should adjust for demographics and comorbidities as well as other factors to reduce confounding bias. This information could be essential for public health policymakers and epidemiologists attempting to model the disease and could help inform future policy on the spread of the disease and the necessity for booster shots.

*Important Notice

medRxiv publishes preprint papers that have not yet undergone peer review. The information contained in this article should not be treated as fact or used to guide research or clinical practice.

Written by

Sam Hancock

Sam completed his MSci in Genetics at the University of Nottingham in 2019, fuelled initially by an interest in genetic ageing. As part of his degree, he also investigated the role of rnh genes in originless replication in archaea.