Specification of the Target Trials
We designed this observational analysis to emulate a target trial (i.e., a hypothetical pragmatic trial that would have answered the causal question of interest) of BNT162b2 as compared with mRNA-1273 for the prevention of Covid-19 outcomes in the VA health care system. The key components of the protocol are summarized in Table S1 in the Supplementary Appendix, available with the full text of this article at NEJM.org.
Eligibility criteria included veteran status, an age of at least 18 years between January 4 and May 14, 2021, no previously documented SARS-CoV-2 infection, no previous Covid-19 vaccination, and a known residential address outside of a long-term care facility, as well as known smoking status and body-mass index recorded within the previous year. Participants needed to have used the VA health care system during the previous year (defined as receiving care at a station eligible to administer the vaccines under study and having at least one primary care visit); however, they had to have had no interactions with the health care system in the previous 3 days (which may have indicated the start of symptomatic disease and precluded vaccination).
The interventions of interest were vaccination with either the BNT162b2 vaccine or the mRNA-1273 vaccine, with a second dose scheduled 21 days later for the BNT162b2 vaccine and 28 days later for the mRNA-1273 vaccine. To ensure balance of important characteristics across groups, eligible veterans in the target trial would be randomly assigned to one of these two vaccine groups within strata defined according to calendar date (5-day bins), age (5-year bins), sex (male or female), race (White, Black, other, or unknown), urbanicity of residence (urban or not urban), and geographic location (coded as one of 19 categories of the Veterans Integrated Services Network).
The five outcomes of interest were documented SARS-CoV-2 infection, documented symptomatic Covid-19, hospital admission for Covid-19, ICU admission for Covid-19, and death from Covid-19. For each eligible participant, follow-up started on the day the first dose of vaccine was received (baseline) and ended on the day of the outcome of interest, death, 168 days (24 weeks) after baseline, or the end of the study period (July 1, 2021), whichever occurred first.
This target trial was designed to evaluate the comparative effectiveness of the vaccines in a period during which the SARS-CoV-2 alpha variant was predominant. However, the alpha variant had decreased to a share of 26% of circulating variants in the United States as of June 26, 2021, as it was quickly displaced by the delta variant, which rose from a 68% share as of July 3, 2021, to 99% as of September 18, 2021.11 To evaluate the comparative effectiveness of the vaccines in a period with delta-variant predominance, we considered a second target trial that was identical to the first trial except that the recruitment period was July 1 to September 20, 2021, and the only outcome of interest was documented SARS-CoV-2 infection (because the period was too short to accumulate a sufficient number of rarer outcomes, such as hospitalization and death).
Emulation of the Target Trials
We emulated the above pragmatic target trials using the VA health care databases, which are described in the Supplementary Methods 1 section in the Supplementary Appendix. Table S2 provides detailed definitions of all study variables. Vaccination was identified with the use of records in the Immunization domain and procedures recorded in the Outpatient or Inpatient domain of the database. SARS-CoV-2 infections were identified with the use of the VA Covid-19 National Surveillance Tool,12 which integrates data on polymerase-chain-reaction (PCR) laboratory tests with natural language processing of clinical notes to capture diagnoses inside and outside the VA health care system. Symptomatic Covid-19 was defined as at least one of the following symptoms documented within the VA health care system within 4 days before or after documentation of SARS-CoV-2 infection: fever, chills, cough, shortness of breath or difficulty breathing, sore throat, loss of taste or smell, headache, myalgia, diarrhea, and vomiting. Symptoms were ascertained with the use of records in the Outpatient, Inpatient, Vital Signs, Health Factors, and Fee domains in the database. Hospitalization for Covid-19 was defined as a hospitalization within 21 days after documentation of SARS-CoV-2 infection (ascertained with the Inpatient domain), ICU admission for Covid-19 was defined as an ICU admission during hospitalization for Covid-19 (ascertained with the Inpatient domain and specialty transfer codes), and death from Covid-19 was defined as a death within 30 days after documentation of SARS-CoV-2 infection (ascertained using the Patient domain).
To mimic the stratified randomization of the target trial, we matched eligible persons who were vaccinated with BNT162b2 in a 1:1 ratio to eligible persons who were vaccinated with mRNA-1273. The matching factors (calendar date, age, sex, race, urbanicity of residence, and geographic location) are associated with the probability of receiving a particular vaccine, as well as with the risk of SARS-CoV-2 infection or severe Covid-19. (Additional details on the matching algorithm are provided in the Supplementary Methods 2 section in the Supplementary Appendix.)
To explore the possibility of residual confounding (e.g., by underlying health status or health care–seeking behavior), we used two negative outcome controls that are not directly affected by vaccination but for which the effect of vaccination might be similarly confounded.13 First, we evaluated the risk of symptomatic Covid-19 in the first 10 days after the first vaccine dose, during which no difference in risk between the vaccines is expected.1,2 Second, we evaluated the risk of death from causes other than Covid-19 during the follow-up period.
Covariate balance after matching was evaluated by plotting the mean differences between variable values (standardized for continuous variables) for the vaccination groups, with a difference of 0.1 or less considered to be acceptable.14 Cumulative incidence (risk) curves for the vaccination groups were estimated with the Kaplan–Meier estimator.15 We considered the period from the day of the first dose of vaccine until the end of follow-up. We used the Kaplan–Meier estimator with daily outcome events to compute the probability (risk) of the outcome during the period. We then calculated 24-week risk differences and risk ratios between the vaccination groups. We conducted subgroup analyses according to age (<70 or ≥70 years) and race (Black or White). Nonparametric bootstrapping with 500 samples was used to calculate percentile-based 95% confidence intervals for all estimates.
Analyses were performed with R software, version 3.6.0 (R Foundation for Statistical Computing), and SAS software, version 8.2 (SAS Institute). Information on authors’ contributions to the study is provided in the Supplementary Methods 3 section in the Supplementary Appendix. The first and last authors vouch for the accuracy and completeness of the data presented in this report.