Julie Dudasova (1,2), Zdenek Valenta (3), Garrett T. Nieddu (4), Alexander D. Becker (4), Jeffrey R. Sachs (4)
(1) Quantitative Pharmacology and Pharmacometrics, MSD Czech Republic, Prague, Czech Republic (2) First Faculty of Medicine, Charles University, Prague, Czech Republic (3) Institute of Computer Science of the Czech Academy of Sciences, Prague, Czech Republic (4) Quantitative Pharmacology and Pharmacometrics, Merck & Co., Inc., Rahway, NJ, USA
Introduction: This work introduces a novel effort to use immune response biomarkers to help identify covariates affecting vaccine efficacy (VE). VE is defined as a proportional reduction in risk of disease for vaccinated subjects compared to control subjects and is often assessed by counting disease cases and non-cases in randomized controlled clinical trials [1]. VE can be affected by “covariates” (demographic characteristics of enrolled subjects), e.g., age or gender. Statistical significance of covariate effects on the binary clinical outcome (diseased versus non-diseased) is typically evaluated by logistic regression.
In most efficacy trials, immune response post vaccination (immunogenicity) is measured in addition to the primary clinical endpoint. An immunogenicity biomarker that reliably predicts protection is a correlate of protection (CoP) [2].
It has been shown that CoP-based VE prediction is more precise than the case-count-based VE estimate [3]. Several approaches have been proposed to model the relationship between immunogenicity and probability of disease (PoD) [3-5].
Objectives:
- To evaluate accuracy and precision of CoP-based VE predictions using logistic function as a PoD curve.
- To compare, between two logistic regression approaches, the ability (sensitivity, specificity) to find covariate effects on VE. Approaches: (1) using CoP as a predictor; (2) using only case-counts (not using a CoP).
Methods: A logistic function is used to estimate the relationship between immunogenicity and PoD. VE is calculated as described in [5]. The 95% confidence interval (CI) associated with estimated VE includes uncertainty (on the PoD curve and on the immunogenicity data) via parametric resampling of the posterior distribution for the PoD curve parameters and bootstrapping the observed immunogenicity data.
Two approaches to covariate effect detection are compared: the CoP-based approach uses CoP data and covariate data as predictors of disease status, whereas the typical approach uses vaccination status (and covariate data). Each approach was tested with and without interaction between the predictors. A test (for covariate effect) is “positive” if either the covariate or interaction term have p-value<0,05.
Results: When applied to simulated data representing a typical phase 3 vaccine efficacy trial (15000 subjects, ~200 disease cases, population with 25% elderly subjects) with a logistic function for the “true” (simulated) PoD, then logistic-based PoD curve fit combined with immunogenicity distributions produced accurate point estimates of VE and well-calibrated CI. If the true PoD model is a Hill function [3] but logistic curve is fit, resulting VE is underestimated by 4% (on average).
If CoP data are used in logistic regression:
- Age group impacting VE (50% true VE in elderly and 80% true VE in younger subjects) is detected ~100% of the time for either form (Hill or logistic) of true PoD model.
- False positive rate of age group effect detection (80% true VE in all subjects) is 9,3% if the true PoD is a logistic function and 5,9% if the true PoD is a Hill function.
- Positive predictive value (PPV, for age group effect) is 91,5% if the true PoD is a logistic function and 94,4% if the true PoD is a Hill function, and negative predictive values (NPV) are ~100% and 99,6% (resp.).
- Area under the ROC curve (AUC) is ~100% if the true PoD is a logistic function and 99,8% if the true PoD is a Hill function.
If CoP data are not used, logistic regression provides similar true positive rates (~99%), false positive rates (~9%), PPVs (~91%), NPVs (~99%), and AUCs (~99%).
If the true PoD is a Hill function, inclusion of a CoP predictor in the logistic regression increases PPV of covariate effect detection by 2,6%.
Conclusions: Logistic regression can be reliably used to detect the effect of a binary covariate on VE. In the simulated trials, the typical approach was 46% more likely (100%-PPV: 8,2% versus 5,6%) than the CoP approach to falsely predict difference between the two subgroups (with a Hill curve as the true PoD). This false prediction could result in failure to license an appropriate indication. CoP-based performance is also better than that of the typical approach (data not shown) for smaller covariate effects (10% difference in VE) and smaller trials (7500 subjects, ~100 disease cases). The difference in this performance is smaller if the true PoD curve is logistic.
References:
[1] Halloran ME et al. Design and Analysis of Vaccine Studies. Springer, 2010.
[2] Plotkin SA et al. Plotkin’s Vaccines. Elsevier, 2017.
[3] Dudasova J et al. A method to estimate probability of disease and vaccine efficacy from clinical trial immunogenicity data. NPJ Vaccines, 6:133, 2021.
[4] Dunning AJ. A model for immunological correlates of protection. Statistics in Medicine, 25:1485–1497, 2006.
[5] Coudeville L et al. A new approach to estimate vaccine efficacy based on immunogenicity data applied to influenza vaccines administered by the intradermal or intramuscular routes. Human Vaccines, 6:841–848, 2010.
Reference: PAGE 30 (2022) Abstr 10095 [www.page-meeting.org/?abstract=10095]
Poster: Methodology - Covariate/Variability Models