Operating Characteristics of Stepwise Covariate Selection in Pharmacometric Modeling
Malidi Ahamadi *(1), Anna Largajolli *(2), Paul M Diderichsen (2), Rik de Greef (2), Thomas Kerbusch (2), Han Witjes (2), Akshita Chawla (1), Casey B Davis (1), Ferdous Gheyas (1),
*equal contributions, (1) Quantitative Pharmacology and Pharmacometrics, MRL, Merck & Co., Inc., Kenilworth, NJ USA, (2) Certara Strategic Consulting
Objectives: Stepwise covariate modeling (SCM) is a widely used tool in pharmacometric analyses to identify covariates that explain source of variability and improve model predictive performance. However, potential weaknesses of this approach include over-estimated covariate effect [1] and incorrect selection of covariates due to collinearity [2]. In this work we have investigated the operating characteristics of SCM in a controlled simulated setting.
Methods: Using a two-compartment model with first-order absorption, 16 scenarios were simulated based on the permutations of 4 covariates (body weight (BW) and creatinine clearance (CrCL) on apparent clearance, BW and SEX on volume of distribution). The simplest case was no covariate relationship and the most complex case was all 4 covariate relationships. For each scenario, 250 datasets were simulated with a sample size of 300 subjects and 6 observations per subject. In total 5 covariates (BW, BMI, CrCL, SEX, RACE), with high collinearity between BMI and BW, were bootstrapped from the NHANES dataset [3]. The scenarios were first assessed by re-estimating the simulated data with the respective true model and RMRSE was evaluated together with the model stability information (convergence, covariance step, etc.). Subsequently, each dataset was analysed by a full SCM procedure, as implemented in PsN and the power to select the true covariate model and RMRSE were derived.
Results: All re-estimated parameters had RMRSEs below 50% in all scenarios, confirming that the simulation design was appropriate. The SCM analysis showed a decrease in the power to detect the true covariates from 96% in the simplest scenario (no true covariates) to 25% in the most complex scenario (4 true covariates). Furthermore, BMI was frequently selected in place of BW in replicates where the true model was not recovered. The RMRSEs were below 50% for all fixed effects parameters, increased with model complexity and were slightly higher than the RMRSE obtained with a simple re-estimation as already observed in [1]. RMRSE on BSV increased with model complexity with the correlation term reaching a RMRSE of 150% in the most complex scenario.
Conclusion: Model complexity can have a great impact on the power to identify the true covariate model and on the accuracy and precision of the parameter estimates. Future research will investigate the effect of different sample sizes and observation schedules on the operating characteristics of SCM.
References:
[1] Ribbing J, Jonsson EN. Power, selection bias and predictive performance of the population pharmacokinetic covariate model. J Pharmacokinet Pharmacodyn. 2004;31(2):109–134.
[2] Bonate PL. The effect of collinearity on parameter estimates in nonlinear mixed effect models. Pharm Res. 1999;16(5):709–717.
[3] CDC - National Center for Health Statistics - Homepage. http://www.cdc.gov/nchs/.
[4] L. Lindbom, P. Pihlgren, and E. N. Jonsson. Psntoolkit: a collection of computer intensive statistical methods for non-linear mixed effect modeling using nonmem. Computer Methods and Programs in Biomedicine, 79(3):241, Sept. 2005
[5] Beal SL, Sheiner LB, Boeckmann AJ & Bauer RJ (Eds.) NONMEM Users Guides. 1989-2017. Icon Development Solutions, Ellicott City, Maryland, USA.
