Jason Chittenden (1) and Klaas Prins (1)
(1) qPharmetra LLC, Andover, MA, USA
Objectives: Stepwise covariate modeling (SCM) is a commonly used tool for covariate model selection. Traditionally, SCM as implemented in Perl Speaks NONMEM (PsN)[1] is performed using deterministic estimation methods (e.g. FOCE-I, Laplace). The recent availability of stochastic methods, such as: stochastic approximation EM (SAEM), importance sampling EM (IMPEM), and Bayesian Markov Chain Monte Carlo methods (BAYES) [2] raises the question of how the SCM process fares in the face of stochastic noise in the objective function. Prompted by a real case where SCM failed with FOCE-I and succeeded with BAYES the current simulation-estimation study aims to test the performance of FOCE-I vs. BAYES methods in a hypothetical SCM process and to challenge the model selection criterion used.
Methods: The true model was a one-compartment elimination, 1st-order absorption PK model with body weight proportionally increasing volume of distribution and CL decreasing with age. From this model a data set with 90 subjects with dense sampling was simulated 100 times. Other, non-impactful and highly correlated covariates were added to the data set. Using the correct structural model, an SCM for three scenarios was run on all data sets and the success rate to recover the true model was retained. The three SCM scenarios were: 1) using FOCE-I and the final objective function value (OFV) as model selection criterion, 2) using MCMC BAYES and the sample mean OFV and 3) using MCMC BAYES and the Deviance Information Criterion[3] (DIC). DIC has been suggested [4,5] as a robust method for model selection in Bayesian analyses. The SCM tool in PsN was modified to compute either the mode of the OFV (current SCM default) or the DIC to be used as model selection criterion. Where SCM failed to identify the true model it was determined if pruning of the selected model by removing non-significant (alpha=5%) effects would arrive at the true model.
Results: Scenario 1 (FOCE_I & mode of OFV) selected the true model in 65% of the cases, with 68% succeeding after pruning. Scenario 2 (MCMC BAYES, mode of OFV) succeeded 60% of the time, increasing to 73% with pruning. Scenario 3 (MCMC BAYES, DIC) performed worst at 14% success rate, and 22% after pruning.
Conclusion: SCM can be performed using MCMC Bayesian or FOCE-I estimation interchangeably using the OFV as model selection criterion. The DIC criterion was found to be unsuitable for the stepwise covariate search when using Bayesian estimation.
References:
[1] Lindbom, L.; Pihlgren, P. & Jonsson, N. PsN-Toolkit — A collection of computer intensive statistical methods for non-linear mixed effect modeling using NONMEM. Computer Methods and Programs in Biomedicine, 2005
[2] Bauer, R.NONMEM Users Guide: Introduction to NONMEM 7.3.0. ICON Development Solutions; 2014
[3] Spiegelhalter, D. J.; Best, N. G.; Carlin, B. P. & Van Der Linde, A. Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B (Statistical Methodology), Blackwell Publishers, 2002
[4] Lunn, D. J.; Best, N.; Thomas, A.; Wakefield, J. & Spiegelhalter, D. Bayesian Analysis of Population PK/PD Models: General Concepts and Software. Journal of Pharmacokinetics and Pharmacodynamics, Journal of Pharmacokinetics and Pharmacodynamics, Kluwer Academic Publishers-Plenum Publishers, 2002
[5] van der Linde, A. DIC in variable selection. Statistica Neerlandica, Blackwell Publishing, 2005
Reference: PAGE 24 (2015) Abstr 3548 [www.page-meeting.org/?abstract=3548]
Poster: Methodology - Covariate/Variability Models