Emmanuelle Comets (1,2)(*), Sofia Kasadiri (1), Moreno Ursino (3)
(1) Université de Paris, IAME, INSERM, F-75018 Paris, France (2) INSERM, CIC 1414, 35700 Rennes, France; Univ Rennes-1, 35700 Rennes, France (3) Centre de Recherche des Cordeliers, Sorbonne Université; Inserm, Université de Paris, F-75006, Paris; F-CRIN PARTNERS platform, AP-HP, Université de Paris, Paris, France
Objectives: Standard errors of estimation (SE) measure the precision of estimation for the parameters in a statistical model. A commonly used approach in non-linear mixed effect models (NLMEM) is to use the inverse of the Fisher information matrix to derive the SE. How well this asymptotic approximation holds in practice is a matter of debate, and alternative approaches have been proposed such as log-likelihood profiling and bootstrap methods.
Bootstrap approaches have been extended to NLMEM by considering the different levels of variability involved at the individual and population level. They include the case bootstrap, resampling individuals, the non-parametric bootstrap, which resamples residuals within and across individuals and the parametric bootstrap, resampling from a distribution. Thai et al. have investigated their properties in NLMEM for different numbers of subjects in sparse or rich designs, and found contrasted results [1]. A question arising was whether it would be possible to improve the correction of the residuals, performed before resampling to compound for shrinkage, to benefit from the non-parametric bootstrap’s ability to maintain the structure of the original dataset, while being less dependent on model assumptions than the parametric bootstrap.
In the present work, we propose an alternative non-parametric bootstrap, resampling from the conditional distribution of the individual parameters, and evaluate its performance.
Methods: A general bootstrap algorithm consists in creating bootstrapped datasets through resampling, fitting a model to the data, and storing the parameter estimates from each replicate to form a bootstrap distribution of the parameters. We implemented four bootstrap approaches: case, parametric (Par), non-parametric (NP) [1], and conditional non-parametric bootstrap (cNP). In this new approach, instead of correcting the estimated random effects for shrinkage, we sample random effects from the conditional distribution [2].
We evaluated the bootstraps using a sigmoid Emax model in a setting proposed previously by Plan et al. to compare estimation methods [3]. We evaluated the impact of rich (N=100 subjects with n=4 doses) and sparse designs (N=200, n=2), including unbalanced designs with varying amounts of information per subject (2 to 6 doses), and we also simulated two scenarios with increased residual error sigma in a rich design.
The coverage rates of the 95% confidence intervals for all parameter estimates were compared for the 4 bootstrap methods and the asymptotic approach, and we also computed bias and standard errors (SE) obtained as the mean and SD of the bootstrap distributions. We used the SAEM algorithm implemented in R in the saemix package [5,6].
Results: The asymptotic method tended to produce suboptimal coverages, especially for the variance terms, due to underestimated SE. Bootstrap approaches provided more adequate coverage, except for the NP bootstrap in the rich design. Overall, the new cNP provided better coverage than NP, with comparable performances to the Case. Increasing the residual error led to a marked degradation of the coverage rates for the random effects for Par and NP, and for sigma with all bootstraps.
Conclusions: Case bootstrap remains a simple and robust method providing adequate coverage. The new cNP based on samples from the conditional distributions offers a good alternative, albeit more time-consuming, for complex designs to avoid stratification. None of the bootstraps could fully recover good estimates of uncertainty, especially for variance terms, when both IIV and sigma were large.
Note: This work was also presented at WCoP 2022.
References:
[1] Thai H, Mentré F, Holford NH, Veyrat-Follet C, Comets E. Evaluation of bootstrap methods for estimating uncertainty of parameters in nonlinear mixed-effects models: a simulation study in population pharmacokinetics. J Pharmacokinet Pharmacodyn 2014;41:15–33.
[2] Lavielle M, Ribba B (2016). Enhanced method for diagnosing pharmacometric models: random sampling from conditional distributions. Pharm Res, 33:2979–88.
[3] Plan E, Maloney A, Mentré F, Karlsson MO, Bertrand J (2012). Performance comparison of various maximum likelihood nonlinear mixed-effects estimation methods for dose-response models. AAPS J, 14:420–432, 2012.
[4] Dumont C, Lestini G, Le Nagard H, Mentré F, Comets E, Nguyen T, the PFIM Group (2018). PFIM 4.0, an extended R program for design evaluation and optimization in nonlinear mixed-effect models. Comput Meth Prog Biomed, 156 : 217–29.
[5] Comets E, Lavenu A, Lavielle M (2017). Parameter estimation in nonlinear mixed effect models using saemix, an R implementation of the SAEM algorithm. J Stat Soft, 801–41.
[6] saemix package version 3.0, Comprehensive R Archive Network, https://CRAN.R-project.org/package=saemix
[7] Lavielle M (2014). Mixed Effects Models for the Population Approach – Models, Tasks, Methods and Tools. Chapman & Hall/CRC, Boca Raton.
Reference: PAGE 30 (2022) Abstr 10097 [www.page-meeting.org/?abstract=10097]
Poster: Methodology - Estimation Methods