Mélanie Karlsen1,2, Jérôme Azé1, Sandra Bringay3, Pascal Poncelet1, David Fabre2, David Marchionni2, Elisa Calvier2
1LIRMM, Laboratory of Computer Science, Robotics and Microelectronics in Montpellier, CNRS, Montpellier University, 2Sanofi R&D Montpellier, Pharmacokinetics Dynamics and Metabolism / Translational Medicine and Early Development, 3Applied Mathematics, Computer Science and Statistics (AMIS), Montpellier 3 University
Introduction Covariate modeling is one of the most time-consuming steps in developing population pharmacokinetics models. While a total of 22 methods have been referenced for this task, relative performances of these methods remain not sufficiently documented due to a lack of consensus on how to evaluate these techniques, from experimental setup to performance metrics [1]. The aim of this work was to build PMX-CovEval, a framework enabling robust evaluation and consistent comparison of covariate model selection techniques. Methods The developed framework includes (1) a collection of simulated pharmacokinetic (PK) datasets under different model scenarios, (2) their corresponding NONMEM and Monolix pharmaco-statistical model files (PSM), (3) individual parameters estimates obtained with the PSMs, and (4) a table summarizing all scenarios of the framework. A total of 103 different model scenarios were considered using different combinations of structural, pharmaco-statistical and covariate models. The open-source R campsis framework [2], including campsismod and campsis with mrgsolve [3], was used to simulate the 103 PK datasets according to those pre-defined model scenarios, all sharing the same clinical trial design and virtual population. The clinical trial design was derived from a dataset of pooled Sanofi studies with two Phase 1 and two Phase 3 studies, and included a total of 2830 concentration sampling times across 624 individuals. Individuals were sampled from the NHANES database for twelve continuous and three categorical covariates, with a filter ensuring that Phase 1 individuals were between 18 and 45 years old. For each scenario, NONMEM control streams and Monolix mlxtran project files of the PSMs were generated using the Python Pharmpy [4] module and the R lixoftConnectors [5] package, respectively. Individual empirical Bayes estimates (EBEs) were retrieved following NONMEM PSM execution of all scenarios. Complete information on true model structure and parameter values for each of these scenarios were provided in the framework summary table. Successful minimization of the NONMEM and Monolix PSMs as well as NONMEM true model (also generated with Python pharmpy [4] module) was checked for. Parameter identifiability was assessed by means of stochastic simulation and estimation (SSE) on scenarios with least expected identifiability, i.e. scenarios with highest unexplained or total inter-individual variability, and those with complex structural models. Finally, complete PK profiles until steady state for three hypothetical individuals representing the 5th, 50th and 95th percentiles of each covariate were simulated to visually ensure consistent PK profiles, even under the strongest covariate effects scenarios. Results PK profiles generated for the three hypothetical individuals confirmed that even under extreme covariate values and strong covariate effects on PK parameters, all model scenarios produced plausible PK profiles. Additionally, 95% of the model files executed with NONMEM and Monolix achieved successful minimization. SSE further confirmed identifiability of the most complex scenarios, since true parameter values were systematically within two standard deviations of the mean estimated parameter values. Providing users with the collection of simulated datasets and scenario summary table enables the investigation of any covariate model selection technique. In particular, users can assess how varying model components (e.g., covariate effect size) across scenarios impacts the performance metrics of their chosen method. Providing additional PSM files for NONMEM and Monolix enables direct execution of any existing or future Perl speaks NONMEM (PsN) or Monolix internal covariate model selection technique, two of the most widely used pharmacometrics tools. Providing additional EBEs estimated under the PSMs allows for direct EBEs-regression based methods evaluation (including machine and deep learning ones), supporting the development and assessment of AI-driven covariate modeling approaches. Conclusion By incorporating scenarios with variability across multiple model elements, the presented framework PMX-CovEval enables rigorous evaluation and consistent comparison of both existing and future covariate modeling methods. The framework will be made available on GitHub by the time of the conference, through the following link: https://github.com/melaniekarlsen/PMX-CovEval.
[1] Mélanie Karlsen, Sonia Khier, David Fabre et al., “Covariate Model Selection Approaches for Population Pharmacokinetics: A Systematic Review of Existing Methods, From SCM to AI”, CPT Pharmacometrics Syst Pharmacol, 2025. [2] https://calvagone.github.io/ or www.campsis.org [3] https://github.com/metrumresearchgroup/mrgsolve [4] https://pharmpy.github.io/ [5] https://monolixsuite.slp-software.com/r-functions/2024R1/reference
Reference: PAGE 33 (2025) Abstr 11492 [www.page-meeting.org/?abstract=11492]
Poster: Methodology - Covariate/Variability Models