Elisa Calvier 1
1 Elisa A. M. Calvier, PMx-D, QP, TMU, Sanofi R&D Montpellier, France (Montpellier, France)
Objectives
For the past few years, pharma industries have been working to shorten development timelines. In 2026, Sanofi aims at halving time from database lock to filling, significantly reducing clinical study report finalization as well as time between key result memo of phase Ib and phase II protocol finalization. This leads to drastic reduction in model development time. In this context, the use of stepwise covariate model building (SCM) becomes impractical and mandates moving to a much faster gold standard. The objectives of this work were threefold: (1) to assess the existing knowledge on relative speed and performance of covariate model building methods (2) identify roadblocks for cross-publication results comparison; and (3) to propose a framework allowing for overcoming these roadblocks and accelerating gold standard evolution in pharmacometrics.
Methods
We conducted a systematic review following PRISMA guidelines to identify all published covariate modeling methods and their comparative performance evaluations [1]. For each method, evaluation settings, performance metrics, and computational times were extracted when available and compared. Limits for cross-publication comparison were identified. Based on these roadblocks and evaluation settings found in literature, we developed PMX-CovEval, an open-source framework consisting of: (1) 106 simulated pharmacokinetic datasets representing diverse model scenarios, (2) corresponding NONMEM and Monolix pharmaco-statistical model files, (3) individual parameter estimates, and (4) a comprehensive scenario summary table. The framework was constructed using the open-source R campsis framework, with clinical trial design derived from pooled Phase 1 and Phase 3 studies including 2830 concentration sampling times across 624 individuals.
Results
The systematic review [1] identified 22 distinct covariate modeling methods across five categories: stepwise procedures, full model-based approaches, genetic algorithms, EBEs-based machine learning, and non-EBEs-based machine learning methods. All methods showed unreported or better computational times than SCM, except for Genetic Algorithms. EBEs‐based ML methods were the best covariate selection methods, and AALASSO, H‐GA‐ML, FREM with a clinical significance criterion and SCM+ with SF were the best covariate model selection methods. Two main roadblocks for cross-study comparisons were identified. The first one is the different settings used to generate simulated datasets for method evaluation and comparison, with a median number of criteria used for simulations across published studies of 1. Secondly, performance metrics varied widely across studies, part of which cannot be compared across publications. 8 simulations settings were used across retrieved publications: Number of covariates, extent of covariate effect, correlations between covariates, number of individuals, sampling design, correlations between PK parameters, extent of interindividual variability and residual variabilities. Each of these settings were used and varied to design the 106 simulation scenarios of PMX-CovEval, together with different covariate relationships and types of covariates (e.g., continuous or categorical).
PMX-CovEval delivers a complete, ready-to-use evaluation environment for covariate model selection techniques. It enables seamless execution of covariate model selection procedures in tools such as PsN [2] and Monolix [3] but also allows direct application of EBE-based regression methods, including machine learning and deep learning. The scenario table and structure of the framework enable users to investigate how method performance varies with key modeling factors such as covariate effect size, unexplained variability, or model complexity.
The proposed metrics allowing for cross-study comparisons of methods performance evaluation are relative root mean square error and relative mean prediction error. Additionally for simulated data, true negative rate, true positive rate and positive predictive value for 1) the whole selected covariate model and 2) selected covariate-PK parameter pairs.
Conclusions
Current literature cannot definitively support the adoption of faster methods than SCM due to two major roadblocks: heterogeneous performance metrics not allowing for cross-publication evaluation, and limited scenario diversity in individual publications. PMX-CovEval framework and our proposed set of metrics address these roadblocks by providing an open and extensible framework for the systematic evaluation of covariate model selection methods in popPK. By offering shared datasets, executable models, EBEs, and detailed scenario documentation, it promotes reproducibility, transparency, and cumulative benchmarking. Widespread community adoption of the framework and of standardized performance metrics has the potential to accelerate methodological development and methods benchmarking, support consensus building, and enhance best practices in covariate modeling. The framework is available on GitHub through the following link: https://github.com/sanofi-public/PMX-CovEval.
References:
1. Karlsen M, Khier S, Fabre D, et al. Covariate Model Selection Approaches for Population Pharmacokinetics: A Systematic Review of Existing Methods, From SCM to AI. CPT Pharmacometrics Syst Pharmacol. 2025;14:621-639.
2. Lindbom L, Pihlgren P, Jonsson EN. PsN-Toolkit–a collection of computer intensive statistical methods for non-linear mixed effect modeling using NONMEM. Comput Methods Programs Biomed. 2005 Sep;79(3):241-57.
3. Lixoft. (2023). Monolix version 2023R1 [Logiciel]. Antony, France. https://lixoft.com/products/monolix/
Reference: PAGE 34 (2026) Abstr 12182 [www.page-meeting.org/?abstract=12182]
Poster: Methodology - New Modelling Approaches