Jan Rohleff 1, Freya Bachmann 1, Britta Steffens 2, Mona Kasper 1, Gilbert Koch 2, Johannes Schropp 1
1 University Of Konstanz (Konstanz, Germany), 2 University Children's Hospital Basel (UKBB) (Basel, Switzerland)
Objectives
Accurate pharmacometrics (PMX) modeling is essential for optimizing drug therapy and understand-ing patient variability. Nonlinear mixed-effect (NLME) models are the gold standard tool in drug de-velopment. Classical software packages for NLME modeling either based on linearization techniques (FO, FOCE, La Place) or, in a more stochastic approach, on maximizing expectation values (SAEM). Recently, the generative AI technique Variational Autoencoder (VAE) was introduced for NLME modeling [1], heralding a new era.
Our objective is to demonstrate how and where VAEs can be applied to improve NLME analyses compared to established methods, specifically in terms of log-likelihood and the corrected Bayesian Information Criterion (BICc). In particular, we show advancements in covariate selection and han-dling time-varying covariates. The VAE framework can estimate population model parameters and perform covariate selection in a single run. Moreover, we demonstrate how generative AI can auto-matically incorporate time-varying covariates into standard NLME models without relying on simple interpolation or expert-defined covariate models.
Methods
To solve NLME problems, we use a VAE, a powerful machine learning technique that allows efficient estimation of model parameters. The VAE employs an encoder–decoder structure. First, the encoder transforms the data into distributions of individual parameters, from which the decoder reconstructs the model predictions. The VAE framework is highly flexible and enables the integration of additional modeling tasks, such as covariate selection and handling time-varying covariates, directly within the model.
Covariate selection is performed by directly minimizing an information criterion. By leveraging mixed-integer optimization procedures, which are well suited for selection problems, our method reformulates covariate selection as a sparse regression problem. This improves both the quality and computational efficiency of the selection process.
Handling time-varying covariates is achieved by adapting the encoder with a neural ODE to learn time-dependent individual parameters. This allows time-varying covariates to be automatically incorporated into the VAE framework.
Results
The proposed approach was successfully applied to two real-world applications [2,3].
The first application [2] demonstrates the VAE’s capability in covariate selection [1]. Here, the struc-tural model was developed to characterize weight progression over the first seven days of life in 2425 neonates [2]. The structural model consists of five parameters and the clinically relevant covariates parity, sex of neonate, gestational age, mother age and delivery mode were applied, resulting in 2^{25} possible covariate models. We compared the VAE results with traditional methods such as COSSAC, SAMBA, and SCM. All methods agreed on eight covariates, while the remaining selec-tions varied slightly. The BICc differed by at most 0.2%, indicating that all models provided approxi-mately equivalent fits. However, the VAE achieved these results in a single run, whereas SAMBA, COSSAC, and SCM required 2, 33, and 244 runs, respectively, to select similarly optimal covariates.
The second application [3] involved newborns and infants with congenital hypothyroidism treated with levothyroxine. Here, body weight changed substantially during the first two years of life and was incorporated as a time-varying covariate in the NLME model. Previously, body weight had to be modeled manually, either using an expert-based approach or standard interpolation methods. Using a VAE, the covariate was modeled automatically, resulting in a 10% improvement in log-likelihood compared with the expert-based model.
Conclusion
VAEs provide an efficient and unified framework for NLME modeling, enabling simultaneous parameter estimation, covariate selection, and automatic handling of time-varying covariates. Across both applications, the VAE achieved comparable or improved log-likelihood and BICc values while substantially reducing computational effort. These results highlight the potential of generative AI to enhance efficiency, flexibility, and objectivity in pharmacometric analyses
References:
[1] Rohleff, J., Bachmann, F., Nahum, U., Bräm, D., Steffens, B., Pfister, M., Koch, G. and Schropp, J. (2025), Redefining Parameter Estimation and Covariate Selection via Variational Autoencoders: One Run Is All You Need. CPT Pharmacometrics Syst Pharmacol, 14: 2232-2243. https://doi.org/10.1002/psp4.70129
[2] Wilbaux M., Kasser S., Gromann J. et al. (2019). Personalized Weight Change Prediction in the First Week of Life. Clin Nutr. 38(2):689-696.
[3] Koch G, Steffens B, Leroux S, Gotta V, Schropp J, Gächter P, Bachmann F, Welzel T, Janner M, L’Allemand D, Konrad D, Szinnai G, Pfister M. Modeling of levothyroxine in newborns and infants with congenital hypothyroidism: challenges and opportunities of a rare disease multi-center study. J Pharmacokinet Pharmacodyn. 2021 Oct;48(5):711-723. doi: 10.1007/s10928-021-09765-w. Epub 2021 Jun 11. PMID: 34117565; PMCID: PMC8405503.
Reference: PAGE 34 (2026) Abstr 12065 [www.page-meeting.org/?abstract=12065]
Poster: Methodology – AI/Machine Learning