Alexander Janssen 1,2, Marjon H. Cnossen 1, Ron A.A. Mathôt 2
1 Department of Paediatric Haematology and Oncology, Erasmus MC Sophia Children's Hospital, University Medical Center Rotterdam (Rotterdam, The Netherlands), 2 Hospital Pharmacy and Clinical Pharmacology, Amsterdam UMC, University of Amsterdam (Amsterdam, The Netherlands)
Background
Recently, NeuralODE-based approaches have emerged as promising tools to learn dynamical systems governing drug exposure and effects directly from data. However, these models are often difficult to train, prone to overfitting, and empirically show unreliable extrapolation beyond training data. To address these limitations, we propose Gaussian Process Flows (GPFlows), a method to perform Bayesian Inference over the space of mechanistic models. The result is a flexible full random effect model that naturally limits model complexity while learning subject-specific dynamics. Similarly to NeuralODEs, model components can be visualised to interpret learned effects. Adopting a Bayesian framework unlocks several additional benefits: principled uncertainty estimates, more faithful and robust model comparison, and continuous learning as new data becomes available. A unique benefit of GPFlows is that it compresses data into a set of inducing points, which can be safely shared without risks of patient re-identification.
Objective
Compare performance of GPFlows to traditional non-linear mixed effect (NLME) models and the NeuralODE model by [1] on four real-world data sets.
Methods
GPFlows follow a typical two-compartment structure where central and peripheral distribution volumes are based on typical (static) random effects while clearance parameters are represented by concentration-dependent GPs. Additional dimensions can be added to model drug absorption and/or effects. GPs can be interpreted as distributions over functions. Population-based priors over the shape of these functions can be learned which naturally favour lower complexity. Corresponding posterior distributions are inferred per subject, enabling the model to learn both mechanistic differences between individuals and population-based variability in the dynamics.
We employed sparse variational GPs (SVGPs)[2], which approximate the posterior using a small set of inducing points u to improve computational efficiency. Parameters of variational distributions q(u) are learned by maximising the evidence lower bound (ELBO). A stochastic variational expectation maximisation (VEM) procedure was implemented. In the E-step, each q(u) was optimised for a fixed number of iterations, while GP hyper-parameters, random effect variance, and residual error variances were optimised until convergence during the M-step. Five to eight VEM iterations with 50 E-steps (100 in the first iteration) were sufficient for convergence.
Four distinct real-world data sets were used:
* Theophylline (THEO): 12 subjects, single dose, two compartments with absorption, additive error[3].
* CP1085: 20 subjects, multiple dosing routes, one compartment with absorption and bioavailability, combined error[4].
* Remifentanil (REMI): 65 subjects, infusion with dense sampling, three compartments, combined error[5].
* Warfarin (WARF): 32 subjects, two compartments with indirect response model, combined error[6].
For NLME models, aforementioned structural models were implemented and random effects were included when objective function value significantly decreased. In the NeuralODEs, typical ODE parameters including random effects were estimated and neural networks with a single hidden layer predicted changes in these parameters based on drug concentration and time. Models were compared based on marginal log likelihood (MLL; estimated via importance sampling), root mean squared error (RMSE), and interpretation of learned effects. Models were trained and evaluated on full data sets (objective was accurate data description, not prediction). Covariate effects were not evaluated.
Results
GPFlows consistently achieved higher MLL and lower RMSE compared to NeuralODE and NLME models, indicating better predictive accuracy with similar or lower model complexity. NeuralODE showed best MLL on the THEO dataset (but worse RMSE than GPFlow), however, these estimates were inflated as neural network parameters are not penalised in the complexity term, highlighting a fundamental limitation in comparing NeuralODE-based models.
NeuralODEs required extensive hyper-parameter tuning to achieve stable performance and training. Extrapolating beyond the training data was also unreliable. In contrast, GPFlow optimisation was stable, while GP priors prevented unexpected behaviour at data extremes.
Visualisations of learned dynamics can enhance model understanding. For example, in the WARF data set, GPFlow visualisations revealed two patient subgroups that differed in the nonlinearity of absorption. In contrast, the canonical NeuralODE-based implementation failed to capture this heterogeneity, as it can only learn average, population-wide effects.
Conclusion
GPFlows provide a compelling framework for learning mechanistic models governing observed drug exposure and effects. The method offers improved fidelity over NLME models while addressing limitations of NeuralODEs. Adapting the method for prediction is straightforward: forecasting is possible given historical observations, while predictions for new patients can be obtained by learning covariate-dependent priors over inducing points.
References:
[1] Bräm, D. S., Nahum, U., Schropp, J., Pfister, M., & Koch, G. (2024). Low-dimensional neural ODEs and their application in pharmacokinetics. Journal of Pharmacokinetics and Pharmacodynamics, 51(2), 123-140.
[2] Titsias, M. (2009). Variational learning of inducing variables in sparse Gaussian processes. In Artificial intelligence and statistics (pp. 567-574). PMLR.
[3] Boeckmann, A. J., Sheiner, L. B., & Beal, S. L. (1994). NONMEM users guide: part V.
[4] Denney W (2023). PKdata: Pharmacokinetic Data Sets. https://github.com/billdenney/PKdata, https://billdenney.github.io/PKdata/
[5] Minto, C. F., Schnider, T. W., Egan, T. D., Youngs, E., Lemmens, H. J., Gambus, P. L., … & Shafer, S. L. (1997). Influence of age and gender on the pharmacokinetics and pharmacodynamics of remifentanil: I. Model development. Anesthesiology, 86(1), 10-23.
[6] O’reilly, R. A., & Aggeler, P. M. (1968). Studies on coumarin anticoagulant drugs: initiation of warfarin therapy without a loading dose. Circulation, 38(1), 169-177.
Reference: PAGE 34 (2026) Abstr 11943 [www.page-meeting.org/?abstract=11943]
Poster: Oral: Methodology - New Modelling Approaches