Development of a tool for fully automatic model development (AMD)
Xiaomei Chen (1), Alzahra Hamdan (1), Shijun Wang (1), Tianwu Yang (1), Rikard Nordgren (1), Stella Belin (1), Zhe Huang (1), Simon J. Carter (1), Simon Buatois (2), João A. Abrantes (2), Andrew C. Hooker (1), Mats O. Karlsson (1)
(1) Department of Pharmacy, Uppsala University, Sweden (2) Roche Pharma Research and Early Development, Roche Innovation Center Basel, Basel, Switzerland
Introduction/Objectives: The development of a population pharmacokinetic (PK) model is a challenging and time-consuming procedure that involves iterative manual model fitting. Although tools for automatic model building[1–3] have been created, those were not designed for developing all model components. We aim to develop a tool for fully automatic model development (AMD) covering all the components of a PK model (structure, random effects, residual error, allometry, and covariate models). The AMD tool is currently under active development and is implemented in Pharmpy, an open-source software package for pharmacometric modeling.[4] We present the current development status and evaluation of the AMD tool covering model building for structure, inter-individual variability (IIV), and residual unexplained variability (RUV).
Methods: A structural model is automatically developed using an exhaustive stepwise algorithm that tests all the possible combinations of models predefined in the search space. The novel algorithm is also robust against local minima by estimating models repeatedly from different development routes (see abstract 10020 for more details[5]). An IIV model is selected using a 2-step exhaustive algorithm where the first step is to select the number of ETAs and the second step is to refine the OMEGA matrix. An RUV model is selected using resmod[6,7], a fast tool that performs modeling of conditional weighted residuals. Specifically, a stepwise procedure with resmod starts from the proportional model with forward addition of candidate residual error models (combined, power, IIV on RUV, and time-varying RUV models in the current version). Then the selected RUV model is re-evaluated on the original dataset. The Bayesian Information Criterion (BIC) for mixed effect models[8,9] is used for structural and IIV model selection during exhaustive searches, whereas the likelihood ratio test is used for RUV model selection.
The AMD tool was evaluated on the datasets of 10 drugs (5 i.v. drugs and 5 oral drugs) using 3 sequential approaches (SIR, SRI, and RSI), which refer to the development order for structural (S), IIV (I), and RUV (R). All the AMD procedures began from the same start model created with the R package assemblerr[4], which was a one-compartment model with first-order absorption (for oral drugs) and first-order elimination with IIV on all the parameters plus a correlation between clearance and volume of distribution. The final models selected through the AMD tool were compared to simplified published models without considering covariates and IOV. All the models were estimated using the FOCE method with interaction.
Results: The run times of the AMD process ranged from 3 min to 10 min for the i.v. drugs and from 16.5 min to 6.5 hr for the oral drugs. For 9 out of the 10 drugs, the same structural model was chosen through the 3 approaches. Comparing the final models from the approaches, the models with the lowest BIC were provided by SIR for 3 drugs, SRI for 6 drugs, and RSI for 4 drugs. The SRI and RSI approaches gave identical models (i.e., identical structural, IIV, and RUV models) for 3 drugs. For 9 drugs, the final models from the AMD tool generally exhibited a lower BIC than the published models. For the other drug, the comparison is not appropriate since the published model was developed based on log-transformed data, which has not been included in the current AMD tool yet.
Conclusions: The developed AMD tool was successfully applied on a collection of datasets with acceptable run times. Most of the selected models showed a lower BIC than the published models indicating reasonable quality. The AMD tool of Pharmpy serves as a promising tool for fast and fully automatic model building with the potential to accelerate pharmacometrics modeling in drug development (see abstract 10051 for integration of the AMD tool in drug development[10]). Our ongoing effort focuses on automatic development for IOV, allometry, and covariate models, identification and handling of outlying and/or influential individuals (abstract 10029[11]) as well as optimizing model building strategies to further improve efficiency and quality.
Acknowledgment: This work was supported by F. Hoffmann-La Roche Ltd., Basel, Switzerland.
References:
[1] E. N. Jonsson and M. O. Karlsson, “Automated covariate model building within NONMEM,” Pharm. Res., vol. 15, no. 9, pp. 1463–1468, 1998, doi: 10.1023/A:1011970125687.
[2] M. Prague and M. Lavielle, “SAMBA: A novel method for fast automatic model building in nonlinear mixed-effects models,” CPT Pharmacometrics Syst. Pharmacol., vol. 11, no. 2, pp. 161–172, Feb. 2022, doi: 10.1002/PSP4.12742.
[3] M. Sale and E. A. Sherer, “A genetic algorithm based global search strategy for population pharmacokinetic/pharmacodynamic model selection,” Br. J. Clin. Pharmacol., vol. 79, no. 1, pp. 28–39, 2015, doi: 10.1111/bcp.12179.
[4] R. Nordgren et al., “Pharmpy and assemblerr - Two novel tools to simplify the model building process in NONMEM,” PAGE 29, 2021.
[5] A. Hamdan et al., “Development of Pharmacokinetic Structural Models – Pharmpy Model Search Tool,” PAGE, 2022.
[6] M. M. A. Ibrahim, R. Nordgren, M. C. Kjellsson, and M. O. Karlsson, “Model-Based Residual Post-Processing for Residual Model Identification,” AAPS J., vol. 20, no. 5, pp. 1–9, 2018, doi: 10.1208/s12248-018-0240-7.
[7] M. M. A. Ibrahim, R. Nordgren, M. C. Kjellsson, and M. O. Karlsson, “Variability Attribution for Automated Model Building,” AAPS J. 2019 213, vol. 21, no. 3, pp. 1–8, Mar. 2019.
[8] M. Delattre, M. Lavielle, and M. A. Poursat, “A note on BIC in mixed-effects models,” Electron. J. Stat., vol. 8, no. 1, pp. 456–475, 2014, doi: 10.1214/14-EJS890.
[9] M. Delattre and M. A. Poursat, “An iterative algorithm for joint covariate and random effect selection in mixed effects models,” Int. J. Biostat., vol. 16, no. 2, pp. 75–83, Dec. 2016, doi: 10.48550/arxiv.1612.02405.
[10] J. A. Abrantes et al., “ADaMO: End-to-end automation of Pharmacometric modelling in drug development, from dataset building to output generation,” PAGE, 2022.
[11] O. Qutishat et al., “The development of artificial neural networks for the prediction of influential individuals and outlying individuals and their application during the model building development process,” PAGE, 2022.
