Douglas Eleveld1
1University Medical Center Groningen
Introduction: The estimation of mixed effect models from data is a backbone of pharmacometrics in industry and academia. Calculating a model fit to data is a critical part of pharmacometrics and requires the correct statistical methods and is often computationally intensive. Specialized software has appeared to help researchers to perform model estimation. The first-order-conditional estimation with interaction (FOCE-I) algorithm has been implemented in the OpenPMX software package and is written in the C-language which is widely available, fast, and efficient. Its source code [1] is available under an open-source license (GPL-3.0) allowing all implementation details to be inspected, audited, and improved. The project has few software dependencies and model estimation can utilize multiple cpu-cores using pthreads or OpenMP. Objectives: The purpose of this investigation was to evaluate the estimation accuracy of FOCE-I estimation algorithm in OpenPMX on three datasets and compare that with NONMEM [2] which is widely used in pharmacometrics. Methods: Three scenarios were investigated with Monte-Carlo analysis: 1) rich-data derived from the Schnider [3] propofol dataset (100 resamples) with a relatively simple model so good estimation performance is expected. 2) Sparse-data from a one-compartment PK model [4] (500 resamples) for which some bias or poor rmse may be present. 3) A dataset known to be difficult to estimate, from a previously performed blind-comparison of model estimation software and algorithms [5] (100 resamples). Simulated data were generated using NONMEM with the $SIM record. The methods OpenPMX and NONMEM 7.5.0 were used to estimate each resampled dataset, and the relative prediction error for the true values was calculated. Parameter initial estimates and ranges were the same for both methods. For OpenPMX, the default settings were used. For NONMEM, estimation settings SIG=5 MAX=5000 METHOD=1 INTERACT were used. The precision of the parameters considered were the fixed effects (THETA), random effects (OMEGA and SIGMA), and parameter correlations if estimated in the model. The comparison for overall performance was a Wilcox test for significant difference in prediction error. If significant (p-value < 0.05), the software with the lowest median prediction error outperformed the other, otherwise no difference was detected. Results: For the rich-data scenario 1) the FOCE-I algorithm in OpenPMX outperformed NONMEM for 6 of 11 parameters, although the differences were quite small (largest 0.4%). For the sparse-data scenario 2), OpenPMX outperformed NONMEM for 5 of 7 parameters (largest 1.6%), whereas NONMEM outperformed OpenPMX for 1 parameter (2.0%). For the difficult to estimate dataset in scenario 3), OpenPMX outperformed NONMEM for 4 of 11 parameters (largest 219%), whereas NONMEM outperformed OpenPMX for 1 parameter (3.5%). Conclusion: For the scenarios tested, the open-source OpenPMX software for mixed-effects modelling in pharmacometrics using the FOCE-I algorithm performed similarly to NONMEM.
[1] https://github.com/deleveld/OpenPMX [2] Icon Development Solutions, Ellicott City, Maryland, USA [3] Schnider TW, Minto CF, Cambus PL, et al. The influence of method of administration and covariates on the pharmacokinetics of propofol in adult volunteers. Anesthesiology 1998; 88: 1170-82. [4] Schoemaker R, Fidler M, Laveille C, et al. Performance of the SAEM and FOCEI algorithms in the open-source, nonlinear mixed effect modeling tool nlmixr. CPT: Pharmacometrics & Systems Pharmacology. 2019; 8(12): 923-30. [5] https://www.page-meeting.org/default.asp?abstract=834
Reference: PAGE 33 (2025) Abstr 11424 [www.page-meeting.org/?abstract=11424]
Poster: Methodology - Estimation Methods