Hailemichael Hishe1, Michael Tagen1, Vijay Ivaturi1,2, Andreas Noack1
1PumasAI, 2Center for Translational Medicine, University of Maryland
Introduction: Traditionally, the first-order estimation method (FOCE) [1] has been associated with the Gaussian error models. However, when FOCE is interpreted as an approximation of the quadratic coefficient in the Laplace approximation of the marginal likelihood function the method applies to any parametric error model. Specifically, it is possible to benefit from the cheaper FOCE approximation compared to a Laplace model using the actual Hessian when estimating Generalized Linear Models (GLM) type models such as logistic and Poisson regression. The generalized FOCE method is available in the Pumas software application [2]. To our knowledge, this feature is not available in any other software package for PKPD modeling. Pumas is implemented in the Julia programming language and builds on top of many open source packages from the Julia ecosystem such as Distributions.jl [3], DifferentialEquations.jl [4], Optim.jl [5], and ForwardDiff.jl. By being implemented in Julia, Pumas runs at speed comparable to C programs while easily allowing for utilization of automatic differentiation which eliminates the requirement for symbolic derivative computations even for models based on ordinary differential equations. The speed and accuracy of this generalized FOCE versus LaplaceI method has been explored previously [6] using a modified HCV model and a simulated dataset. The parameter estimates from the two methods were similar and the FOCE was 12 times faster than the Laplace estimation. However, these methods have not been compared with models of discrete response data. Objectives: Assess the speed and accuracy of the generalized FOCE support in Pumas software based on four pharmacokinetic/pharmacodynamic (PK/PD) models of discrete response data. Methods: Four types of discrete response models were assessed: (1) logistic regression, (2) ordinal regression, (3) Poisson regression, and (4) negative binomial regression. Datasets were obtained from PharmaDatasets.jl [5] or simulated based on published PK/PD models. For all models tested, the parameters were estimated in Pumas with both FOCE and full Hessian-based LaplaceI (with interaction) methods. The inner optimization for the computation of the empirical Bayes estimates used a Newton trust region solver and the outer optimization used a quasi Newton (BFGS) solver both from Optim.jl. All gradients were computed with automatic differentiation through the ForwardDiff.jl package. Results: The parameter estimates from the two methods were equivalent with all models assessed. In models with more than one random effect, FOCE completed marginally faster than LaplaceI estimation. The relative performance benefit of using FOCE increased with the number of random effects in the model. Conclusions: The FOCE approximation of the marginal likelihood can be an alternative to the Hessian based Laplace approximation for PD models of discrete response data. There may be a minor advantage of reduced run times in models with multiple random effects. For more expensive models, the effect is expected to be more pronounced. The FOCE approximation might also allow for estimation of models where estimation with LaplaceI terminates with numerical issues.
[1] Beal SL, Sheiner LB, Boeckmann A, Bauer RJ. NONMEM users guides. NONMEM Project Group, University of California, San Francisco. 1992. [2] Nelder JA, Wedderburn RW. Generalized linear models. Journal of the Royal Statistical Society: Series A (General). 1972 May;135(3):370-84. [3] Besançon M, Anthoff D, Arslan A, Byrne S, Lin D, Papamarkou T, Pearson J. Distributions. jl: Definition and modeling of probability distributions in the JuliaStats ecosystem. arXiv preprint arXiv:1907.08611. 2019 Jul 19. [4] Rackauckas, C. & Nie, Q., (2017). DifferentialEquations.jl – A Performant and Feature-Rich Ecosystem for Solving Differential Equations in Julia. Journal of Open Research Software. 5(1), p.15. DOI: http://doi.org/10.5334/jors.151 [5] Mogensen PK, Riseth AN. Optim: A mathematical optimization package for Julia. Journal of Open Source Software. 2018 Apr 5;3(24). [6] Noack A, Mogensen PK, Nyberg J, Ivaturi J. Generalized FOCE with Pumas. Poster presented at PAGE 2021.
Reference: PAGE 33 (2025) Abstr 11692 [www.page-meeting.org/?abstract=11692]
Poster: Methodology - Estimation Methods