Alex Al-Mashat (1), Hanna Kunina (1), Jenny Chien (2), Parag Garhyan (2), Maria C. Kjellsson (1)
(1) Department of Pharmacy, Uppsala University, Uppsala, Sweden, (2) Eli Lilly and Company, Indianapolis, USA
Objectives: The standard pharmacometric approach of model selection ignores model uncertainty, potentially leading to impaired predictive performance, which may partially be solved by the model averaging approach (MA) [1]. Within the area of type 2 diabetes, the choice of the most appropriate model to predict glycated hemoglobin (HbA1c) is crucial and remains challenging. The models, used within the field, are of different complexity and use multiple biomarker/biomarker combinations to drive the HbA1c formation. To help in model decision-making, this project aimed to compare several published models for type 2 diabetes using the MA approach with different weighting methods and to assess potential models’ contribution to the HbA1c prediction on an individual level.
Methods: Data were simulated using the integrated IGI-IGRH model [2, 3], including a placebo arm (100 individuals) and two treatment arms with 2 doses (100 individuals per dose and drug). The treatment arms were designed to illustrate a drug directly affecting postprandial glucose through insulin (Liraglutide) or a delayed effect on glucose production (Metformin). The performance of the following models of HbA1c modelling was explored: ADOPT [4], IGRH [3], FFH [5], FHH [6] and LILLY [7]. The MA approach was performed with three different weighting methods: Akaike information criterion (AIC), cross-validation (CV) [8] and bootstrap (BS) [9], where the AIC accounted for the models’ descriptive performance, CV for the models’ predictive performance and BS for the uncertainty. Each method was conducted in a three-step way: 1) parameter estimation using candidate models and required biomarker data, 2) updating final estimates and rerun with only HbA1c data, 3) calculate weights based on the OFV values from step 2 using the equation from Buatois et al [1]. In step 2, the AIC method was performed in 2 ways: a) omitting the re-estimation or b) allowing the HbA1c related parameters to be re-estimated. For CV and BS no re-estimation was performed in step 2. The AIC and CV were performed on population and individual level, however, the BS was only performed on the population level. On the individual level, the weighting was calculated using the individual OFVs. For all methods, the impact of the type of drug on the weighting was evaluated. Data simulation and model-based analyses were performed using NONMEM V7.4.4 and PsN V5.2.0. Weighting calculations and a graphical evaluation of the results were performed using R V4.0.4.
Results: Allowing re-estimation of HbA1c-related parameters (2b) had little impact on the weighting (2b vs 2a): ADOPT (22.3% vs 22.6%), IGRH (22% vs 22.3%), FFH (15.1% vs 14.6%), FHH (21.9% vs 21.6%), LILLY (18.7% vs 18.9%); thus, the estimation step was ignored in further investigations. On the population level, all three methods gave similar results in terms of weights for candidate models (AIC, CV, BS): ADOPT (22.6%, 22.6%, 22.6%), IGRH (22.3%, 21.1%, 22.3%), FFH (14.6%, 14.8%, 14.6%), FHH (21.6%, 21.9%, 21.6%), LILLY (18.9%, 19.6%, 18.9%). Consistent across all three methods, the weights of the models decreased in the following order: ADOPT-IGRH-FHH-LILLY-FFH, with only a marginal difference in the weights of the first three models. In the majority of cases, ADOPT was selected as the best one, having the lowest OFV value. The same result was shown for both treatment arms. However, on the individual level, the selection of model varied greatly, and for many individuals, the models with low weights on a population level were selected as the best ones; for example, LILLY, with an AIC-weight of only 18.9%, was selected as the best model in 22.8% of the individuals (114 of 500), while the FHH-model, with AIC-weight of 21.6%, was selected in only 17% of individuals (85/500). Some individuals were found to have extremely high (>40%) or low (<5%) weights for some of the candidate models.
Conclusions: With this project, we have illustrated how to perform MA for the models using different biomarkers as well as the difference between weights on population and individual level. On the population level, ADOPT was selected as the best model with all three MA weighting methods. On the individual level, more investigations have to be done to define specific patient populations that benefit from certain models.
References:
[1] Buatois et al. Comparison of Model Averaging and Model Selection in Dose Finding Trials Analyzed by Nonlinear Mixed Effect Models. AAPS J. maj 2018;20(3):56.
[2] Jauslin, P.M. et al. An integrated glucose-insulin model to describe oral glucose tolerance test data in type 2 diabetics. J. Clin. Pharmacol. 47, 1244–1255 (2007).
[3] Lled_o-Garci´a, R., Mazer, N.A. & Karlsson, M.O. A semi-mechanistic model of the relationship between average glucose and HbA1c in healthy and diabetic subjects. J. Pharmacokinet. Pharmacodyn. 40, 129–142 (2013).
[4] Møller J, Overgaard R, Kjellsson M, Kristensen N, Klim S, Ingwersen S, m.fl. Longitudinal Modeling of the Relationship Between Mean Plasma Glucose and HbA1c Following Antidiabetic Treatments. CPT Pharmacomet Syst Pharmacol. oktober 2013;2(10):82.
[5] Choy S, Kjellsson MC, Karlsson MO, de Winter W (2015). Weight-HbA1c-insulin-glucose model for describing disease progression of type 2 diabetes. CPT Pharmacometrics Syst Pharmacol. 2016 Jan;5(1):11-9.
[6] Hamrén B, Björk E, Sunzel M, Karlsson M. Models for Plasma Glucose, HbA1c, and Hemoglobin Interrelationships in Patients with Type 2 Diabetes Following Tesaglitazar Treatment. Clin Pharmacol. 2008;84(2):8.
[7] Gaitonde P, Hurtado FK, Garhyan P, Chien JY, Schmidt S. Development and quantification of a drug-disease modeling platform to characterize clinically relevant endpoints in type 2 diabetes trials. Clin Pharmacol Ther 2018 104(4): 699-708.
[8] Erik Salomonsson. Model-averaging using a cross-validation weighting method. Unpublished MS-thesis. 2019. Uppsala University.
[9] Aoki Y, Röshammar D, Hamrén B, Hooker AC. Model selection and averaging of nonlinear mixed-effect models for robust phase III dose selection. J Pharmacokinet Pharmacodyn. december 2017;44(6):581–97.
Reference: PAGE 29 (2021) Abstr 9615 [www.page-meeting.org/?abstract=9615]
Poster: Methodology - Model Evaluation