Moustafa M. A. Ibrahim (1), Sebastian Ueckert (1), Svetlana Freiberga (1), Maria C. Kjellsson (1), Mats O. Karlsson (1)
(1) Department of Pharmaceutical Biosciences, Uppsala University, Sweden
Background and Objectives: Conditional weighted residuals (CWRES) modelling has been proposed as easily automated diagnostic tool for model development/evaluation process, as it provides guidance for the nature and magnitude of potential model misspecification/improvements. [1] In this work, a method based on CWRES modelling was developed to assess prediction bias by back-extrapolating a CWRES-based bias using the first order conditional estimation (FOCE) approximation. We illustrate this method by assessing prediction bias in two integrated models for glucose homeostasis, the integrated glucose insulin (IGI) model and the integrated minimal model (IMM) [2,3,4]. Both models consist of glucose and insulin sub-models with interconnecting control mechanisms, and were proposed to describe simultaneously the glucose-insulin regulation system following intravenous glucose tolerance test (IVGTT) in healthy subjects.
Methods: One dataset was simulated from each model according to a standard IVGTT protocol, then analyzed by the two models and visual predictive checks VPCs were performed to investigate the goodness of each fit. CWRES outputted from each model fitting was separated based on the two dependent variables (DVs) glucose and insulin, where after CWRES for each DV was modelled by a base model to estimate CWRES distribution’s mean and variance. The base model was then extended to estimate different means for ten bins of the independent variable (IDV) at cutoffs points of the IDV dictated by data-density, this could capture systematic trends in CWRES as well as the magnitude of structural model misspecifications in terms of difference in objective function values ΔOFVBias (when significant) between this model and CWRES base model. The estimates of the bin-specific means were used to calculate the corresponding bias in conditional predictions at each bin of the IDV by the inversion of FOCE-based CWRES equation. TIME, glucose PRED, and insulin PRED were the investigated IDV, and a random binning technique was implemented to avoid binning introduced bias. [5]
Results: When either of the two data sets were analyzed with the IGI model, or data simulated by the IMM was analyzed by the IMM, ΔOFVBias was insignificant for both DVs glucose and insulin. When data simulated by the IGI was analyzed with the IMM, ΔOFVBias was significant for glucose, but not for insulin. Over prediction bias in glucose sub-model was found at early time points (
Conclusions: New method for predication bias assessment based on CWRES was developed and successfully applied to two integrated, complex models for glucose homeostasis. The new method identified correctly the bias in glucose sub-model of the IMM, when this bias occurred, and calculated the magnitude of the resulting bias. This new method is fast and easily automated diagnostic tool for model development process, and it is already implemented as part of the qa tool in PsN.
References:
[1] Ibrahim M. M. A., Nordgren R., Kjellsson M. C., & Karlsson, M. O. Model-based diagnostics post-processing for fast automated model building; show-cased with residual error models and CWRES. PAGE 26 (2017) Abstr 7276 [www.page-meeting.org/?abstract=7276].
[2] Silber H. E., Jauslin P. M., Frey N., Gieschke R., Simonsson U. S., & Karlsson M. O. An integrated model for glucose and insulin regulation in healthy volunteers and type 2 diabetic patients following intravenous glucose provocations. Journal of clinical pharmacology 2007, 47(9): 1159-1171.
[3] Largajolli A., Bertoldo A., Cobelli C., & Denti P., An integrated glucose-insulin minimal model for IVGTT, PAGE 22 (2013) Abstr 2762 [www.page-meeting.org/?abstract=2762].
[4] Ibrahim M. M. A., Largajolli A., Kjellsson M. C., & Karlsson, M. O. Translation between two models; Application with integrated glucose homeostasis models. WCOP (2016) Abstr 249.
https://www.go-wcop.org/2016/translation-between-two-models-application-with-integrated-glucose-homeostasis-models/
[5] Pavan Kumar V. V., & Duffull S. B. Evaluation of graphical diagnostics for assessing goodness of fit of logistic regression models. Journal of Pharmacokinetics and Pharmacodynamics 2011, 38(2), 205–222.
Reference: PAGE 27 (2018) Abstr 8716 [www.page-meeting.org/?abstract=8716]
Poster: Methodology - Model Evaluation