Sandra Witta1, Sebastian G. Wicha1
1University of Hamburg
Introduction: A primary challenge for the integration of model informed precision dosing (MIPD) in a clinical setting is selecting the best-fit model for a newly presented patient. A multi-model averaging algorithm (MAA) has been proposed by Uster et al., to alleviate the difficulty of model selection through leveraging a combination of models, facilitating the MIPD approach [1]. However, with the further development of models in varying populations, the “library” of models available for a given therapeutic expands and the ‘true’ model remains unknown. An understanding of how model misspecifications influences MAA can provide insight to guide optimal model combinations. The presented simulation study thus aims to evaluate the individual impact of parameter model misspecifications in the context of MAA. Methods: A simulation study was conducted using nlmixr2 (version 2.1.1) [2] and the MAA algorithm from Uster et al [1]. A 1-compartment linear elimination model with interindividual variability on clearance and volume and a proportional residual error was parameterized according to 3 cases to mimic common heterogeneity in model structures: Different (i) typical clearance value, (ii) interindividual variability (IIV) and (iii) residual error and (iv) combinations of simultaneous parameter differences. Each case consisted of 6 sub-models, for which the parameters were varied to mimic a heterogenous patient population. Parameterization of the sub-models for cases i-iii incremented from 1- 6 L/h (i), and 10-60 %CV (ii and iii) respectively, while all other parameters were constant. Each sub-model was used to simulate peak and trough concentrations for 100 virtual patients each following a 4th and 6th dose yielding a total population size of 600 for each case i-iv. The predictive performance was then evaluated using each sub-model individually, the MAA approach using sum of squared residuals (SSE) and objective function value (OFV) weighting schemes, and a re-estimated single model on the heterogenous dataset to evaluate. Therefore, the relative prediction error (PE) and both accuracy and precision using median prediction error (MPE) and median absolute prediction error (MAPE), respectively, were assessed. Results: Predictive performance for the case of misspecification of a structural model parameter (scenario i) substantially varied for single sub-models in comparison to MAA using all models (MPE: -11.3%-35.1% vs -1.56% and -1.59%, MAPE: 9.9-35.1% vs. 9.39% and 9.56% for OFV and SSE, respectively). Inclusion of only two models in MAA with opposing precision bias and largest error in accuracy (MPE: 35.1 and -17.2, MAPE: 35.1 and 20.66) resulted in performance comparable to inclusion of all models and the re-estimated model (MPE: 1.30% for n=2 with SSE weighting, -2.55 for re-est., MAPE: 10.8, 9.44). MAA with models of same polarity for bias approached performance of the least biased model down-weighting the impact of a highly biased model. MAA with regards to models misspecified in an IIV parameter (scenario ii), showed improvement in minimizing outliers compared to the re-estimated model and a tendency to add more weight to models with greater IIV on clearance, improving performance for simulated patients poorly described by the model through the “flattening of priors”. In the case of a misspecification in residual error (scenario iii), re-estimation of parameters showed greater bias in prediction in contrast to MAA (MPE: -5.70 vs. 0.505 for n=2 with SSE weighting). Notably, the results for the weighting schemes in this scenario showed greater discrepancies between SSE and OFV weighting, where SSE weighting intrinsically preferred the model with the lowest residual error, while OFV weighting allocated higher weights towards the appropriate model as in the other scenarios. Conclusion: The presented simulation study provides insight into the effect of model parameter misspecifications in the context of the recently proposed MAA. The findings presented herein can be used for guiding selection of optimal model combinations.
[1] D. W. Uster et al., “A Model Averaging/Selection Approach Improves the Predictive Performance of Model-Informed Precision Dosing: Vancomycin as a Case Study,” Clin. Pharmacol. Ther., vol. 109, no. 1, pp. 175–183, Jan. 2021, doi: 10.1002/cpt.2065. [2] M. Fidler et al., “nlmixr2: Nonlinear Mixed Effects Models in Population PK/PD.” May 30, 2024. Accessed: Jul. 15, 2024. [Online]. Available: https://cran.r-project.org/web/packages/nlmixr2/index.html
Reference: PAGE 33 (2025) Abstr 11750 [www.page-meeting.org/?abstract=11750]
Poster: Methodology - Model Evaluation