Romain Leroux (1), Emmanuelle Comets (1,2)
(1) Université de Paris, INSERM, IAME, F-75018 Paris, France (2) Université Rennes 1, CIC1414, INSERM, Rennes, France
Objectives: This poster presents the new version of the npde library (version 3.1) for the computation and diagnostic of normalised prediction discrepancies [1].
Methods: The normalised prediction discrepancy (npd) for observation yij is obtained by computing the prediction discrepancy (pd) as the quantile of yij within its predictive distribution, and transforming the pd using the inverse normal function [2]. Normalised prediction distribution errors (npde) are obtained using a similar approach but after decorrelation of both the observed and simulated values, and are needed for statistical tests [3]. npde was first uploaded as an R package on the CRAN in 2007 [4]. npde have since then been extended to handle BQL data using imputation methods [5].
Under the null hypothesis that a model describes the data adequately, both npd and npde should follow a normal distribution. Model adequacy can be checked by plotting their distribution versus their theoretical distribution as QQ-plots or histograms, or by scatterplots versus time or population predictions. Prediction intervals are added to the graphs to illustrate the regions where the model predicts specific quantiles of the data to lie [6].
Results: In the new version of the npde package, the graphs have been reprogrammed using the ggplot2 package to take advantage of this new and powerful graphical language [7]. The library provides a new graph, the transformed npde: these graphs add a reference profile to npd/npde, which helps visualise the shape and variability of the process being modelled [8]. Transformed npde resemble standard VPC plots but retain the good statistical properties of the npde without the need to stratify. The npde package remains fully backwards compatible as the core computational engine remains the same.
We demonstrate the use of the package by producing diagnostic graphs for simulated PK data, using the same simulation example as in [9] of a one-compartmental model with a weight effect on volume of distribution and a treatment effect on clearance. The design included 180 patients grouped in 3 dose levels, and we show the reference profile for the intermediate dose group without treatment effect.
Conclusion: npd/npde are widely used to assess nonlinear mixed effect models, and show good statistical properties [10]. They handle design and covariate heterogeneity naturally. This is particularly valuable in unbalanced designs, and distinguish these metrics from VPC and their many flavors (pcVPC [11],…). Version 3.1 of the npde library is available on CRAN (https://cran.r-project.org/web/packages/npde/index.html).
References:
[1] Comets E and Mentré F (2021). Developing tools to evaluate non-linear mixed effect models: 20 years on the npde adventure. AAPS Journal (accepted).
[2] Mentré F and S Escolano S (2006). Prediction discrepancies for the evaluation of nonlinear mixed-effects models. J Pharmacokinet Biopharm 33:345-67.
[3] Brendel K, Comets E, Laffont C, Laveille C, Mentré F (2006). Metrics for external model evaluation with an application to the population pharmacokinetics of gliclazide. Pharm Res 23:2036-49.
[4] Comets E, Brendel K, Mentré F (2008). Computing normalised prediction distribution errors to evaluate nonlinear mixed-effect models: The npde add-on package for R. Comput Meth Prog Biomed 90:154-66.
[5] Nguyen THT, Comets E, Mentré F (2012). Extension of NPDE for evaluation of nonlinear mixed effect models in presence of data below the quantification limit with applications to HIV dynamic model. J Pharmacokinet Pharmacodyn 39: 499-518.
[6] Comets E, Brendel K, Mentré F (2010). Model evaluation in nonlinear mixed effect models, with applications to pharmacokinetics. Journal of the SFdS 1:106-28.
[7] H. Wickham (2016). ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York.
[8] Comets E, Nguyen THT, Mentré F (2013). Additional features and graphs in the new npde library for R. PAGE 22 Abstr 2775.
[9] Nguyen THT, Mouksassi M, Holford N, Al-Huniti N, Freedman I, Hooker AC, et al (2017). Model evaluation of continuous data pharmacometric models: metrics and graphics. CPT Pharmacometrics Syst Pharmacol 6:87–109.
[10] Brendel K, Comets E, Laffont C, Mentré F. Evaluation of different tests based on observations for external model evaluation of population analyses. J Pharmacokinet Pharmacodyn 2010; 37:49-65.
[11] Bergstrand M, Hooker AC, Wallin JE, Karlsson MO (2011). Prediction-corrected visual predictive checks for diagnosing nonlinear mixed-effects models. AAPS Journal 13:143-51.
Reference: PAGE 29 (2021) Abstr 9664 [www.page-meeting.org/?abstract=9664]
Poster: Methodology - Model Evaluation