Capacities of NPDE, VPC and pvcVPC at detecting model misspecification: a simulation study of a PK model showed no apparent difference
Anaïs Glatard (1), Thomas Dumortier (1), Jean-Louis Steimer (1), Céline Sarr (1)
1. Pharmacometrics, Novartis, Basel, Switzerland
Objectives: Simulation-based methods such as Normalised-Prediction Distribution Errors (NPDE)1, Visual Predictive Check (VPC) and prediction- and variability-corrected VPC (pvcVPC)2 are commonly used to evaluate pharmacometric models. There is a lack of clarity about the respective capacities of these methods at detecting model misspecification. For instance, pvcVPC intends to remove the covariate-induced heterogeneity in prediction distribution by prediction and variability correction2. But, by design, this correction may possibly not remove all the heterogeneity in case of model non-linearity; if this is so, pvcVPC would display wide simulated percentiles 95%CI and show weaknesses at rejecting a wrong model. The capacity of these different methods at detecting a misspecified model was investigated in the context of high non-linearity and covariate heterogeneity.
Methods: A 2-compartment and a 1-compartment PK models (true and wrong models, respectively) were defined in 2 situations: 1. the heterogeneity was introduced by an allometric relationship of the weight on the peripheral volume of distribution for the true model and on the central volume of distribution for the wrong model; 2. unrealistic covariate values were used to test the prediction and variability correction in extreme cases of heterogeneity and the covariate was introduced as a linear relationship on the absorption constant. In both situations, 1000 simulations of prediction distributions per covariate level were performed before and after correction, with both models using R. A dataset including 5 subjects per covariate level with the true model was simulated and used to estimate the true and the wrong model and to generate NPDE, VPC and pvcVPC (using NONMEM and PsN).
Results: The results for the situation 1 and 2 were similar. After correction and across covariate levels, the distributions of the predictions obtained by the true (or the wrong model) have same mean and same standard error. But the distributions still had some degrees of heterogeneity in shape. The wrong model was rejected by the 3 methods after visual inspection of the respective plots.
Conclusions: Although the prediction and variability correction did not completely remove the heterogeneity of the predictions distribution across covariate levels, the remaining heterogeneity did not prevent the pvcVPC method to detect model misspecification. Similar conclusions would be reached with NPDE and pvcVPC for the model building decision.
References:
[1] Comets E, Brendel K, Mentre F. Computing normalized prediction distribution errors to evaluate nonlinear mixed-effects models: the npde add-on package for R. Computer Methods and Programs in Biomedecine, 2008, 90:154-166.
[2] Bergstrand M, Hooker AC, Wallin JE, Karlsson MO. Prediction-Corrected visual predictive checks for diagnosing nonlinear mixed-effects models. AAPS Journal, 2011, 13(2):143-151
Acknowledgements This work was supported by the DDMoRe project (www.ddmore.eu).
