Metrics based on objective function for external validation of a population pharmacokinetic model
K. Brendel(1), E. Comets(1), C.Laveille(2), C. Laffont(2), F. Mentré(1)
(1)INSERM U738, Paris, France, (2) Servier, Courbevoie, France
Objectives: An important step in population pharmacokinetic model building is to evaluate the model’s adequacy. We propose 3 metrics based on objective function for external validation. These metrics were applied to four validation datasets: three simulated datasets and a dataset from a phase I study.
Methods: Let MB be a model and V a validation dataset. We defined 3 metrics for model validation based on the objective function: 1) Prediction Error on Objective Function (PEOF), which is the difference between the objective function (OF) determined in V with MB before fitting (OFnofitV; all parameters fixed) and after fitting (OFfitV ; all parameters estimated); 2) Prediction Error on Objective Function with Simulation (PEOFS), in which OFnofitV is compared to the posterior predictive distribution of the objective function estimated from k datasets simulated with MB; 3) A third approach compares the PEOF with the posterior predictive distribution of the difference between OFnofitV, k and OFfitV, k for the kth simulated dataset (ΔOFV, k) .
In the present case, MB was a one-compartment model with zero-order absorption and first-order elimination, built from two phase II studies. The metrics defined above were applied on 4 validation datasets, the real data (Vreal) and 3 datasets simulated according to the design of Vreal: the first (Vtrue) was simulated using MB; the second and third datasets were simulated using the same model, but with a bioavailability multiplied by two (Vfalse1) or divided by two (Vfalse2) .
A likelihood ratio test was performed for PEOF with a p-value of 0.05. For the metrics with simulations, we calculated the p-value based on the empirical distribution. The pharmacokinetic evaluations and simulations were performed with NONMEM and SAS was used for statistical tests.
Results: The 3 metrics performed similarly on the 4 datasets. They all rejected Vfalse1, Vfalse2 (p<0.001) as well as Vreal, but not Vtrue. When plotting the empirical posterior distribution of the two metrics based on simulations (PEOFS, ΔOFV), ΔOFV appeared to have a better discriminant power. Also, as it is applied on exactly the same kth simulated dataset, this metric handled BQL observations more appropriately than PEOFS. PEOF provides a simple alternative in this example and was efficient here to detect model inadequacy.