Inga Ludwig, Guenter Heimann, Sebastian Weber, Thomas Dumortier
Novartis
Objectives:
Recruitment of patients into pediatric studies is difficult and slow, hence fully powered pivotal trials are prohibitive.
For indications and drugs where the disease progression in children is similar to that in adults, and where the pharmacology of the drug is similar to that in adults, one may fully extrapolate efficacy from adults to children. It suffices to demonstrate that the selected pediatric dose provides comparable plasma concentrations as the registered adult dose, and to demonstrate adequate safety.
Often, there is not yet enough evidence to apply full extrapolation, and one may want to apply a partial extrapolation approach. Here, one needs to collect some efficacy data in children to demonstrate that the adult and the pediatric efficacy are similar. However, for ethical and practical reasons one wants to avoid large studies in children, and one often cannot include a comparator group.
The goal then is to demonstrate similarity between adult and pediatric efficacy. We use the adult data set to develop an exposure-response model for the clinical outcome. Such a model accounts for exposure and baseline risk factors. The model is used to predict the clinical outcome of the children, conditional on their observed exposures and covariates. These predicted outcomes are then compared with the observed outcomes in children, to validate that the adult model is adequate to predict pediatric efficacy.
The objective of this paper is to develop quantitative methods to compare such predicted with observed outcomes, and to understand the operating characteristics of these methods via simulations. The operating characteristic should improve with increasing sample size.
Methods:
For continuous or time to event data, we generate a prediction distribution for each of the n children in the pediatric data set from an adult model via simulations. These individual prediction distributions will differ from child to child according to the observed exposures and covariates. If P1 is the prediction distribution for the outcome of child 1, and Y1 is the corresponding observed outcome, then P1(Y1) should be approximately uniformly distributed on the interval [0,1]. Applying this approach to all children provides n uniformly distributed data points.
Note that the prediction distributions may differ between the children, because they are obtained conditionally on the observed exposures, covariates, and censoring times.
We apply a Cramer von Mises goodness of fit test to check whether the data are uniformly distributed. The Cramer von Mises test statistic measures the distance between the empirical distribution function based on the P1(Y1), … Pn(Yn) from a uniform distribution function. If the adult model is a good predictor, then the true distribution of the Pj(Yj) is close to a uniform one, and the Cramer von Mises test statistics should be close to zero. One can use the well-known asymptotic distribution of this test statistic (see [1], [2], and [3]) to obtain a (one-sided) asymptotic confidence interval for deviation from uniformity.
Note that the confidence intervals defined here are closely related and applicable VPCs and the normalized prediction distribution errors (NPDE) as discussed in [6] and earlier by [4] and [5]. In our case, each pediatric subject only contributes one observation Pj(Yj) and hence the issue of decorrelation does not apply.
Results:
Our simulations show that proposed approach works well. This is true when using the asymptotic confidence interval for deviation from uniformity, as well as when using a corresponding bootstrap version. The bootstrap confidence interval has slightly better coverage probabilities for small data sets.
In simulation scenarios where the adult and children data were generated from the same distribution, the confidence intervals get narrower and approach zero when increasing the pediatric sample size to a very large n. In scenarios where the children data were generated from a different distribution, our simulations demonstrate that the width of the confidence interval will not reduce to zero with increasing sample size n. The results were comparable when using completely artificial data, or when simulations were based on real data examples.
Conclusions:
Our simulations demonstrate that the proposed approach for model validation works well. The outcome of the simulations is as expected. The coverage probability of the CI is as desired.
References:
[1] Baringhaus, Ebener, and Henze.The limit distribution of weighted L2-goodness-of-fit statistics under fixed alternatives. Ann Inst Stat Math (2017) 69:969–995.
[2] Baringhaus and Henze. Cramér–von Mises distance – probabilistic interpretation confidence intervals and neighbourhood-of-model validation. Journal of Nonparametric Statistics, 29:2, 167-188.
[3] Baringhaus, Gaigall, and Thiele. Statistical inference for L2-distances to uniformity. Computational Statisitcs (2018) 33:1863–1896.
[4] Brendel, Comets, Laffont, Laveille, Mentré. Metrics for external model evaluation with an application to the population pharmacokinetics of gliclazide. Pharm Res. 2006;23 (9):2036–49.
[5] Karlsson and Savic. Diagnosing model diagnostics. Clin Pharmacol Ther. 2007;82(1):17–20.
[6] Laffont Concordet. A new exact test for the evaluation of PK-PD models using random projections. Pharm Res (2011) 28:1948–1962.
Reference: PAGE 28 (2019) Abstr 9157 [www.page-meeting.org/?abstract=9157]
Poster: Methodology - Model Evaluation