Ron Keizer1, Paola Mian2, Michael McCarthy1, Jordan Brooks1
1InsightRX, 2University Medical Center Groningen
Introduction Many population PK models are being developed from routinely collected, observational data, such as TDM data—especially those for use in precision dosing. When covariate effects are studied but identified as non-significant in such opportunistic datasets, it is often unclear whether the failure to reject the null hypothesis was due to an effect being truly absent, or whether the dataset and analysis was underpowered to identify the effect. Conversely, when an effect is found to be (borderline) significant, it is often unclear whether the effect is in fact spurious or not. Prospective design calculations are used extensively within pharmacometrics when designing a clinical trial, typically in the form of power analysis implemented using simulations (stochastic simulation and re-estimation, sse) or derived approaches (e.g. Monte-Carlo mapped power, mcmp).[1] However, design calculations can also be performed retrospectively to contextualize the results of an analysis, and may be particularly beneficial for observational analyses where prospective design evaluation is not possible. In the broader field of statistics, Gelman et al have presented this concept as “design analysis”, in which also type S (sign) and type M (magnitude) errors were proposed as informative outcomes.[2] Importantly, design analysis is different from the calculation of “retrospective power” or “posthoc power”, in which the value of the analysis’ test statistic is in some way back-calculated to a power. This latter approach is a fallacy, as demonstrated by many.[e.g. 3] We show here that retrospective design analysis (rDA), in contrast, is a valid and useful approach for observational pharmacokinetic analyses as it can provide valuable information with regards to covariate inclusion or exclusion. Methods In two recent popPK analyses, decisions were required around the inclusion of covariates, in which covariate effects were non-significant or borderline significant. Analysis 1 looked into the effect of pregnancy on PK parameters for two separate datasets for two monoclonal antibodies in IBD (Unpublished). No significant effect of pregnancy was identified for either mAb, but it was unclear whether the datasets were sufficiently informative to conclude that. Analysis 2 concerned a popPK analysis aimed to update an existing model for busulfan (Unpublished). In the earlier popPK analysis co-medication [4] was identified as a significant covariate on drug clearance. However, in a more recent analysis the covariate was still found to be significant but the sign of the covariate had flipped. We implemented an rDA to evaluate the power of the collected data to identify the parameter-covariate relationship. Both rDAs were implemented using the sse tool in PsN[5], based on the original datasets and the final model. A range of effect sizes (-50% to +50%) for the binary covariates on clearance (CL) and/or volume (V) were simulated and re-estimated. For each simulated dataset the likelihood ratio test (LRT) was performed using the model with and without the covariate effect. The power for each assumed effect size was calculated as the % of runs in which the LRT was significant. Results Plots of retrospective power versus assumed effect sizes were useful to understand the ability of the specific dataset to identify covariate effects of interest. Especially the calculation of the power at predefined clinical relevance cutoffs (i.e. 20% effect) was useful to identify whether the study was underpowered or not. The rDA for analysis 1 revealed that the dataset for one mAb was in fact not sufficiently powered to identify clinically relevant effects for CL and Vd at p<0.05, but that the dataset for the second mAb was. For analysis 2, the rDA revealed that the previous analysis was in fact sufficiently powered and the effect not inflated (low type M error), and that covariate inclusion into the final model was justified at the time. Discussion rDA offers useful information in observational analyses where prospective design evaluation is not possible. While its use may benefit all observational analyses, rDA is especially useful when data is sparse. rDA is explicitly not proposed as a substitute for proper prospective study design.
Reference: PAGE 33 (2025) Abstr 11497 [www.page-meeting.org/?abstract=11497]
Poster: Methodology - Covariate/Variability Models