Daniel Wojtyniak (1,2), Jinju Guk (2), Sebastian G. Wicha (1)
(1) Dept. of Clinical Pharmacy, Institute of Pharmacy, University of Hamburg, Germany, (2) TMCP Data Sciences, Boehringer Ingelheim Pharma GmbH & Co. KG
Objectives:
When assessing treatment effects in clinical studies by pharmacometrics modelling, type 1 error (T1) may be inflated in presence of model misspecification using conventional approaches [1]. Individual model averaging (IMA) was introduced as a technique to control the T1 rate under model uncertainty [1]. The IMA method was developed using real clinical data, but the impact of different model properties, or study design components on the performance of the IMA method is yet unknown.
The aim of this simulation study was to perform a sensitivity analysis on the T1 for the standard approach (STD) and IMA, varying systematically used drug and placebo (PLC) models, PLC interindividual variability (IIV)-, drug-effect-, PLC-effect-and sample-size.
Methods:
A simulation study for a hypothetic antidiabetic drug was conducted using the $PRED function to describe the change in fasting blood glucose over time in hours (h) for a PLC- and a treatment-(TRM) arm. The baseline value was set to 130 mg/dL with a standard deviation of 3. The data was simulated and re-estimated using NONMEM (version 7.5.0).
To calculate the T1 rate a base- and a full-model for STD (no drug effect, drug effect) as well as IMA (as done in the paper from Chasseloup et al. [1]) were built.
The datasets were then simulated for PLC- and TRM-arm and the arm allocation was permuted, thereby making every observed drug effect random. Then the parameters of both base- and full-model for IMA as well as STD were estimated, and the resulting objective function values (OFV) were compared. The T1 rate is the percentage of which the full model described the data wrongly better than the base model, which means that the full model had a OFV difference compared to the base model higher than the chi-square distribution limit at the significance level α=0.05. The following degrees of freedom (df) for each method were used, while performing 2000 simulations for each scenario. IMA df=1, STD df=corresponding to the number of parameters added in the drug model.
For PLC data, offset-, slope-, power-, and Emax-model were used while IIV was considered. For TRM data, the same model structure was considered on top of PLC model specified in PLC data. For each model, with and without IIV.
To obtain similar effect sizes for each PLC-and drug-model, a PLC effect- as well as a drug effect-dataset was created and all different combinations of TRM- and PLC-model parameters were estimated using this dataset (with mean fasting blood glucose changes over 72 h of -7.6mg/dL for the TRM-,and of -3.5 mg/dL for the PLC-arm).
To test the influence of the variability in response of PLC group, 4 different PLC IIV sizes (0%, 22%, 65%, 125%) were tested.
On top, the influence of drug effect-, PLC effect- and study-size (varying number of patients and measurements per patient) was assessed. For each scenario only one parameter was adjusted to evaluate the influence on the T1 rate. Drug effect sizes tested ranged from a slope per h of 0.005 to 0.18, PLC effect sizes tested ranged from an offset of 0 to 15, study sizes tested ranged from 10 to 200 patients with 5 or 10 measurements over 72 h.
Results:
Overall IMA showed lower T1 rates than STD. IMA T1 rates inflated for certain scenarios a bit, but not above 38.5% while STD T1 rates were often inflated up to 100%.
In the 32 evaluated scenarios, for different drug and PLC model combinations, IMA showed less inflated (IMA T1 > 6.5%: 10/32, STD T1 > 6.5%: 27/32) and often lower T1 rate than STD (IMA T1 > STD T1: 4/32) (min.-, median-, max.-T1 rate: IMA: 3.9%, 5.63%, 28.7%; STD: 1.5%, 93.4%, 100%).
IMA and STD T1 rates were only significantly altered by PLC IIV size in one of the six tested scenarios, namely for the combination of power with IIV drug- and PLC-model. (min.-, max.-PLC IIV size T1 rate: IMA: 38,5%, 14.9%, STD: 66.9%, 85,7%)
For different drug effect sizes IMA showed no significant T1 rate inflation, while for STD the T1 rate decreased with decreasing drug effect size (min., max.-drug effect size T1 rate: IMA: 4.5%, 5.8%; STD: 10.1%, 100%).
PLC effect size seemed to not alter IMA or STD T1 rate significantly.
For larger study sizes the T1 rate within IMA decreased- , while within STD the T1 rate highly increased.(min.-, max.-study size T1 rates: IMA: 8.8%, 4.5%; STD: 30.8%, 100%).
Conclusions:
This study showed that IMA have reduced T1 inflation. On top, the T1 rate is also more stable to changes in influential factors than STD.
References:
[1] Chasseloup E, Tessier A, Karlsson MO. Assessing Treatment Effects with Pharmacometric Models: A New Method that Addresses Problems with Standard Assessments. AAPS J. 2021;23:63.
Reference: PAGE 32 (2024) Abstr 11016 [www.page-meeting.org/?abstract=11016]
Poster: Methodology - Other topics