Estelle Chasseloup (1), Adrien Tessier (2), Mats O. Karlsson (1)
(1) Dept. of Pharmaceutical Biosciences, Uppsala University, Uppsala, Sweden; (2) Division of Clinical Pharmacokinetics and Pharmacometrics, Institut de Recherches Internationales Servier, Suresnes, France
Objectives Controlling the type I error of the clinical trials designed to evidence drug efficacy is critical during drug development, to avoid unnecessary and unethical studies. To analyse the results of such proof-of-concept (POC) studies, non-linear mixed-effect models (NLMEM) are a good alternative to the classical approaches and are increasingly recommended. NLMEM are very efficient in terms of power for the POC studies[1,2], but inflated type I error rate, as a consequence of model misspecifications, is a main drawback. In this work, we used real data to compare both the type I error rate and the bias in drug effect estimates, together with their sensitivity to model misspecifications, for two NLMEM-based approaches: the standard approach, and a new approach using mixture models called individual model averaging (IMA)[3].
Methods Two real data set were used: pain score from patients with neuropathic pain receiving placebo[4,5], and the severity test ADAS-Cog from the natural history of patients with Alzheimer’s disease[6]. The type I error rate was computed as the frequency with which there is a significant improvement in the model’s likelihood, according to the likelihood ratio test (LRT; =0.05), by allowing different predictions based on the randomized treatment. By randomly (1:1) assigning patients in the above studies (n=800 for the ADAS-Cog data, and n=230 for the pain data) to “drug” and “placebo” treatment, we created data sets where any significant drug effect is known to be a false positive. Repeating the process of random assignment and analysis for significant drug effect many times (n=1000) for each placebo-drug model combination, statistics of the type I error were obtained.
For each combination in the standard approach, the base (only placebo effect) and the full (placebo and drug effect) models were compared. In the IMA approach, two submodels (placebo and placebo+drug effect) were present in both the base and the full model. In the base model, the probability for each subject to be described by one of the two models was 0.5. In the full model, the association between the treatment and the probability of a submodel allocation was estimated, using the treatment as a covariate.
Results The results showed that the IMA approach had a better type I error control than the standard approach, both on the pain and the ADAS-Cog data. 32 placebo-drug model combinations were tested on the pain data, and 110 on the ADAS-Cog data. For the pain data, the type I error rate of the standard approach was (minimum, 25th, 50th, 75th percentile, maximum) 6.2, 41.8, 96.3, 100.0,100.0; and 2.5, 4.2, 4.9, 5.6, 6.3, for the IMA approach. For the ADAS-Cog data, the type I error was 3.6, 8.8, 26.4, 100.0, 100.0, and 0.3, 2.2, 3.5, 4.2, 5.8, for the standard and the IMA approach respectively. In terms of bias in the drug effect estimates, the IMA showed no bias, whereas in the standard approach the bias was frequently present.
Conclusion The results showed that the IMA approach has a better type I error control and less biased drug effect estimates than the standard approach. The IMA approach was more robust towards misspecifications in the placebo model. Using IMA to analyse confirmatory trials seems a promising method to address the drawbacks of the standard NLME approach.
References
[1] Karlsson KE, Vong C, Bergstrand M, Jonsson EN, Karlsson MO. Comparisons of Analysis Methods for Proof-of-Concept Trials. CPT Pharmacomet Syst Pharmacol. 2013 Jan;2(1):e23.
[2] Jonsson EN, Sheiner LB. More efficient clinical trials through use of scientific model-based statistical tests. Clin Pharmacol Ther. 2002 Dec;72(6):603–14.
[3] Tessier A, Chasseloup E, Karlsson MO. Use of mixture models in pharmacometric model-based analysis of confirmatory trials: part I – simulation study evaluating type I error and power of proof-of-concept trials, Abstract 9233. Poster presented at: PAGE 28; 2019; Stockholm, Sweden.
[4] Plan EL, Elshoff J-P, Stockis A, Sargentini-Maier ML, Karlsson MO. Likert pain score modeling: a Markov integer model and an autoregressive continuous model. Clin Pharmacol Ther. 2012 May;91(5):820–8.
[5] Schindler E, Karlsson MO. A Minimal Continuous-Time Markov Pharmacometric Model. AAPS J. 2017;19(5):1424–35.
[6] Ito K, Corrigan B, Zhao Q, French J, Miller R, Soares H, Katz E, Nicholas T, Billing B, Anziano R, Fullerton T, Alzheimer’s Disease Neuroimaging Initiative. Disease progression model for cognitive deterioration from Alzheimer’s Disease Neuroimaging Initiative database. Alzheimers Dement J Alzheimers Assoc. 2011 Mar;7(2):151–60.
Reference: PAGE 28 (2019) Abstr 9149 [www.page-meeting.org/?abstract=9149]
Poster: Methodology - New Modelling Approaches