Elham Haem (1), Mats O. Karlsson (2), Sebastian Ueckert (2)
(1) Department of Biostatistics, School of Medicine, Shiraz University of Medical Sciences, Shiraz, Iran, (2) Pharmacometrics Research Group, Department of Pharmacy, Uppsala University, Uppsala, Sweden
Objectives: Many clinical trial endpoints are measured with composite scales which provide a measure of disease severity. A composite scale consists of several assessments or questions that generates a total score. Item response theory (IRT) models, as a gold standard method, are applied in a pharmacometric framework to analysis composite scale [1-3]. However, this method is complex to develop and also item level data may not be available [4]. Thus, it seems crucial to focus on approaches that model total score of a composite scale. There are several dedicated approaches to describe total score data: continuous variable (CV), bounded integer (BI) [5], coarsened grid (CG) [6]and IRT-informed models [7].
One of the most common of total score models is continuous variable (cv) model which treats the outcomes as a continuous variable while the underlying data has a categorical or integer nature. CV models are easy to implement but they might cause a problem especially at the scale boundaries. [5]. BI models respect the boundaries and data nature by a latent grid defined by quantiles of the normal distribution[8].
An alternative approach is to use logit transformation for bounded outcome scores which called coarsened grid (CG) model. [6]. Moreover, in CG model, an additional complex link function from Czado family can modify both tails of the standard logit transformation [6].
Recently IRT-informed model was presented as a method for describing total score data when there exists an IRT model for the same composite scale. In IRT-informed CV (I-CV) model the mean and variability at each latent variable value in an IRT model are computed through the item characteristic curves. As well, IRT-informed BI (I-BI) model allows a direct translation between the latent variable of the IRT model and the expected mean and SD were derived on the Z score scale [7].
In the current work, we compared total score models through a simulation study in terms of power and type-1 error to detect a drug effect.
Methods Simulation model & design: A previously published IRT model [9] was used to simulate MDS-UPDRS motor data from a clinical trial setting in Parkinson’s disease during two years. In this clinical trial, 50 subjects were simulated in each arm and followed across 5 visits. The simulation was performed under two settings: (i) simulation with a drug effect to evaluate power and (ii) simulation without a drug effect to evaluate the type-1 error. For the power evaluation, a drug effect was chosen to achieve approximately 80% power with the true IRT model. In both settings, 250 samples were generated to calculate the power and type-1 error.
Estimation models & hypothesis test: Six different models for total score data were compared in this study: CV, BI, CG, CG with Czado transformation (CG_Czado) as well as IRT-informed CV and BI models (I-CV and I-BI). The IRT-informed models were generated using the true as well as using misspecified IRT models. The log-likelihood ratio test was used to test for a significant drug effect.
Software: The total score models were generated through the piraid package in R software. Afterwards the simulation and the estimation of the models were performed in NONMEM with the help of PsN. The SAEM algorithm was used as an estimation algorithm followed by importance sampling to evaluate the log-likelihood.
Results The power of drug effect detection for the true IRT, CV, BI, CG, CG_Czado, I-CV and I-BI were 74, 61, 60, 49, 65, 66 and 69 percent ,respectively. Both IRT-informed models had a higher power than the corresponding standard model. The I-BI model was more powerful than the I-CV model while they have the same type-1 error. The CG_Czado model had a high power but an inflated type 1 error (8%). BI modeling showed nearly similar power to the CV approach.
The power for the misspecified I-CV and I-BI models, did not change significantly when only the difficulty parameter was changed. However, misspecifying the discrimination parameters remarkably decreased the power of IRT-informed models and increased type-1 error of the I-CV model.
Conclusion The choice of the modeling approach for a composite score based analysis can have a significant impact on the power to detect a drug effect. The use of IRT-informed models can considerably increase the drug effect detection power.
References:
[1] Schindler, E., Friberg, L. E., Lum, B. L., Wang, B., Quartino, A., Li, C., et al. (2018). A pharmacometric analysis of patient-reported outcomes in breast cancer patients through item response theory. Pharmaceutical research, 35(6), 1-14.
[2] Krekels, E. H. J., Novakovic, A. M., Vermeulen, A. M., Friberg, L. E., & Karlsson, M. O. (2017). Item Response Theory to Quantify Longitudinal Placebo and Paliperidone Effects on PANSS Scores in Schizophrenia. CPT: Pharmacometrics & Systems Pharmacology, 6(8), 543-551.
[3] Haem, E., Doostfatemeh, M., Firouzabadi, N., Ghazanfari, N., & Karlsson, M. O. (2020). A longitudinal item response model for Aberrant Behavior Checklist (ABC) data from children with autism. Journal of pharmacokinetics and pharmacodynamics, 47(3), 241-253.
[4] Wellhagen, G. J., Karlsson, M. O., & Kjellsson, M. C. (2021). Comparison of precision and accuracy of five methods to analyse total score data. The AAPS journal, 23(1), 1-10.
[5] Wellhagen, G. J., Kjellsson, M. C., & Karlsson, M. O. (2019). A bounded integer model for rating and composite scale data. The AAPS journal, 21(4), 1-8.
[6] Hu, C., Yeilding, N., Davis, H. M., & Zhou, H. (2011). Bounded outcome score modeling: application to treating psoriasis with ustekinumab. Journal of pharmacokinetics and pharmacodynamics, 38(4), 497-517.
[7] Wellhagen, G. J., Ueckert, S., Kjellsson, M. C., & Karlsson, M. O. (2021). An Item Response Theory–Informed Strategy to Model Total Score Data from Composite Scales. The AAPS journal, 23(3), 1-10.
[8] Ueckert, S., & Karlsson, M. O. (2020). Improved numerical stability for the bounded integer model. Journal of pharmacokinetics and pharmacodynamics, 1-11.
[9] Buatois, S., Retout, S., Frey, N., & Ueckert, S. (2017). Item Response Theory as an Efficient Tool to Describe a Heterogeneous Clinical Rating Scale in De Novo Idiopathic Parkinson’s Disease Patients. Pharmaceutical research, 34(10), 2109-2118.
Reference: PAGE 29 (2021) Abstr 9788 [www.page-meeting.org/?abstract=9788]
Poster: Methodology - New Modelling Approaches