Stepwise Covariate Modelling (SCM) versus Full Random Effects Modelling (FREM) for covariate modelling of correlated covariates exemplified with tigecycline and liver function markers
Lisa F. Amann (1), Rawan Alraish (2), Astrid Broeker (1), Magnus Kaffarnik (2), Sebastian G. Wicha (1)
(1) Dept. of Clinical Pharmacy, Institute of Pharmacy, University of Hamburg, Hamburg, Germany (2) Charité—Universitätsmedizin Berlin, Department of Surgery, Campus Charité Mitte, Berlin, Germany
Objectives: Covariate analysis in small datasets or in datasets with correlated covariates is challenging. This study compared the automated covariate selection methods SCM and FREM by power of including the true covariate (COVtrue), precision and accuracy for different number of individuals (IDs) and grades of correlation of covariates. Moreover, the predictive performance and clinical relevance of a SCM- vs. a FREM-based modelling approach was exemplified with a clinical dataset of tigecycline (TGC) in liver failure, as hepatic covariates are often correlated.
Methods: A model including the COVtrue was used to simulate datasets which were utilized for SCM and FREM (n=1000). Covariate correlation (cov-corr) of 15%, 50% and 80% for 20, 50, and 100 IDs were investigated. The dataset included three covariates, of which two were correlated. The COVtrue had an effect of 25% from the 5th to 95th percentile of covariate values on clearance. For the SCM method, the FOCE-I based likelihood ratio test with forward inclusion (α<=0.05) and backward elimination was employed (α<=0.01)[2]. FREM treats covariates as observations and the effects of all covariates on clearance were estimated simultaneously through co-variances within the ?-matrix [3,4]. The power of each method was calculated as frequency when COVtrue was selected (SCM), or the CI95% of the estimate of COV did not overlap with zero (FREM). Moreover, relative bias (rbias) and relative root mean squared error (rRMSE) were used to compare the accuracy and precision of the estimate of COVtrue . The clinical dataset included 39 patients with 13 covariates (correlation: 0 – 78%).
To investigate the predictive performance for the SCM and FREM model, derived from the clinical dataset, the results were compared in stratified prediction-corrected visual predictive checks (pcVPC). All data was analysed using NONMEM 7.5 and PSN version 4.5.0.
Results: With 15% cov-corr the power of the SCM to select COVtrue was 19%, 55%, 89% for 20, 50, 100 IDs, respectively. In contrast to that, the power to detect COVtrue with the FREM was 51%, 80%, 98%. As cov-corr was increased to 80%, the final SCM model included COVtrue in 16%, 45%, and 78% for 20, 50, 100 IDs, respectively. On the other hand, the FREM found COVtrue with 80% correlation in 43%, 67%, 85% for 20, 50, 100 IDs. The rbias for 20 IDs was 3.3% for 80% correlated covariates in the FREM model and 81.2% for the SCM. In the scenario with 15% cov-corr and 100 IDs the bias of the SCM were reduced to 4.5%, but still much higher as for the FREM (-0.3%).
Evaluating the predictive performance in the clinical dataset, the pcVPC’s indicated a better alignment of predicted vs. observed PK profiles for selected covariate stratifications from the FREM model compared to the SCM model.
Conclusions: The power to include the COVtrue in the SCM or FREM model was dependent on the number of IDs and much smaller for the SCM in presence of correlated covariates compared to the FREM. The results showed, that FREM detected the COVtrue more often in small n datasets. A strong selection bias for SCM with overestimation of the covariate effect was observed. Moreover, the SCM suffers from unknown ‘true’ p-values due to multiple testing. Due to the automated SCM process, these problems are shaded and partly subjective scientific or clinical interest usually guides the selection of covariates to the final model. In the clinical dataset, a better predictive performance of the FREM model was observed as compared to the SCM model. Hence, the FREM approach represents an attractive modelling approach for studies with a low number of subjects and presence of correlated covariates.
References:
[1] Delcò F, Tchambaz L, Schlienger R, Drewe J, Krähenbühl S. Dose adjustment in patients with liver disease. Drug Saf 2005; 28:529–45.
[2] PsN. SCM user guide. PsN 2015; 4.7.0:1–23.
[3] Nyberg J, Jonsson EN, Karlsson MO, Häggström J. Properties of the full random effect modelling approach with missing covariates. BioRxiv 2019:656470.
[4] FREM userguide 2018:1–19c