Comparison of Recursive Partitioning Analysis and Receiver Operating Characteristic (ROC) Analysis for Patient Identification
Shang-Chiung Chen, Matthew Riggs, Eveline Nueesch, Alexander Strasak, Jonathan French, Jin Jin, Sandhya Girish, Chunze Li
Genentech, Inc.
Objectives: Exposure–response (E-R) analyses have become an integral part in oncology drug development and have played a critical role in identifying patients with low exposure who might benefit from an alternative dose and/or regimen. Therefore, reliably and consistently identifying these patients is important to allow evaluation of alternative dose and/or regimen in these patients. The objective of this analysis is to compare two methodologies using a case study, namely recursive partitioning analysis and receiver operating characteristic (ROC) analysis, to identify the key baseline predictors of patients with low exposure
Methods: Methodologies for patient identification were exemplified using data from an oncology Phase 3 trial. All data exploration and analyses were conducted using version 3.2.1 of R. For recursive partitioning analysis, two packages (rpart and party) were used and different analysis-level variables were explored, including minimum bin sizes (range: 15–40) and the inclusion/exclusion of missing dependent variable values. Prediction of low exposure patients by baseline characteristics was also evaluated using receiver operating characteristics (ROC) analysis and the corresponding area under curve (AUC)
Results: Results from two recursive partitioning packages were typically similar, with shed antigen, tumor burden, SGOT, and albumin as the most prevalent baseline patient characteristics chosen. Across various analysis-level variables evaluated for recursive partitioning analysis, only one level of split can be reached except for small minimum bin sizes, and the cut-offs of the baseline patient characteristics for the splits are comparable. The resulting patient identification criteria, in general, had low (range 23.5 - 44%) sensitivity and relatively high (range 81.3 – 97.9%) specificity. Visualized by ROC curve and corresponding AUC, the baseline patient characteristics predictive of low exposure were consistent to those identified by recursive partitioning.
Conclusions: Two methodologies commonly used for patient identification resulted in similar set of baseline patient characteristics predictive of low exposure. While recursive partitioning analysis can provide initial cut-offs of baseline characteristics for patient identification, the cut-offs may be further refined based on ROC curve for logistical purpose. Reliability to identify low exposure patients in an external population remains to be evaluated.