Jasmine Hughes, Dominic Tong, Srijib Goswami, Ron Keizer
InsightRX, San Francisco, CA
Background: Model-informed precision dosing (MIPD) promises to improve drug efficacy and reduce toxicity by tailoring drug regimens for individual patients [1]. Technical challenges in the implementation of MIPD include selection and validation of an adequately predictive model [2]. However, one such challenge is that even validated models will show limited predictive performance in patients with more extreme pharmacokinetic (PK) parameters (i.e., “outlier” patients). An approach to overcome this challenge is to downweigh model priors when fitting individual data, thus relying more on observed data [3]. However, deciding when to flatten the distribution of the priors and by how much is inherently subjective. Machine learning (ML) could assist the clinician in making this and similar point-of-care decisions.
Objectives: Evaluate the feasibility and predictive performance of ML models to assist in clinical decision making. Here, we focus on identifying when model priors should be downweighed to improve predictive performance.
Methods: De-identified data collected during routine clinical care of adult patients from three US hospitals treated with vancomycin and dosed with the InsightRX MIPD platform were used. Data from treatment courses with two or more vancomycin TDM levels were included (n=3820 patients). Two published and externally validated PK models were used to describe vancomycin PK [4,5]. For each sample, patient PK parameters were estimated using MAP Bayes estimation with either the interindividual variability magnitude (IIV) as reported (MAP-R) or by inflating IIV four-fold (MAP-FP). These parameters were then used to predict the value of the subsequent TDM level. Features thought to be related to model predictiveness were calculated for each predicted level, using only data available at the time of the previous level, and used to predicted if MAP-FP would reduce the absolute residual by ≥5%. The dataset was split into testing, training and cross-validation sets. Two ML algorithms were evaluated: random forest (RF) and logistic regression (LR).
Results: Given perfect knowledge of which weighting schema would result in the best prediction, Model 1 [4] residuals could be reduced by a mean of 0.66 mg/L (10th-90th percentile range (p10-90): 0-2.0 mg/L). Of the TDM samples in the test set, 29% were preceded by a TDM with a residual ≥5 mg/L. For these “high error” instances, residuals could be reduced by a mean of 1.3 mg/L (p10-90: 0-3.3 mg/L). Model 2 [5] overall produced slightly higher error (RMSE: 5.5 vs 5.2 mg/L). Given perfect knowledge, Model 2 residuals could be reduced by a mean of 0.87 mg/L (p10-90: 0-2.7 mg/L). 33% of TDM samples were preceded by a TDM with a residual ³5 mg/L. Residuals in high error instances could be reduced by a mean of 1.5 mg/L (p10-90: 0-3.8 mg/L).
The final Model 1 RF accurately classified 62% of test set decisions, significantly outperforming the No Information Rate (NIR) of 0.51 (p < 2E-16). This resulted in a mean improvement of 0.22 mg/L (p10-90: -0.85 – 1.7 mg/L) in residuals if RF recommendations were followed compared to using the reported IIV in all instances. For high error instances, RF lead to more significant improvements: residuals were reduced by 0.75 mg/L (p10-90: -1.9 – 2.9 mg/L).
The final Model 2 RF accurately classified 61% of decisions (accuracy > NIR: p < 2E-16), resulting in a mean improvement of 0.27 mg/L (p10-90: -0.44 – 1.6 mg/L) in residual value if RF recommendations were followed, relative to MAP-R. For decisions with a previous residual ≥5 mg/L, the RF reduced the residuals by 0.44 mg/L (p10-90: -1.5-3.0 mg/L).
The LRs performed similarly to the RFs. The most important features for all ML models were largely consistent between methods: the value of the most recent TDM’s residual, the magnitude of all residuals to date, the rate of change of TDMs over time, patient body mass index (BMI) and age.
Conclusion: ML provided predictive value for the point-of-care decision of downweighing model priors. Good agreement was observed between ML methods with respect to which features were most predictive. Although gathering additional data (TDM, creatinine) should be prioritized in instances with poor model fit, predictive ML models could provide additional guidance for clinicians using MIPD at the bedside. The presented approach could also be applied to other individual point-of-care decisions, like model selection and the weighting of historic patient data.
References:
[1] D. Gonzalez et al., “Precision Dosing: Public Health Need, Proposed Framework, and Anticipated Impact,” Clinical and Translational Science. 2017.
[2] R. Keizer, R. Mangat, and S. Goswami, “Experiences in applied clinical pharmacometrics: challenges, recommendations, and research opportunities,” in PAGE. Abstracts of the Annual Meeting of the Population Approach Group in Europe., 2018, p. Abstr 8758.
[3] R. J. Keizer, R. ter Heine, A. Frymoyer, L. J. Lesko, R. Mangat, and S. Goswami, “Model-Informed Precision Dosing at the Bedside: Scientific Challenges and Opportunities,” CPT: Pharmacometrics and Systems Pharmacology, vol. 7, no. 12. 2018.
[4] A. H. Thomson, C. E. Staatz, C. M. Tobin, M. Gall, and A. M. Lovering, “Development and evaluation of vancomycin dosage guidelines designed to achieve new target concentrations,” J. Antimicrob. Chemother., vol. 63, no. 5, pp. 1050–1057, Mar. 2009.
[5] V. Goti, A. Chaturvedula, M. J. Fossler, S. Mok, and J. T. Jacob, “Hospitalized Patients with and Without Hemodialysis Have Markedly Different Vancomycin Pharmacokinetics: A Population Pharmacokinetic Model-Based Analysis,” Ther. Drug Monit., 2018.
Reference: PAGE () Abstr 9536 [www.page-meeting.org/?abstract=9536]
Poster: Oral: Clinical Applications