Suchaya Sanhajariya1, Stephen Duffull1, Amy S. Y. Cheung1, Yu-Wei Lin1, Samer Mouksassi1,2
1Certara, 2University of California San Francisco
Introduction: Visual predictive check (VPC) is a core simulation-based diagnostic tool for evaluating nonlinear mixed effects models [1]. However, traditional VPC generation can be computationally intensive and time-consuming, particularly when multiple simulation runs are needed for different strata structures. To improve speed and facilitate the construction of VPCs, tidyvpc, an R package [2] was developed as a versatile and comprehensive toolkit for creating both continuous and categorical VPCs. It allows for the creation of multiple VPCs with different settings from only one simulation run. The package employs the data.table package and various performance optimizations to enhance computational efficiency. In addition, it leverages the tidyverse-style syntax, enabling efficient data manipulation and dynamic adaptation of VPCs, allowing for the handling of different binning and stratification methods, data censoring, and prediction correction without the need to run multiple VPC simulations. tidyvpc is software agnostic, meaning simulations can be performed in different software used to fit the data, such as NONMEM and Phoenix. In terms of plot customization, tidyvpc provides all the data needed to create various styles of VPC plots, including rectangles and smoothed plots. The tidyvpc package also integrates with the Certara.VPCResults, an R package [3], which utilizes a Shiny application to provide an interactive user-friendly interface for parameterizing and plotting VPCs. The tidyvpc package is currently available for installation on GitHub [2] and CRAN [4]. Objectives: To explore a use case example of the tidyvpc package for creating continuous VPCs. Methods: An observed and a simulated (N=100) dataset from a hypothetical PK model were used to generate various VPC plots by tidyvpc, using R version 4.3.1. The dataset consisted of 300 IDs with fixed intensive sampling scheduled after the first dose and sparse sampling following the second dose. Two continuous (age, weight) and three categorical (sex, study, patient status) covariates were included. The tidyvpc functions implemented were: •General VPC construction -The process begins with loading and preprocessing observed and simulated datasets, followed by specifying these datasets using the observed() and simulated() functions. Binning methods (e.g. jenks, pams, fisher) were applied using the binning() or binless() function. Bin information can be obtained and used for visualization using bininfo() function. Finally, the vpcstats() function calculates the necessary statistics for VPC visualization, with customizable quantiles. The build-in plot() function allows quick visualization of derived VPCs, although this can also be performed and customized using ggplot2. •VPC stratification -stratify() was used to evaluate model performance across different groups or strata. This package allows for the inclusion of as many stratification variables as deemed appropriate. Binning can also be specified for each subgroup to improve the visualization of the data. •Prediction corrected VPC -predcorrect() function was incorporated prior to the vpcstats() calculation to adjust for biases by normalizing the observed and simulated values within each bin. •Censoring for continuous VPC -censoring() function was used to handle censored data or data points that fall outside the specified quantifiable range (i.e. upper/lower limit of quantification). Different limit of quantifications can be specified for each variable of interest, such as study. Results: More than 100 variants of VPCs could be generated from a simulation file for this example using the tidyvpc package. The VPC set up customizations were performed post-simulation in R. Eleven versions of VPCs were explored: standard VPC (both bin and binless); prediction-corrected VPC (both bin and binless); VPC with censoring (either bin or binless); pcVPC with censoring (either bin or binless); and five categorical VPC versions with the optimal setting from previous evaluations). Once the code is set up, the generation time for these plots was less than 5 minutes. Conclusions: The tidyvpc package offers a fast, flexible and effective approach to generating continuous VPCs. This approach significantly reduces the computational and time burdens associated with performing VPCs and supports the routine inclusion of VPCs as a diagnostic tool during model development process.
1. Nguyen TH et al. CPT Pharmacometrics Syst Pharmacol. 2017 Feb;6(2):87-109. 2. Barriere O, Rich B, Craig J, Mouksassi S (2024). tidyvpc: VPC Percentiles and Prediction Intervals. R package version 1.5.1, https://github.com/certara/tidyvpc. 3. Craig J, Talley M (2024). Certara.VPCResults: Generate Visual Predictive Check (VPC) from Shiny GUI. R package version 3.0.0, https://certara.github.io/R-VPCResults/. 4. https://cran.r-project.org/web/packages/tidyvpc/index.html
Reference: PAGE 33 (2025) Abstr 11663 [www.page-meeting.org/?abstract=11663]
Poster: Methodology - Model Evaluation