Importance of Shrinkage in Empirical Bayes Estimates for Diagnostics and Estimation: Problems and Solutions
Radojka M. Savic and Mats O. Karlsson
Div. of Pharmacokinetics and Drug Therapy, Dept of Pharmaceutical Biosciences, Faculty of Pharmacy, Uppsala University, Sweden
Introduction: Empirical Bayes estimates (EBEs), known also as POSTHOC estimates, provide modelers with individual-specific values, such as individual predictions (IPRED), estimates of interindividual (η) differences and residual error (IWRES) components. These estimates are conditional on individual data and on population parameters. Thus, whenever data becomes sparse or uninformative, the EBE distribution will shrink towards zero (η-shrinkage = 1-SD(ηEBE,P)/ωP, for given parameter P), IPREDs towards the corresponding observations, and IWRES towards zero (ε-shrinkage = 1-SD(IWRES)).(1) However EBEs are widely used in estimation process (FOCE), graphical diagnostics and recently as the points of support for nonparametric (NONP) distribution estimation in NONMEM VI.(1-3) In this work, we investigate how phenomenon of shrinkage affects these three aspects of population analysis, qualitatively and quantitatively. Additionally, solutions to some of these issues are suggested and explored.
Methods: (i) Shrinkage and EBE-based diagnostics This aspect was explored using PK (one compartment, first/zero order absorption, transit compartment model) and PD (Emax, indirect effect) models. Multiple datasets were simulated from these models such that estimated EBE distribution would results in gradually increasing extent of shrinkage for each simulation case. EBEs were estimated in NONMEM based on the true or a misspecified model. Standard errors (SEs) of EBEs were also estimated. Quantitative relationships between η- and ε-shrinkage and informativeness of EBE-based diagnostics were evaluated.
(ii) Shrinkage and Estimation Method Pharmacokinetic data sets (100 sets for each condition) were simulated from a one compartment iv bolus model. Three scenarios with respect to EBE shrinkage magnitude were studied: (a) < 5% (low), (b) < 25% (medium) and (c) > 25% (high.). The extent of shrinkage was calculated after fitting the true model to this data and it was assessed by varying residual error magnitude (higher the residual variability greater the shrinkage) and with choice of sampling times. Each simulated data set was analyzed with the true model using FOCE and FO estimation method. To compare the estimated and the true parameter values, the relative estimation error and the absolute value of the relative bias (ARB) in parameter estimates were evaluated.
(iii) Shrinkage and NONP distribution estimation The same datasets from part (ii) were used to estimate nonparametric distributions. These distributions were evaluated at different percentiles and compared to the true ones using same statistics as described above. QQ plots and marginal cumulative density distribution plots were used as an additional tool to inspect the estimated distributions. Three methods for generating the support points in the presence of shrinkage were explored: (a) posthoc ηs based on the final parametric model (default NONMEM method); (b) posthoc ηs based on the final parametric model and inflated variances (inflation method), and (c) posthoc ηs from the final parametric model enhanced with n (here n=200) additional support points generated by simulation from the final model (simulation method).
Results: (i) Shrinkage and EBE-based diagnostics Already 20-40% of η-shrinkage magnitude was sufficiently high to render EBE-based model diagnostics fundamentally misleading (hidden or falsely induced EBE-EBE correlations, distorted, hidden or falsely induced covariate relationships etc.) IPRED failed to detect structural model misspecification already at the ε-shrinkage magnitude of 20-30%. Similar extent of ε-shrinkage was sufficiently high to diminish the power of IWRES to identify the residual model misspecification. Estimation of EBE standard errors was valuable to determine the informative / uninformative EBEs.
(ii) Shrinkage and Estimation Method With FO, the ARB in estimated parameters ranged from 0-15 % no matter which study design was used to generate datasets. With FOCE and low shrinkage, ARB in parameter estimates was negligible (< 2%); with medium shrinkage ARB increased up to 5% while for the analysis of data with high shrinkage, ARB in parameter estimates approached the ARB value observed with FO.
(iii) Shrinkage and NONP distribution estimation Increased ARB in nonparametric distribution (up to 25%) was observed for studies with both medium and high extent of shrinkage as a consequence of rather sparse and restricted grid of support points used in the default nonparametric estimation. For the case with medium shrinkage, the bias was reduced down to < 5% using a new set of support points (EBEs) computed using the inflation method. However, when shrinkage had become substantial, EBEs would remain located around population mean regardless the variance used for inflation. To resolve this issue, the new "simulation" method was developed that uses a denser grid of support points at which the NONP distribution is to be evaluated. This approach was tested using datasets with low to high shrinkage extent. QQ plots showed an excellent agreement between the true and improved nonparametric distributions which was also confirmed by inspection of cumulative probability density functions.
Conclusions: The shrinkage phenomenon affects all three evaluated aspects of population analysis: EBE-based diagnostics are essentially of no value, parameter estimation bias using the FOCE method becomes similar to the FO method and estimated nonparametric distribution shows increased bias when the default method is used. For diagnostic purposes, it is desirable to report extent of ε- and η-shrinkage to assess the relevance of graphs employing EBEs, IPRED and IWRES. SEs of individual ηs can provide additional information to allow informative diagnostics even in the presence of shrinkage. A new approach to obtain points of support for the nonparametric method has been developed that resulted in good estimation properties even in the presence of high shrinkage.
. Savic RM, Wilkins JJ, Karlsson MO. (Un)informativeness of Empirical Bayes Estimate-Based Diagnostics. In: The AAPS Journal; 2006; Vol. 8, No. S2, Abstract T3360 (2006): American Association of Pharmaceutical Scientists; 2006.
. Sheiner LB, Beal SL. NONMEM Users Guide: NONMEM Project Group, University of California, San Francisco; 1992.
. Savic RM., Kjellsson MC., Karlsson MO. Evaluation of the nonparametric estimation method in NONMEM VI beta. PAGE 15 (2006) Abstr 937 [ http://www.page-meeting.org/?abstract=937 ].