Two new user-friendly approaches to assess pharmacometric model identifiability
Martijn van Noort, Joost de Jongh
LAP&P Consultants, The Netherlands
Objectives:
In pharmacometric modelling, it is often important to know whether the data is sufficiently rich to identify the parameters of a proposed model. While it may be possible to assess this based on the results of a model fit, it may be difficult to disentangle identifiability issues from other model fitting and numerical problems. Furthermore, it can be of value to ascertain identifiability beforehand from study design. Several methods are available to assess parameter identifiability, such as DAISY [1], Aliasing [2] or the DESIGN option in NONMEM [3]. However, they are sometimes limited in scope (both regarding the models to which they can be applied and the issues that can be identified) and may make unrealistic assumptions. Also, they may be difficult to use, and implementations are not always available.
The aim of this work was to develop two new methods for identifiability analysis prior to model optimization that address these drawbacks. Their use is illustrated with an example problem.
Methods:
The parameters of a model are identifiable if two different parameter vectors always lead to two different model outputs. In practice one often focuses on local unidentifiability, characterized by a curve in parameter space of constant model output. The tangent to the curve is the unidentifiable direction. If a model is locally unidentifiable then its parameters cannot be determined uniquely from the data.
Two methods for local identifiability were developed.
Method 1, using the Sensitivity Matrix (SM)
The SM is the matrix of derivatives of the model outputs with respect to the structural parameters. Unidentifiable directions in parameter space correspond to vectors in the null space of this matrix. As the null space is difficult to determine numerically, several proxy indicators were developed that identify near-singularities of the SM and the corresponding parameter vectors. They include the Skewing Angle, which measures the angle between the images in the output space of the parameter vectors; the Minimal Parameter Relations, listing the parameter directions closest to singularity; and the Least Identifiable Parameters, showing which parameters are closest to linear dependence with the others.
Method 2, using the Fisher Information Matrix (FIM)
The FIM determines the approximate shape of the objective function value (OFV) with respect to the parameters. Unidentifiable directions in parameter space correspond to vectors in the null space of this matrix. As for the SM method, proxy indicators are determined. They include the curvatures of the OFV surface, the maximum change in parameter values at a given OFV change, and relative standard errors (RSE). Unlike the SM method, this method can handle random effect parameters. It assumes Gaussian distributions.
Application: the methods were applied to a quasi-equilibrium (QE) approximation to a TMMD model describing leukemia inhibitory factor data in sheep, with parameter values, sample times and dose levels obtained from Abraham et al. [4]. The output consisted of the PK concentration at the reported sample times.
Results:
The methods were implemented in R [5] and validated on standard examples (data on file). To reduce numerical approximation errors, the implementation optionally derives the variational equations symbolically.
The methods were applied to The QE model at three individual dose levels and to those three combined. Results from each of the three individual dose levels showed identifiability issues with both methods. Even at the highest dose level, where the model traversed all phases of the TMDD profile, there were two unidentifiable directions, involving the nonlinear binding parameters. All parameters became identifiable in a scenario where the dose levels were combined.
Conclusions:
Two new methods were developed for identifiability analysis and were applied to a TMDD example. They can be applied to any smooth model described by ordinary differential equations. They can detect any local identifiability issues and find the corresponding directions in parameter space. The FIM method can handle random effect parameters (IIV and residual errors) and the effects of population size, while the SM method is limited to structural models and independent of size. Both methods are easy to use and will be publicly available. They require as input a definition of the model and dosing, its parameters, and sample times. Actual observations are not required.
References:
[1] Bellu, Saccomani, Audoly, d'Angio. DAISY. A new software tool to test global identifiability of biological and physiological systems. Comput Methods Programs Biomed. 2007 October ; 88(1): 52–61.
[2] Augustin, Braakman, Paxson. A Workflow To Detect Non-Identifiability In Parameter Estimation Using SimBiology.
https://github.com/mathworks-SimBiology/AliasingScoreApp/blob/master/AliasingScore_Poster.pdf
[3] Bauer, Hooker, Mentré. Tutorial for $DESIGN in NONMEM: Clinical trial evaluation and optimization. CPT-PSP 10, 2021, 1452-1465.
[4] Abraham, Krzyzanski, Mager. Partial Derivative-Based Sensitivity Analysis of Models Describing Target-Mediated Drug Disposition. AAPS 9(2), 2007.
[5] R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2020. https://www.R-project.org/
