Xinnong Li1, Alex Mazur2, Mark Sale2, James Craig2, Keith Nieforth2, Robert Bies1
1University at Buffalo, 2Certara
Introduction: The selection of a “good” model typically involves both objective (numerical) and subjective criteria. Subjective criteria, such as interpretation of plots and consideration of biological plausibility. While both of these can in part be described numerically, there still remain aspects of a “good” model that are difficult to quantify formally. This challenge of combining both objective and subjective criteria in solution selection can be addressed by multi-objective optimization (MOO). MOO allows for the simultaneous optimization of multiple criteria. This approach generates a Pareto front, representing a set of non-dominated models where no single solution can be improved in one objective without sacrificing the performance in another objective [1]. An advantage of MOO over single objective optimization is that no, potentially arbitrary, penalties to the OFV are needed. Objectives: i.Identify a set of non-dominated models by simultaneously optimizing objective function value (OFV) and the number of estimated parameters (n_parms) ii.Compare the search results with and without local downhill search to assess its impact on MOO Methods: This study utilized a 17-DMAG dataset, compromising 66 subjects and 951 observations. The user-defined model search space included 1.57 E6 candidate models, consistent with our previously published study [2]. NSGA-II algorithm [3] recently added to pyDarwin [4] was used for multi-objective model selection. In this case study, the model selection is driven by two objectives, which are OFV (minus two log likelihood), and n_parms. A local downhill search that is defined as the numerical sum of two objectives was also assessed in this study. The optimization process ran for 20 generations, with 80 models in each generation. The search was combined with local downhill search, the local search step was implemented every 10 generations. Models were run in parallel, 32 at a time on a Windows server located at SUNY Buffalo. Results: As the generations progressed, the front that included non-dominated models within each generation shifted toward an improved region, optimizing both OFV and model parsimony. While both objectives improved, the trade-off between them was clearly observed: models with lower OFV generally contained more estimated parameters. In the search without local downhill search, 17 optimal models were identified on the Pareto front, with OFV ranging from 8034.493 to 9813.408 and n_parms ranging from 5 to 22. The globally best model, as determined by exhaustive search [2] using single objective GA, also appeared on the Pareto Front. All of the optimal models passed the covariance step, while 3 of them (n_parms = 13, 18, 21) failed the convergence step. When the local downhill search was implemented, 22 nondominated models were selected within the search space. OFV ranges from 8023.747 to 9813.408, and n_parms ranges from 5 to 26. Similarly, the globally best model was included on the Pareto front. Models identified through the downhill run could perform better, achieving lower OFV with the same level of parsimony. All non-dominated models on the final Pareto front passed the covariance step, while 8 of them failed the convergence step (n_parms = 18, 19, 20, 22, 23, 24, 25, 26), indicating a tendency toward overparameterization. Notably, even though some downhill-searched models have lower OFV, they are more prone to fail the convergence. These models, selected based on objective criteria, are then presented to the user to further examine a manageable set and select one or more from that set based on subjective criteria such as biological plausibility and diagnostic graphics as the “best” model(s). Conclusions: The multi-objective optimization method successfully identified a set of non-dominated models within the search space by evaluating OFV and parsimony at the same time. Both search strategies (with and without local downhill search) successfully found the globally best model, while the incorporation of downhill search expanded the exploration area, yielding more non-dominated models. Therefore, integrating the local downhill search into the multi-objective optimization is recommended to enhance the model selection. Overall, the application of NSGA-II successfully generated a set of non-dominated PopPK models, offering greater insight and flexibility for decision-making in model selection.
[1] M. Kochenderfer and T. Wheeler. Algorithms for optimization. 2019. The MIT Press. [2] Li X, Sale M, Nieforth K, Craig J, Wang F, Solit D, Feng K, Hu M, Bies R, Zhao L. pyDarwin machine learning algorithms application and comparison in nonlinear mixed-effect model selection and optimization. J Pharmacokinet Pharmacodyn. 2024 Dec;51(6):785-796. [3] K. Deb, A. Pratap, S. Agarwal and T. Meyarivan, “A fast and elitist multiobjective genetic algorithm: NSGA-II,” in IEEE Transactions on Evolutionary Computation, vol. 6, no. 2, pp. 182-197, April 2002. [4] https://github.com/certara/pyDarwin
Reference: PAGE 33 (2025) Abstr 11383 [www.page-meeting.org/?abstract=11383]
Poster: Methodology - New Modelling Approaches