Chin Feng Ng, Chee Meng Ng
University of Kentucky at Lexington, USA
Objectives: Two levels of parallelization can be used to accelerate the performance of the parametric expectation-maximization (EM) estimation methods in population quantitative pharmacology (PQP) analysis [1]. First, conditional mean (µ) and variance (B) of individual subject in the computational intensive E-step is determined independently using different computing nodes (Level 1) [2]. All parametric EM methods in the commercial PQP software (NONMEM and Phoenix) used the Level 1 parallelization (L1P). The limitations of the L1P become apparent for complex PQP analysis as solving a large system of ordinary differential equations (ODEs) with thousands of simulated random model parameters sets (ISAMPLE) for µ and B is extremely computational and time intensive. Even for simple PQP models, computing E-step with large number of ISAMPLE to achieve consistent objective function for model selection can become rate-limiting step of the EM algorithm [3]. Level 2 parallelization (L2P) can alleviate these problems by further parallelizing the L1P task within the same subject but has never been implemented and tested in parametric EM methods. In this study, we developed the first novel MEPM with dual parallelization levels to optimize the parallel efficiency of parametric EM methods in PQP analysis.
Methods: A MPEM method based on Monte-Carlo Parametric EM algorithm facilitated by Maximum a Posterior (MCPEM-MAP) was developed and written in C programming language. Two levels of parallelization were implemented as follows:
Level 1 – The determination of µ and B for all subjects in the E-step was divided and computed independently as parallel computing tasks (L1P).
Level 2 – A large ISAMPLE was needed to compute the likelihood for µ and B determination in each subject. Therefore, under each L1P task, further parallelization was applied to compute the likelihood of each ISAMPLE within the same subject using parallel computing threads (L2P).
The performances of the MPEM were assessed in the UK HPC cluster with 256 basic compute nodes with each node contained dual Intel E5-2670 2.6 GHz 8-core processors. Runtimes of the MEPM with a single CPU core (M-S), L1P (M-L1), and L1P/L2P (M-L12) were recorded and compared. Runtime of the MCPEM-MAP in NONMEM 7.3 with a single CPU core (N-S) was determine and used as an external reference for comparison. A one-compartment linear PK model was used to simulate PK of 100 subjects with intensive sampling design for the analysis.
Results: We first assessed the performances using a PK data of 10 subjects because UK HPC cluster policy only allowed up to 10 compute nodes per single job. ISAMPLE of 100,000 and 100 EM iterations were used to achieve small OBJ variation at steady-state and model convergence [3]. In M-L1, the µ and B of 10 subjects was computed separately in 10 compute nodes (10 L1P) and this represented the maximum parallel gain of the L1P task. In M-L12, ISAMPLE within the same subject assigned to each compute node was further distributed to 16 processor cores (16 L2P) within the compute node for µ and B computation. The runtimes were 41.8, 8.6, 1.28 and 0.800 min for N-S, M-S, M-L1 and M-L12 respectively. M-L12 computed 38% faster than the M-L1.
We then expanded the analysis to include the PK data with 100 subjects in exploring the performances of MPEM with various L1P/L2P combinations. The runtimes were 418 and 87.7 min for N-S and M-S, respectively. The runtimes were 58.3, 29.2 and 12.1 min for M-L1 with numbers of L1P task of 2, 4, and 10, respectively. The runtimes were 34.9, 17.3, and 6.97 min for M-L12 with L1P/L2P combinations of 2/16, 4/16, and 10/16, respectively. The M-L12 consistently performed faster than the M-L1 regardless of the numbers of L1P task. Similar results were observed for ISAMPLE=50,000.
Conclusions: The implementation of extra parallelization level L2P is able to overcome the computational bottleneck of L1P in accelerating the performance of parametric EM in PQP analysis. To our best knowledge, MPEM is the first reported parametric EM method with dual levels of parallelization in population data analysis. Study is ongoing to assess the performance of this novel MPEM in complex PQP analysis with large system of ODEs and complicated dataset. In addition, further parallelization with GPUs is being implemented to optimize the L2P process which is expected to further
References:
[1] Ng CM. AAPSJ 2013;15(4):1212-21
[2] Bauer J, Serge G, Ng CM. AAPSJ 2007;9(1):E60-83
[3] Zou Y, Ng CM. ACOP 2017 Abstract T-048
Reference: PAGE 27 (2018) Abstr 8570 [www.page-meeting.org/?abstract=8570]
Poster: Methodology - Estimation Methods