Component concurrency increases the parallel efficiency of Earth system models

In a recent study in the journal of Geoscientific Model Development Leonidas Linardakis and his colleagues demonstrate how coarse-grained component concurrency increases the parallel workload of Earth system models and results in an increased number of compute nodes that can be used efficiently in parallel. The additional dimension of parallelism allows the scientists to extend scalability beyond the limits set by established parallelization techniques. It also offers a way to maintain performance (by using more compute power) when the model complexity is increased by adding components, such as biogeochemistry or ice sheet models. Furthermore, concurrency allows each component to run on different hardware, thus leveraging the usage of heterogeneous hardware configurations.

Our current model parallelization techniques, such as domain decomposition and loop-level parallelization, decompose the data-space, typically by decomposing the 2-dimensional grid. This limits the availability for parallelism to the availability of enough grid points. An exception to this approach is when running the coupled model, where the atmosphere and the ocean models run in parallel. This is an example of coarse-grained concurrency, where large parts of the model run in parallel throughout a time-step, and are coupled every one or more time-steps.

Component concurrency decomposes the algorithmic space, providing an additional parallelism dimension to the data-decomposition techniques. This increases the parallel workload, that is the amount of operations available within any parallel region. In turn, it allows us to use more computational resources efficiently. Moreover, data-parallel approaches do not help when introducing more processes and details of the Earth system model, such as the cryosphere, atmospheric chemistry, and particle microphysics, while concurrency can absorb part of the additional computational cost. An application to the HAMOCC ocean biogeochemistry model, running concurrently to the ICON-O ocean model, shows that we can double the parallel efficiency on the high-performance computer “Levante” at the German Climate Computing Center (DKRZ).

Concurrency becomes even more important when moving to massively parallel machines. These machines require several millions of 3D grid points per node in order to run efficiently. This limits the number of nodes we can use efficiently through data-parallel approaches, and in return, it limits the simulation time compression. In the context of the project IFCES2 (Intra-model Functional Concurrency towards efficient Exascale Earth System prediction), one task aims to port the HAMOCC model to GPUs, allowing it to run concurrently in hybrid mode to the ICON-O model. We expect that the benefits from concurrency will be even more pronounced in this setup. 

Original publication

L. Linardakis, I. Stemmler, M. Hanke, L. Ramme, F. Chegini, T. Ilyina, and P. Korn (2022) Improving scalability of Earth system models through coarse-grained component concurrency – a case study with the ICON v2.6.5 modelling system. Geosci. Model Dev., 15, 9157–9176, doi.org/10.5194/gmd-15-9157-2022

Contact:

Dr. Leonidas Linardakis
Max Planck Institute for Meteorology
Email: leonidas.linardakis@mpimet.mpg.de