High-Performance Computer “Mistral” at German Climate Computing Center

The climate researchers at the Max Planck Institute for Meteorology (MPI-M) use the high-performance computing and data-storage systems of the German Climate Computing Center (DKRZ) in Hamburg for their climate simulations. This “laboratory” is their most important research infrastructure. Since July 2015 the scientists have worked with the new high-performance computer “Mistral” delivered by the French company Atos/Bull. “Mistral” was officially inaugurated by the First Mayor of the City State of Hamburg, Olaf Scholz, and representatives of the Federal Ministry of Education and Research (BMBF). The ceremony on October 5th, 2015, was accompanied by a two-day user workshop. More computing power and larger data storage systems enable the scientists to perform their model simulations faster and more accurately.

The climate and Earth system models run on high-performance computers with thousands and in the future even tens of thousands of processors, and they generate huge amounts of data. The new High-Performance Computer System for Earth System Research (HLRE-3) “Mistral” at the DKRZ is substantially more powerful and more energy-efficient than its predecessor, an IBM Power6 named “Blizzard”. The computer manufacturer Atos/Bull installed the first phase of the new system in spring 2015. Since July 1st, 2015, “Mistral” has been fully operational, and “Blizzard”, in service since 2009, was switched off by the end of September.

A second phase of “Mistral”, to become operational in summer 2016, will additionally more than double both computing power and storage space. With a computing power of 3 PetaFLOPS (3x1015 FLOPS, FLoating-point Operations Per Second) and a 50-PetaByte parallel file system, it will be possible to conduct and evaluate high-resolution climate simulations at an unprecedented regional scale, to represent more Earth-system processes than before, and to reduce the uncertainties in climate projections. The smaller the grid cells of the three-dimensional model grid, the more detailed simulations are possible. “Mistral” will allow computations on a 100-meter grid, making feasible the simulation of clouds in a regional model for Germany (Opens external link in current windowproject HD(CP)2).

HLRE-3 “Mistral” starts working (Photos: M. Böttinger, DKRZ)

The HLRE-3 was assembled from computer components delivered by Atos/Bull, a disk storage system by Xyratex/Seagate, and high-performance network switches by Mellanox. These components are distributed over 41 racks the size of telephone booths and weighing up to a ton, connected by bundles of fiber cables. About 1,500 compute nodes reach a peak performance of 1.4 PetaFLOPS. This is an increase by a factor of nine compared with “Blizzard”. On the TOP500 list of the fastest computers worldwide, “Mistral” was ranked 56th in July 2015. The processors and memory modules of the system are cooled with water of 35°C temperature. This so-called “warm-water cooling” is innovative and energy-efficient, since it allows “free cooling” throughout the year; there is no need to acquire and operate additional cooling aggregates.

Climate modelers have special requirements for the data-storage systems, because their models produce huge amounts of data. The HLRE-3’s main memory – its short-term memory – is with 120 TeraBytes six times larger than that of “Blizzard”. The actual workspace is the particularly powerful hard-disk memory. The new parallel file system will offer 20 PetaBytes, more than three times larger than the current disk system. This corresponds to hard-disks of 20,000 well-equipped notebooks. High-performance computer systems and the associated infrastructure for data storage are not only operated at the DKRZ; other climate computing centers around the world also offer such a symbiotic infrastructure, optimized for the climate modelers’ needs. But comparing the infrastructure at the DKRZ with other, general-purpose computing centers in Germany, the advantage climate researchers have with the DKRZ becomes very clear: measured relative to the computing system, the data storage system is significantly less powerful elsewhere. Additionally, for the longer-term usability of computed data the DKRZ has one of the largest and most powerful scientific data archive systems in the world. The World Data Center for Climate (WDCC) of the DKRZ collects climate data products resulting from climate simulations, archives them, and makes them available worldwide.

Owing to technological trends, the storage capacity at DKRZ does not grow as fast as the computing power. In the future, investments need to be shifted more clearly towards storage capacity, because the growing computing power will be accompanied by an increase in data production. To offer a balanced system in the future, additional investments in on-line diagnosis, smart analysis procedures, newest compression algorithms, and performance optimizations will help counteract the expected bottleneck. The “Scientific Computing Laboratory” (ScLab) at the MPI-M has the task to support such investments in software infrastructure, in coordination with the DKRZ. The members of this team also cooperate closely with colleagues from the DKRZ in the area of model development and optimization.

The entire HLRE-3 project with a total funding of 41 million Euros comprises diverse modifications and extensions of DKRZ’s infrastructure, including a research cooperation with Atos/Bull aimed at the development of more efficient computing algorithms for climate modeling. About two-thirds of the funding is supplied by the German Federal Ministry of Education and Research (BMBF), the other one-third by the Helmholtz Association. The operating costs of the DKRZ are borne by about 55% by the Max Planck Society, about 27% by the Free Hanseatic State of Hamburg through its University, and about 9% each by the Alfred Wegener Institute (Helmholtz Centre for Polar and Marine Research) and the Helmholtz Centre Geesthacht.

More information and sources:
Opens external link in current windowhttp://www.dkrz.de/?set_language=en

Opens external link in current windowhttp://www.dkrz.de/dkrz-en/about-en/contact/press/news-archive/active-news/hlre3-installation

Press release DKRZ:
Opens external link in current windowhttp://www.dkrz.de/pdfs/presse-und-artikel/Juli2015_ISC15_HLRE3-Inbetriebnahme_ENG.pdf

Prof. Dr. Jochem Marotzke
Max Planck Institute for Meteorology
Phone: +49 (0) 40 41173 311 (Kornelia Müller)
Email: Opens window for sending emailjochem.marotzke(at)mpimet.mpg.de

Reinhard G. Budich
Max Planck Institute for Meteorology
Strategic IT Partnerships
Phone: +49 (0) - 40 41173 369
Email: Opens window for sending emailreinhard.budich(at)mpimet.mpg.de

Michael Böttinger
German Climate Computing Center (DKRZ)
Public Relations
Phone: +49 (0) 40 460094 344
E-Mail: Opens window for sending emailboettinger(at)dkrz.de