The Lenovo solution has a total computing power of 240 TeraFLOPS and triples the computing power of CERFACS.
For research center CERFACS, finding better solutions to scientific problems requires more complex simulations and ultimately, more powerful computing resources. Resolving to triple the power of its internal computing cluster, the organization found the optimal balance between throughput and total cost of ownership in a Lenovo supercomputer with 6,000 cores. Enabling up to 30 percent performance gains and 50 percent energy savings, the solution is helping CERFACS lead the way in research innovation.

A leading research organization in France, CERFACS (Centre Européen de Recherche et de Formation Avancée en Calcul Scientifique) aims to develop advanced methods for numerical simulations and algorithmic solutions for scientific problems of interest for research and industry. The CERFACS shareholders are Airbus Group, CNES, EDF, Météo France, Onera, Safran, and Total.

Endless appetite for resources
CERFACS relies on multiple layers of computational resources in its work: an internal computing cluster and allocated hours of computation on the infrastructure of national partners Météo-France, CEA-CCRT and GENCI, as well as international supercomputing resources provided by PRACE and INCITE. Naturally, there are limitations around the work the organization is able to do using external resources, so the internal cluster is critical to CERFACS’ success.

Nicolas Monnier, CIO at CERFACS, explains: “As the simulations we use increase in complexity, so do our requirements for computing power. On average, our computational requirements increase by a factor of 1.8 every year. Our clients are seeing their demand for resources grow at the same rate—to retain our position as a research leader, we need to periodically update our internal cluster.”

One example where CERFACS urgently required greater computing power was in climate modelling. The organization had developed a model based on a geographical mesh, which divides the earth’s surface into 150 kilometer-square sections.

“Major geographical features that happened to fall entirely inside a grid square in the 150-km mesh—for example, the gap between the Pyrenees and massif central mountain range (seuil of Naurouze)!—were effectively not taking into account during the computations,” says Nicolas Monnier. “To build a better understanding of the factors that influence climate, we needed to move to a much more detailed grid with 50 kilometer-square sections, which dramatically increases the required computational power.”

CERFACS has also invested hundreds of man-years into developing its in-house AVBP code, co-developed with IFPEN for piston engine aspects and used to simulate combustion in turbines and engines. AVBP—which is CPU-bound workload—accounts for 75 percent of the organization’s computing and is used by shareholders and university labs in their own research. As CERFACS delves deeper into coupled simulations that model different components within an engine in parallel, its need for greater internal computing resources is constantly growing.

Choosing the best
Resolving to triple the power of its internal cluster, CERFACS began looking for a new high-performance computing solution that could support greater numbers of more complex simulations, while meeting a number of other selection criteria. The solution needed to offer good value for money, rapid deployment, high reliability and—since climate change is a key area of interest for CERFACS—good energy efficiency.

Nicolas Monnier continues: “We evaluated a number of solutions, asking vendors to commit to a certain levels of performance against benchmarks for our different workloads. The Lenovo solution stood out, demonstrating the optimal ratio of computational throughput to cost.”

CERFACS engaged Lenovo to help deploy the new cluster, based on 252 Lenovo NeXtScale nx360 M5 compute nodes with 6,000 cores connected using InfiniBand fabric and with IBM Spectrum Scale (IBM GPFS) technology providing file-based storage. The solution includes two nodes featuring NVIDIA Quadro K600 graphics cards for post-processing visualization. Equipped with Intel Xeon Haswell processors, the Lenovo solution offers excellent performance. Nicolas Monnier comments: “Another key factor in our choice for a Lenovo solution was the expertise of the Lenovo HPC Technical team. Lenovo helped us to integrate the new solution rapidly and smoothly, and the team was fully committed to making our project a success.”

Solution in action
With significantly more computing power at its fingertips, CERFACS is already seeing the impact on its productivity. The organization has tripled the global capacity of the solution and also the size of the queue of the jobs that will run.

“It is only in the last month or two that we have reached 95 percent utilization of the Lenovo solution and started seeing what it is really capable of doing,” explains Nicolas Monnier. “Even though the new cores run at 2.5GHz compared to the 2.6GHz of our old system, we have seen 15 percent performance gains for our CPU-bound workload, and 30 percent gains for our memory-bound workloads.”

CERFACS has also been impressed by the energy-efficiency of the solutions, as Nicolas Monnier elaborates: “A simulation running on 240 computational cores on the new Lenovo solution, compared to the same number of cores on our original machine, uses half the amount of energy. Since our previous system was only three years old, this is a remarkable improvement.”

Looking to the future
For CERFACS, the relationship with Lenovo is an important one, with the organization recently entering into a partnership with the Lenovo HPC Innovation Center located in Stuttgart.

Nicolas Monnier concludes: “Working with the Lenovo HPC Innovation Center, we hope to investigate how we can adapt our codes to future architectures such as Intel Xeon Phi. We are at the very beginning of our relationship with Lenovo, and we very much look forward to taking advantage of their technical expertise to optimize performance so as to retain our position at the leading edge of computational research.”