code_saturne is based on a massively parallel architecture. It uses the Message Passing Interface (MPI) paradigm as its primary level of parallelism and can optionally use shared-memory parallelism through OpenMP.

HPC platforms

code_saturne and neptune_cfd are used extensively on HPC systems at various sites:

• EDF clusters
  • Intel-based architectures.
• PRACE machines
  • ARCHER2 at EPCC;
  • Jean Zay at IDRIS.
• DOE machines, through INCITE access
  • Summit at ORNL.

In the past, both codes have also been used on IBM Blue Gene/L, Blue Gene/P, and Blue Gene/Q architectures at:

• EDF;
• STFC Daresbury;
• Argonne National Laboratory (ANL).

Tests conducted by STFC Daresbury and IMFT have reached several billion cells. These tests have led to extensive parallel optimization and debugging work in collaboration with EDF.

Work is in progress to support GPU execution, although this capability is not yet used in production.

Typical production scale

Typical production studies involve between 10 and 50 million cells, with some studies ranging from 200 to 400 million cells.

Notable large-scale runs include:

• the first run exceeding one billion cells, conducted by STFC and EDF Energy UK in 2013 using four Blue Gene/Q racks;
• several production runs exceeding one billion cells and using more than 10,000 MPI ranks on EDF clusters.

European application benchmarks

code_saturne is one of the 12 codes selected for the PRACE and DEISA Unified European Application Benchmark Suite (UEABS).

It is one of the two CFD codes included in this benchmark suite, alongside ALYA.

Parallel domain partitioning

Parallelism is based mainly on classical domain partitioning. The available partitioning methods include:

• ParMETIS;
• PT-SCOTCH;
• an internal partitioner based on a Morton space-filling curve.

These partitioning methods can be combined with any current MPI library.

Input and output operations are independent of the domain partitioning.

Ghost-cell management

Parallelism and periodicity are managed using a classical ghost-cell method.

Most operations require only ghost cells associated with cells sharing faces. Extended neighborhoods, which are used for gradient calculations, also require ghost cells associated with cells sharing vertices.