Parallel benchmarks 8a & 8b¶
The following examples show parallel reading and writing of domain-decomposed data with MPI.
The Message Passing Interface (MPI) is an open communication standard for scientific computing. MPI is used on clusters, e.g. large-scale supercomputers, to communicate between nodes and provides parallel I/O primitives.
This benchmark writes a few meshes and particles, either 1D, 2D or 3D.
The meshes are viewed as grid of mini blocks. As an example, we assume the mini blocks dimension are [16, 32, 32].
Next we define the grid based on the mini block. say, [32, 32, 16]. Then our actual mesh size is [16x32, 32x32, 32x16].
Here is a sample input file (“w.input”):
dim=3 balanced=true ratio=1 steps=10 minBlock=16 32 32 grid=32 32 16
With the above input file, we will create an openPMD file with the above mesh using
particle to mesh ratio = 1
10 iteration steps
Note: All files generated are group based. i.e. One file per iteration.
then the file generated are: ../samples/8a_parallel_3Db_*
This benchmark is to read from the files written by 8a.
The options are: a file prefix, and a read pattern
For example, if the files are in the format of
the input will be:
The Read options intent to measure overall processing time in the following categories:
Metadata only (option = m)
or data retrieval (after metadata loaded)
The data retrieval is furthur divided into:
- slice the “rho” mesh
sx/sy/szdepends on which direction. e.g.
- slice on the 3D magnetic field(e.g. find values for “Bx”, “By” and “Bz”)
fx/fy/fzdepends on which direction. e.g.
So here are the options one can use to read a file:
For example, To read files generated by the above write commmand, metadata only:
./8b_benchmark_read_parallel ../samples/8a_parallel_3Db m
More complicated Writing options (Applies to ADIOS BP)¶
The ADIOS BP files uses subfiles to store data from each rank. We have an option to provide hint on how data should be divided per rank in the command line: the order of options are:
grid of minimal blocks|balance|particle2mesh ratio
use multiple blocks
num of timesteps,
hint on work load arrangement.
Example: “mpirun -n 4 ./8a_benchmark_write_parallel 400801 16016 1 5 3 4004002 “
Here 4 ranks are used to write a 3D mesh, minimal block is [16,16,16], grid of minimal block is [8,4,4], so the actual mesh = [16x8, 16x4, 16x4]. Number of timestep = 5.
The hint is asking each rank to work on a [16x2, 16x4, 16x4] block. It precisely cover the mesh with 4 ranks, so will be applied.