openPMD supports writing to and reading from HDF5
For this, the installed copy of openPMD must have been built with support for the HDF5 backend.
To build openPMD with support for HDF5, use the CMake option
For further information, check out the installation guide,
build dependencies and the build options.
HDF5 internally either writes serially, via
POSIX on Unix systems, or parallel to a single logical file via MPI-I/O.
The following environment variables control HDF5 I/O behavior at runtime.
Sets the MPI-parallel transfer mode to collective (
Tuning parameter for parallel I/O, choose an alignment which is a multiple of the disk block size.
OPENPMD_HDF5_INDEPENDENT: by default, we implement MPI-parallel data
storeChunk (write) and
loadChunk (read) calls as none-collective MPI operations.
Attribute writes are always collective in parallel HDF5.
Although we choose the default to be non-collective (independent) for ease of use, be advised that performance penalties may occur, although this depends heavily on the use-case.
For independent parallel I/O, potentially prefer using a modern version of the MPICH implementation (especially, use ROMIO instead of OpenMPI’s ompio implementation).
Please refer to the HDF5 manual, function H5Pset_dxpl_mpio for more details.
OPENPMD_HDF5_ALIGNMENT This sets the alignment in Bytes for writes via the
According to the HDF5 documentation:
For MPI IO and other parallel systems, choose an alignment which is a multiple of the disk block size.
On Lustre filesystems, according to the NERSC documentation, it is advised to set this to the Lustre stripe size. In addition, ORNL Summit GPFS users are recommended to set the alignment value to 16777216(16MB).
H5_COLL_API_SANITY_CHECK: this is a HDF5 control option for debugging parallel I/O logic (API calls).
Debugging a parallel program with that option enabled can help to spot bugs such as collective MPI-calls that are not called by all participating MPI ranks.
Do not use in production, this will slow parallel I/O operations down.
GitHub issue #554
Axel Huebl, Rene Widera, Felix Schmitt, Alexander Matthes, Norbert Podhorszki, Jong Youl Choi, Scott Klasky, and Michael Bussmann. On the Scalability of Data Reduction Techniques in Current and Upcoming HPC Systems from an Application Perspective, ISC High Performance 2017: High Performance Computing, pp. 15-29, 2017. arXiv:1706.00522, DOI:10.1007/978-3-319-67630-2_2