openPMD-api
|
Implementation for the root level of the openPMD hierarchy. More...
#include <Series.hpp>
Classes | |
struct | ParsedInput |
Public Types | |
enum class | IterationOpened : bool { HasBeenOpened , RemainsClosed } |
using | IterationIndex_t = Iteration::IterationIndex_t |
An unsigned integer type, used to identify Iterations in a Series. | |
using | IterationsContainer_t = internal::SeriesData::IterationsContainer_t |
Type for a container of Iterations indexed by IterationIndex_t. | |
using | iterations_t = decltype(internal::SeriesData::iterations) |
using | iterations_iterator = iterations_t::iterator |
using | Data_t = internal::SeriesData |
Public Types inherited from openPMD::Attributable | |
enum | ReadMode { IgnoreExisting , OverrideExisting , FullyReread } |
Public Member Functions | |
Series (std::string const &filepath, Access at, MPI_Comm comm, std::string const &options="{}") | |
Construct a new Series. More... | |
Series (std::string const &filepath, Access at, std::string const &options="{}") | |
Construct a new Series. More... | |
Series (Series const &)=default | |
Series (Series &&)=default | |
Series & | operator= (Series const &)=default |
Series & | operator= (Series &&)=default |
operator bool () const | |
Is this a usable Series object? More... | |
std::string | openPMD () const |
Series & | setOpenPMD (std::string const &openPMD) |
Set the version of the enforced openPMD standard. More... | |
uint32_t | openPMDextension () const |
Series & | setOpenPMDextension (uint32_t openPMDextension) |
Set a 32-bit mask of applied extensions to the openPMD standard. More... | |
std::string | basePath () const |
Series & | setBasePath (std::string const &basePath) |
Set the common prefix for all data sets and sub-groups of a specific iteration. More... | |
std::string | meshesPath () const |
Series & | setMeshesPath (std::string const &meshesPath) |
Set the path to mesh records, relative(!) to basePath . More... | |
std::string | particlesPath () const |
Series & | setParticlesPath (std::string const &particlesPath) |
Set the path to groups for each particle species, relative(!) to basePath . More... | |
std::string | author () const |
Series & | setAuthor (std::string const &author) |
Indicate the author and contact for the information in the file. More... | |
std::string | software () const |
Series & | setSoftware (std::string const &newName, std::string const &newVersion=std::string("unspecified")) |
Indicate the software/code/simulation that created the file. More... | |
std::string | softwareVersion () const |
Series & | setSoftwareVersion (std::string const &softwareVersion) |
Indicate the version of the software/code/simulation that created the file. More... | |
std::string | date () const |
Series & | setDate (std::string const &date) |
Indicate the date of creation. More... | |
std::string | softwareDependencies () const |
Series & | setSoftwareDependencies (std::string const &newSoftwareDependencies) |
Indicate dependencies of software that were used to create the file. More... | |
std::string | machine () const |
Series & | setMachine (std::string const &newMachine) |
Indicate the machine or relevant hardware that created the file. More... | |
IterationEncoding | iterationEncoding () const |
Series & | setIterationEncoding (IterationEncoding iterationEncoding) |
Set the encoding style for multiple iterations in this series. More... | |
std::string | iterationFormat () const |
Series & | setIterationFormat (std::string const &iterationFormat) |
Set a pattern describing how to access single iterations in the raw file. More... | |
std::string | name () const |
Series & | setName (std::string const &name) |
Set the pattern for file names. More... | |
std::string | backend () const |
The currently used backend. More... | |
std::string | backend () |
void | flush (std::string backendConfig="{}") |
Execute all required remaining IO operations to write or read data. More... | |
ReadIterations | readIterations () |
Entry point to the reading end of the streaming API. More... | |
void | parseBase () |
Parse the Series. More... | |
WriteIterations | writeIterations () |
Entry point to the writing end of the streaming API. More... | |
void | close () |
Close the Series and release the data storage/transport backends. More... | |
Data_t & | get () |
Data_t const & | get () const |
void | setData (std::shared_ptr< internal::SeriesData > series) |
std::unique_ptr< ParsedInput > | parseInput (std::string) |
template<typename TracingJSON > | |
void | parseJsonOptions (TracingJSON &options, ParsedInput &) |
Parse non-backend-specific configuration in JSON config. More... | |
bool | hasExpansionPattern (std::string filenameWithExtension) |
bool | reparseExpansionPattern (std::string filenameWithExtension) |
template<typename... MPI_Communicator> | |
void | init (std::string const &filepath, Access at, std::string const &options, MPI_Communicator &&...) |
template<typename TracingJSON , typename... MPI_Communicator> | |
std::tuple< std::unique_ptr< ParsedInput >, TracingJSON > | initIOHandler (std::string const &filepath, std::string const &options, Access at, bool resolve_generic_extension, MPI_Communicator &&...) |
void | initSeries (std::unique_ptr< AbstractIOHandler >, std::unique_ptr< ParsedInput >) |
void | initDefaults (IterationEncoding, bool initAll=false) |
std::future< void > | flush_impl (iterations_iterator begin, iterations_iterator end, internal::FlushParams const &flushParams, bool flushIOHandler=true) |
Internal call for flushing a Series. More... | |
void | flushFileBased (iterations_iterator begin, iterations_iterator end, internal::FlushParams const &flushParams, bool flushIOHandler=true) |
void | flushGorVBased (iterations_iterator begin, iterations_iterator end, internal::FlushParams const &flushParams, bool flushIOHandler=true) |
void | flushMeshesPath () |
void | flushParticlesPath () |
void | readFileBased () |
void | readOneIterationFileBased (std::string const &filePath) |
std::optional< std::deque< IterationIndex_t > > | readGorVBased (bool do_always_throw_errors, bool init, std::set< IterationIndex_t > const &ignoreIterations={}) |
Note on re-parsing of a Series: If init == false, the parsing process will seek for new Iterations/Records/Record Components etc. More... | |
void | readBase () |
std::string | iterationFilename (IterationIndex_t i) |
IterationOpened | openIterationIfDirty (IterationIndex_t index, Iteration iteration) |
void | openIteration (IterationIndex_t index, Iteration iteration) |
iterations_iterator | indexOf (Iteration const &) |
Find the given iteration in Series::iterations and return an iterator into Series::iterations at that place. | |
AdvanceStatus | advance (AdvanceMode mode, internal::AttributableData &file, iterations_iterator it, Iteration &iteration) |
In step-based IO mode, begin or end an IO step for the given iteration. More... | |
AdvanceStatus | advance (AdvanceMode mode) |
void | flushStep (bool doFlush) |
Called at the end of an IO step to store the iterations defined in the IO step to the snapshot attribute and to store that at least one step was written. More... | |
std::optional< std::vector< IterationIndex_t > > | currentSnapshot () const |
AbstractIOHandler * | runDeferredInitialization () |
AbstractIOHandler * | IOHandler () |
AbstractIOHandler const * | IOHandler () const |
template<typename... MPI_Communicator> | |
void | init (std::string const &filepath, Access at, std::string const &options, MPI_Communicator &&...comm) |
template<typename TracingJSON , typename... MPI_Communicator> | |
auto | initIOHandler (std::string const &filepath, std::string const &options, Access at, bool resolve_generic_extension, MPI_Communicator &&...comm) -> std::tuple< std::unique_ptr< ParsedInput >, TracingJSON > |
Public Member Functions inherited from openPMD::Attributable | |
Attributable (NoInit) | |
Attribute | getAttribute (std::string const &key) const |
Retrieve value of Attribute stored with provided key. More... | |
bool | deleteAttribute (std::string const &key) |
Remove Attribute of provided value both logically and physically. More... | |
std::vector< std::string > | attributes () const |
List all currently stored Attributes' keys. More... | |
size_t | numAttributes () const |
Count all currently stored Attributes. More... | |
bool | containsAttribute (std::string const &key) const |
Check whether am Attribute with a given key exists. More... | |
std::string | comment () const |
Retrieve a user-supplied comment associated with the object. More... | |
Attributable & | setComment (std::string const &comment) |
Populate Attribute corresponding to a comment with the user-supplied comment. More... | |
void | seriesFlush (std::string backendConfig="{}") |
Flush the corresponding Series object. More... | |
MyPath | myPath () const |
The path to this object within its containing Series. More... | |
protected Series | retrieveSeries () const |
void | seriesFlush (internal::FlushParams const &) |
void | flushAttributes (internal::FlushParams const &) |
void | readAttributes (ReadMode) |
template<typename T > | |
T | readFloatingpoint (std::string const &key) const |
Retrieve the value of a floating point Attribute of user-defined precision with ensured type-safety. More... | |
template<typename T > | |
std::vector< T > | readVectorFloatingpoint (std::string const &key) const |
Retrieve a vector of values of a floating point Attributes of user-defined precision with ensured type-safety. More... | |
AbstractIOHandler * | IOHandler () |
AbstractIOHandler const * | IOHandler () const |
Writable *& | parent () |
Writable const * | parent () const |
Writable & | writable () |
Writable const & | writable () const |
void | setData (std::shared_ptr< internal::AttributableData > attri) |
internal::AttributableData & | get () |
internal::AttributableData const & | get () const |
bool | dirty () const |
bool | dirtyRecursive () const |
O(1). | |
void | setDirty (bool dirty_in) |
void | setDirtyRecursive (bool dirty_in) |
bool | written () const |
bool & | written () |
template<typename T > | |
bool | setAttribute (std::string const &key, T value) |
Populate Attribute of provided name with provided value. More... | |
bool | setAttribute (std::string const &key, char const value[]) |
Iteration const & | containingIteration () const |
Returns the corresponding Iteration. More... | |
Iteration & | containingIteration () |
Public Attributes | |
IterationsContainer_t | iterations |
std::shared_ptr< Data_t > | m_series = nullptr |
Static Public Attributes | |
static constexpr private char const *const | BASEPATH = "/data/%T/" |
Friends | |
class | Attributable |
class | Iteration |
class | Writable |
class | ReadIterations |
class | SeriesIterator |
class | internal::SeriesData |
class | WriteIterations |
Additional Inherited Members | |
Protected Types inherited from openPMD::Attributable | |
using | Data_t = internal::AttributableData |
Protected Attributes inherited from openPMD::Attributable | |
std::shared_ptr< Data_t > | m_attri |
Implementation for the root level of the openPMD hierarchy.
Entry point and common link between all iterations of particle and mesh data.
openPMD::Series::Series | ( | std::string const & | filepath, |
Access | at, | ||
MPI_Comm | comm, | ||
std::string const & | options = "{}" |
||
) |
Construct a new Series.
For further details, refer to the documentation of the non-MPI overload.
filepath | The file path. |
at | Access mode. |
comm | The MPI communicator. |
options | Advanced backend configuration via JSON. May be specified as a JSON-formatted string directly, or as a path to a JSON textfile, prepended by an at sign '@'. |
openPMD::Series::Series | ( | std::string const & | filepath, |
Access | at, | ||
std::string const & | options = "{}" |
||
) |
Construct a new Series.
For details on access modes, JSON/TOML configuration and iteration encoding, refer to:
In case of file-based iteration encoding, the file names for each iteration are determined by an expansion pattern that must be specified. It takes one out of two possible forms:
simData_T.bp
becomes simData_50.bp
.simData_%06T.bp
becomes simData_000050.bp
.The backend is determined:
backend
, e.g. {"backend": "adios2"}
.simData_T.h5
.The filename extension can be replaced with a globbing pattern E. It will be replaced with an automatically determined file name extension:
filepath | The file path. |
at | Access mode. |
options | Advanced backend configuration via JSON. May be specified as a JSON/TOML-formatted string directly, or as a path to a JSON/TOML textfile, prepended by an at sign '@'. |
AdvanceStatus openPMD::Series::advance | ( | AdvanceMode | mode, |
internal::AttributableData & | file, | ||
iterations_iterator | it, | ||
Iteration & | iteration | ||
) |
In step-based IO mode, begin or end an IO step for the given iteration.
Called internally by Iteration::beginStep and Iteration::endStep.
mode | Whether to begin or end a step. |
file | The Attributable representing the iteration. In file-based iteration layout, this is an Iteration object, in group- based layout, it's the Series object. |
it | The iterator within Series::iterations pointing to that iteration. |
iteration | The actual Iteration object. |
std::string openPMD::Series::author | ( | ) | const |
no_such_attribute_error | If optional attribute is not present. |
std::string openPMD::Series::backend | ( | ) | const |
The currently used backend.
std::string openPMD::Series::basePath | ( | ) | const |
void openPMD::Series::close | ( | ) |
Close the Series and release the data storage/transport backends.
This is an explicit API call for what the Series::~Series() destructor would do otherwise. All backends are closed after calling this method. The Series should be treated as destroyed after calling this method. The Series will be evaluated as false in boolean contexts after calling this method.
std::string openPMD::Series::date | ( | ) | const |
no_such_attribute_error | If optional attribute is not present. |
void openPMD::Series::flush | ( | std::string | backendConfig = "{}" | ) |
Execute all required remaining IO operations to write or read data.
backendConfig | Further backend-specific instructions on how to implement this flush call. Must be provided in-line, configuration is not read from files. |
std::future< void > openPMD::Series::flush_impl | ( | iterations_iterator | begin, |
iterations_iterator | end, | ||
internal::FlushParams const & | flushParams, | ||
bool | flushIOHandler = true |
||
) |
Internal call for flushing a Series.
Any flushing of the Series will pass through this call.
begin | Start of the range of iterations to flush. |
end | End of the range of iterations to flush. |
flushParams | Flush params, as documented in AbstractIOHandler.hpp. |
flushIOHandler | Tasks will always be enqueued to the backend. If this flag is true, tasks will be flushed to the backend. |
void openPMD::Series::flushStep | ( | bool | doFlush | ) |
Called at the end of an IO step to store the iterations defined in the IO step to the snapshot attribute and to store that at least one step was written.
doFlush | If true, flush the IO handler. |
IterationEncoding openPMD::Series::iterationEncoding | ( | ) | const |
std::string openPMD::Series::iterationFormat | ( | ) | const |
std::string openPMD::Series::machine | ( | ) | const |
no_such_attribute_error | If optional attribute is not present. |
std::string openPMD::Series::meshesPath | ( | ) | const |
no_such_attribute_error | If optional attribute is not present. |
basePath
. std::string openPMD::Series::name | ( | ) | const |
std::string openPMD::Series::openPMD | ( | ) | const |
uint32_t openPMD::Series::openPMDextension | ( | ) | const |
openPMD::Series::operator bool | ( | ) | const |
void openPMD::Series::parseBase | ( | ) |
Parse the Series.
Only necessary in linear read mode. In linear read mode, the Series constructor does not do any IO accesses. This call effectively triggers the side effects of Series::readIterations(), for use cases where data needs to be accessed before iterating through the iterations.
The reason for introducing this restricted alias to Series::readIterations() is that the name "readIterations" is misleading for that use case: When using IO steps, this call only ensures that the first step is parsed.
void openPMD::Series::parseJsonOptions | ( | TracingJSON & | options, |
ParsedInput & | input | ||
) |
Parse non-backend-specific configuration in JSON config.
Currently this parses the keys defer_iteration_parsing, backend and iteration_encoding.
TracingJSON | template parameter so we don't have to include the JSON lib here |
std::string openPMD::Series::particlesPath | ( | ) | const |
no_such_attribute_error | If optional attribute is not present. |
basePath
. auto openPMD::Series::readGorVBased | ( | bool | do_always_throw_errors, |
bool | init, | ||
std::set< IterationIndex_t > const & | ignoreIterations = {} |
||
) |
Note on re-parsing of a Series: If init == false, the parsing process will seek for new Iterations/Records/Record Components etc.
If series.iterations contains the attribute snapshot
, returns its value. If do_always_throw_errors is false, this method will try to handle errors and turn them into a warning (useful when parsing a Series, since parsing should succeed without issue). If true, the error will always be re-thrown (useful when using ReadIterations since those methods should be aware when the current step is broken).
ReadIterations openPMD::Series::readIterations | ( | ) |
Entry point to the reading end of the streaming API.
Creates and returns an instance of the ReadIterations class which can be used for iterating over the openPMD iterations in a C++11-style for loop. Series::readIterations()
is an intentionally restricted API that ensures a workflow which also works in streaming setups, e.g. an iteration cannot be opened again once it has been closed. For a less restrictive API in non-streaming situations, Series::iterations
can be accessed directly. Look for the ReadIterations class for further documentation.
Series & openPMD::Series::setAuthor | ( | std::string const & | author | ) |
Indicate the author and contact for the information in the file.
author | String indicating author and contact for the information in the file. |
Series & openPMD::Series::setBasePath | ( | std::string const & | basePath | ) |
Set the common prefix for all data sets and sub-groups of a specific iteration.
basePath | String of the common prefix for all data sets and sub-groups of a specific iteration. |
Series & openPMD::Series::setDate | ( | std::string const & | date | ) |
Indicate the date of creation.
date | String indicating the date of creation. |
Series & openPMD::Series::setIterationEncoding | ( | IterationEncoding | iterationEncoding | ) |
Set the encoding style for multiple iterations in this series.
A preview on the openPMD 2.0 variable-based iteration encoding can be activated with this call. Making full use of the variable-based iteration encoding requires (1) explicit support by the backend (available only in ADIOS2) and (2) use of the openPMD streaming API. In other backends and without the streaming API, only one iteration/snapshot may be written in the variable-based encoding, making this encoding a good choice for single-snapshot data dumps.
iterationEncoding | Desired encoding style for multiple iterations in this series. |
Series & openPMD::Series::setIterationFormat | ( | std::string const & | iterationFormat | ) |
Set a pattern describing how to access single iterations in the raw file.
iterationFormat | String with the iteration regex %T defining either the series of files (fileBased) or the series of groups within a single file (groupBased) that allows to extract the iteration from it. For fileBased formats the iteration must be included in the file name. The format depends on the selected iterationEncoding method. |
Series & openPMD::Series::setMachine | ( | std::string const & | newMachine | ) |
Indicate the machine or relevant hardware that created the file.
newMachine | String indicating the machine or relevant hardware that created the file (semicolon-separated list if needed).. |
Series & openPMD::Series::setMeshesPath | ( | std::string const & | meshesPath | ) |
Set the path to mesh records, relative(!) to basePath
.
meshesPath | String of the path to mesh records, relative(!) to basePath . |
Series & openPMD::Series::setName | ( | std::string const & | name | ) |
Set the pattern for file names.
name | String of the pattern for file names. Must include iteration regex %T for fileBased data. |
Series & openPMD::Series::setOpenPMD | ( | std::string const & | openPMD | ) |
Set the version of the enforced openPMD standard.
Series & openPMD::Series::setOpenPMDextension | ( | uint32_t | openPMDextension | ) |
Set a 32-bit mask of applied extensions to the openPMD standard.
openPMDextension | Unsigned 32-bit integer used as a bit-mask of applied extensions. |
Series & openPMD::Series::setParticlesPath | ( | std::string const & | particlesPath | ) |
Set the path to groups for each particle species, relative(!) to basePath
.
particlesPath | String of the path to groups for each particle species, relative(!) to basePath . |
Series & openPMD::Series::setSoftware | ( | std::string const & | newName, |
std::string const & | newVersion = std::string("unspecified") |
||
) |
Indicate the software/code/simulation that created the file.
newName | String indicating the software/code/simulation that created the file. |
newVersion | String indicating the version of the software/code/simulation that created the file. |
Series & openPMD::Series::setSoftwareDependencies | ( | std::string const & | newSoftwareDependencies | ) |
Indicate dependencies of software that were used to create the file.
newSoftwareDependencies | String indicating dependencies of software that were used to create the file (semicolon-separated list if needed). |
Series & openPMD::Series::setSoftwareVersion | ( | std::string const & | softwareVersion | ) |
Indicate the version of the software/code/simulation that created the file.
softwareVersion | String indicating the version of the software/code/simulation that created the file. |
std::string openPMD::Series::software | ( | ) | const |
no_such_attribute_error | If optional attribute is not present. |
std::string openPMD::Series::softwareDependencies | ( | ) | const |
no_such_attribute_error | If optional attribute is not present. |
std::string openPMD::Series::softwareVersion | ( | ) | const |
no_such_attribute_error | If optional attribute is not present. |
WriteIterations openPMD::Series::writeIterations | ( | ) |
Entry point to the writing end of the streaming API.
Creates and returns an instance of the WriteIterations class which is an intentionally restricted container of iterations that takes care of streaming semantics, e.g. ensuring that an iteration cannot be reopened once closed. For a less restrictive API in non-streaming situations, Series::iterations
can be accessed directly. The created object is stored as member of the Series object, hence this method may be called as many times as a user wishes. There is only one shared iterator state per Series, even when calling this method twice. Look for the WriteIterations class for further documentation.