openPMD-api
Classes | Public Types | Public Member Functions | Public Attributes | Static Public Attributes | Friends | List of all members
openPMD::Series Class Reference

Implementation for the root level of the openPMD hierarchy. More...

#include <Series.hpp>

Inheritance diagram for openPMD::Series:
Inheritance graph
[legend]
Collaboration diagram for openPMD::Series:
Collaboration graph
[legend]

Classes

struct  ParsedInput
 

Public Types

enum class  IterationOpened : bool { HasBeenOpened , RemainsClosed }
 
using IterationIndex_t = Iteration::IterationIndex_t
 An unsigned integer type, used to identify Iterations in a Series.
 
using IterationsContainer_t = internal::SeriesData::IterationsContainer_t
 Type for a container of Iterations indexed by IterationIndex_t.
 
using iterations_t = decltype(internal::SeriesData::iterations)
 
using iterations_iterator = iterations_t::iterator
 
using Data_t = internal::SeriesData
 
- Public Types inherited from openPMD::Attributable
enum  ReadMode { IgnoreExisting , OverrideExisting , FullyReread }
 

Public Member Functions

 Series (std::string const &filepath, Access at, MPI_Comm comm, std::string const &options="{}")
 Construct a new Series. More...
 
 Series (std::string const &filepath, Access at, std::string const &options="{}")
 Construct a new Series. More...
 
 Series (Series const &)=default
 
 Series (Series &&)=default
 
Seriesoperator= (Series const &)=default
 
Seriesoperator= (Series &&)=default
 
 operator bool () const
 Is this a usable Series object? More...
 
std::string openPMD () const
 
SeriessetOpenPMD (std::string const &openPMD)
 Set the version of the enforced openPMD standard. More...
 
uint32_t openPMDextension () const
 
SeriessetOpenPMDextension (uint32_t openPMDextension)
 Set a 32-bit mask of applied extensions to the openPMD standard. More...
 
std::string basePath () const
 
SeriessetBasePath (std::string const &basePath)
 Set the common prefix for all data sets and sub-groups of a specific iteration. More...
 
std::string meshesPath () const
 
SeriessetMeshesPath (std::string const &meshesPath)
 Set the path to mesh records, relative(!) to basePath. More...
 
std::string particlesPath () const
 
SeriessetParticlesPath (std::string const &particlesPath)
 Set the path to groups for each particle species, relative(!) to basePath. More...
 
std::string author () const
 
SeriessetAuthor (std::string const &author)
 Indicate the author and contact for the information in the file. More...
 
std::string software () const
 
SeriessetSoftware (std::string const &newName, std::string const &newVersion=std::string("unspecified"))
 Indicate the software/code/simulation that created the file. More...
 
std::string softwareVersion () const
 
SeriessetSoftwareVersion (std::string const &softwareVersion)
 Indicate the version of the software/code/simulation that created the file. More...
 
std::string date () const
 
SeriessetDate (std::string const &date)
 Indicate the date of creation. More...
 
std::string softwareDependencies () const
 
SeriessetSoftwareDependencies (std::string const &newSoftwareDependencies)
 Indicate dependencies of software that were used to create the file. More...
 
std::string machine () const
 
SeriessetMachine (std::string const &newMachine)
 Indicate the machine or relevant hardware that created the file. More...
 
IterationEncoding iterationEncoding () const
 
SeriessetIterationEncoding (IterationEncoding iterationEncoding)
 Set the encoding style for multiple iterations in this series. More...
 
std::string iterationFormat () const
 
SeriessetIterationFormat (std::string const &iterationFormat)
 Set a pattern describing how to access single iterations in the raw file. More...
 
std::string name () const
 
SeriessetName (std::string const &name)
 Set the pattern for file names. More...
 
std::string backend () const
 The currently used backend. More...
 
std::string backend ()
 
void flush (std::string backendConfig="{}")
 Execute all required remaining IO operations to write or read data. More...
 
ReadIterations readIterations ()
 Entry point to the reading end of the streaming API. More...
 
void parseBase ()
 Parse the Series. More...
 
WriteIterations writeIterations ()
 Entry point to the writing end of the streaming API. More...
 
void close ()
 Close the Series and release the data storage/transport backends. More...
 
Data_tget ()
 
Data_t const & get () const
 
void setData (std::shared_ptr< internal::SeriesData > series)
 
std::unique_ptr< ParsedInputparseInput (std::string)
 
template<typename TracingJSON >
void parseJsonOptions (TracingJSON &options, ParsedInput &)
 Parse non-backend-specific configuration in JSON config. More...
 
bool hasExpansionPattern (std::string filenameWithExtension)
 
bool reparseExpansionPattern (std::string filenameWithExtension)
 
template<typename... MPI_Communicator>
void init (std::string const &filepath, Access at, std::string const &options, MPI_Communicator &&...)
 
template<typename TracingJSON , typename... MPI_Communicator>
std::tuple< std::unique_ptr< ParsedInput >, TracingJSON > initIOHandler (std::string const &filepath, std::string const &options, Access at, bool resolve_generic_extension, MPI_Communicator &&...)
 
void initSeries (std::unique_ptr< AbstractIOHandler >, std::unique_ptr< ParsedInput >)
 
void initDefaults (IterationEncoding, bool initAll=false)
 
std::future< void > flush_impl (iterations_iterator begin, iterations_iterator end, internal::FlushParams const &flushParams, bool flushIOHandler=true)
 Internal call for flushing a Series. More...
 
void flushFileBased (iterations_iterator begin, iterations_iterator end, internal::FlushParams const &flushParams, bool flushIOHandler=true)
 
void flushGorVBased (iterations_iterator begin, iterations_iterator end, internal::FlushParams const &flushParams, bool flushIOHandler=true)
 
void flushMeshesPath ()
 
void flushParticlesPath ()
 
void readFileBased ()
 
void readOneIterationFileBased (std::string const &filePath)
 
std::optional< std::deque< IterationIndex_t > > readGorVBased (bool do_always_throw_errors, bool init, std::set< IterationIndex_t > const &ignoreIterations={})
 Note on re-parsing of a Series: If init == false, the parsing process will seek for new Iterations/Records/Record Components etc. More...
 
void readBase ()
 
std::string iterationFilename (IterationIndex_t i)
 
IterationOpened openIterationIfDirty (IterationIndex_t index, Iteration iteration)
 
void openIteration (IterationIndex_t index, Iteration iteration)
 
iterations_iterator indexOf (Iteration const &)
 Find the given iteration in Series::iterations and return an iterator into Series::iterations at that place.
 
AdvanceStatus advance (AdvanceMode mode, internal::AttributableData &file, iterations_iterator it, Iteration &iteration)
 In step-based IO mode, begin or end an IO step for the given iteration. More...
 
AdvanceStatus advance (AdvanceMode mode)
 
void flushStep (bool doFlush)
 Called at the end of an IO step to store the iterations defined in the IO step to the snapshot attribute and to store that at least one step was written. More...
 
std::optional< std::vector< IterationIndex_t > > currentSnapshot () const
 
AbstractIOHandlerrunDeferredInitialization ()
 
AbstractIOHandlerIOHandler ()
 
AbstractIOHandler const * IOHandler () const
 
template<typename... MPI_Communicator>
void init (std::string const &filepath, Access at, std::string const &options, MPI_Communicator &&...comm)
 
template<typename TracingJSON , typename... MPI_Communicator>
auto initIOHandler (std::string const &filepath, std::string const &options, Access at, bool resolve_generic_extension, MPI_Communicator &&...comm) -> std::tuple< std::unique_ptr< ParsedInput >, TracingJSON >
 
- Public Member Functions inherited from openPMD::Attributable
 Attributable (NoInit)
 
Attribute getAttribute (std::string const &key) const
 Retrieve value of Attribute stored with provided key. More...
 
bool deleteAttribute (std::string const &key)
 Remove Attribute of provided value both logically and physically. More...
 
std::vector< std::string > attributes () const
 List all currently stored Attributes' keys. More...
 
size_t numAttributes () const
 Count all currently stored Attributes. More...
 
bool containsAttribute (std::string const &key) const
 Check whether am Attribute with a given key exists. More...
 
std::string comment () const
 Retrieve a user-supplied comment associated with the object. More...
 
AttributablesetComment (std::string const &comment)
 Populate Attribute corresponding to a comment with the user-supplied comment. More...
 
void seriesFlush (std::string backendConfig="{}")
 Flush the corresponding Series object. More...
 
MyPath myPath () const
 The path to this object within its containing Series. More...
 
protected Series retrieveSeries () const
 
void seriesFlush (internal::FlushParams const &)
 
void flushAttributes (internal::FlushParams const &)
 
void readAttributes (ReadMode)
 
template<typename T >
readFloatingpoint (std::string const &key) const
 Retrieve the value of a floating point Attribute of user-defined precision with ensured type-safety. More...
 
template<typename T >
std::vector< T > readVectorFloatingpoint (std::string const &key) const
 Retrieve a vector of values of a floating point Attributes of user-defined precision with ensured type-safety. More...
 
AbstractIOHandlerIOHandler ()
 
AbstractIOHandler const * IOHandler () const
 
Writable *& parent ()
 
Writable const * parent () const
 
Writablewritable ()
 
Writable const & writable () const
 
void setData (std::shared_ptr< internal::AttributableData > attri)
 
internal::AttributableDataget ()
 
internal::AttributableData const & get () const
 
bool dirty () const
 
bool dirtyRecursive () const
 O(1).
 
void setDirty (bool dirty_in)
 
void setDirtyRecursive (bool dirty_in)
 
bool written () const
 
bool & written ()
 
template<typename T >
bool setAttribute (std::string const &key, T value)
 Populate Attribute of provided name with provided value. More...
 
bool setAttribute (std::string const &key, char const value[])
 
Iteration const & containingIteration () const
 Returns the corresponding Iteration. More...
 
IterationcontainingIteration ()
 

Public Attributes

IterationsContainer_t iterations
 
std::shared_ptr< Data_tm_series = nullptr
 

Static Public Attributes

static constexpr private char const *const BASEPATH = "/data/%T/"
 

Friends

class Attributable
 
class Iteration
 
class Writable
 
class ReadIterations
 
class SeriesIterator
 
class internal::SeriesData
 
class WriteIterations
 

Additional Inherited Members

- Protected Types inherited from openPMD::Attributable
using Data_t = internal::AttributableData
 
- Protected Attributes inherited from openPMD::Attributable
std::shared_ptr< Data_tm_attri
 

Detailed Description

Implementation for the root level of the openPMD hierarchy.

Entry point and common link between all iterations of particle and mesh data.

See also
https://github.com/openPMD/openPMD-standard/blob/latest/STANDARD.md#hierarchy-of-the-data-file
https://github.com/openPMD/openPMD-standard/blob/latest/STANDARD.md#iterations-and-time-series

Constructor & Destructor Documentation

◆ Series() [1/2]

openPMD::Series::Series ( std::string const &  filepath,
Access  at,
MPI_Comm  comm,
std::string const &  options = "{}" 
)

Construct a new Series.

For further details, refer to the documentation of the non-MPI overload.

Parameters
filepathThe file path.
atAccess mode.
commThe MPI communicator.
optionsAdvanced backend configuration via JSON. May be specified as a JSON-formatted string directly, or as a path to a JSON textfile, prepended by an at sign '@'.

◆ Series() [2/2]

openPMD::Series::Series ( std::string const &  filepath,
Access  at,
std::string const &  options = "{}" 
)

Construct a new Series.

For details on access modes, JSON/TOML configuration and iteration encoding, refer to:

In case of file-based iteration encoding, the file names for each iteration are determined by an expansion pattern that must be specified. It takes one out of two possible forms:

  1. Simple form: T is replaced with the iteration index, e.g. simData_T.bp becomes simData_50.bp.
  2. Padded form: e.g. %06T is replaced with the iteration index padded to at least six digits. simData_%06T.bp becomes simData_000050.bp.

The backend is determined:

  1. Explicitly via the JSON/TOML parameter backend, e.g. {"backend": "adios2"}.
  2. Otherwise implicitly from the filename extension, e.g. simData_T.h5.

The filename extension can be replaced with a globbing pattern E. It will be replaced with an automatically determined file name extension:

  1. In CREATE mode: The extension is set to a backend-specific default extension. This requires that the backend is specified via JSON/TOML.
  2. In READ_ONLY, READ_WRITE and READ_LINEAR modes: These modes require that files already exist on disk. The disk will be scanned for files that match the pattern and the resulting file extension will be used. If the result is ambiguous or no such file is found, an error is raised.
  3. In APPEND mode: Like (2.), except if no matching file is found. In that case, the procedure of (1.) is used, owing to the fact that APPEND mode can be used to create new datasets.
Parameters
filepathThe file path.
atAccess mode.
optionsAdvanced backend configuration via JSON. May be specified as a JSON/TOML-formatted string directly, or as a path to a JSON/TOML textfile, prepended by an at sign '@'.

Member Function Documentation

◆ advance()

AdvanceStatus openPMD::Series::advance ( AdvanceMode  mode,
internal::AttributableData file,
iterations_iterator  it,
Iteration iteration 
)

In step-based IO mode, begin or end an IO step for the given iteration.

Called internally by Iteration::beginStep and Iteration::endStep.

Parameters
modeWhether to begin or end a step.
fileThe Attributable representing the iteration. In file-based iteration layout, this is an Iteration object, in group- based layout, it's the Series object.
itThe iterator within Series::iterations pointing to that iteration.
iterationThe actual Iteration object.
Returns
AdvanceStatus

◆ author()

std::string openPMD::Series::author ( ) const
Exceptions
no_such_attribute_errorIf optional attribute is not present.
Returns
String indicating author and contact for the information in the file.

◆ backend()

std::string openPMD::Series::backend ( ) const

The currently used backend.

See also
AbstractIOHandler::backendName()
Returns
String of a pattern for data backend.

◆ basePath()

std::string openPMD::Series::basePath ( ) const
Returns
String representing the common prefix for all data sets and sub-groups of a specific iteration.

◆ close()

void openPMD::Series::close ( )

Close the Series and release the data storage/transport backends.

This is an explicit API call for what the Series::~Series() destructor would do otherwise. All backends are closed after calling this method. The Series should be treated as destroyed after calling this method. The Series will be evaluated as false in boolean contexts after calling this method.

◆ date()

std::string openPMD::Series::date ( ) const
Exceptions
no_such_attribute_errorIf optional attribute is not present.
Returns
String indicating date of creation.

◆ flush()

void openPMD::Series::flush ( std::string  backendConfig = "{}")

Execute all required remaining IO operations to write or read data.

Parameters
backendConfigFurther backend-specific instructions on how to implement this flush call. Must be provided in-line, configuration is not read from files.

◆ flush_impl()

std::future< void > openPMD::Series::flush_impl ( iterations_iterator  begin,
iterations_iterator  end,
internal::FlushParams const &  flushParams,
bool  flushIOHandler = true 
)

Internal call for flushing a Series.

Any flushing of the Series will pass through this call.

Parameters
beginStart of the range of iterations to flush.
endEnd of the range of iterations to flush.
flushParamsFlush params, as documented in AbstractIOHandler.hpp.
flushIOHandlerTasks will always be enqueued to the backend. If this flag is true, tasks will be flushed to the backend.

◆ flushStep()

void openPMD::Series::flushStep ( bool  doFlush)

Called at the end of an IO step to store the iterations defined in the IO step to the snapshot attribute and to store that at least one step was written.

Parameters
doFlushIf true, flush the IO handler.

◆ iterationEncoding()

IterationEncoding openPMD::Series::iterationEncoding ( ) const
Returns
Current encoding style for multiple iterations in this series.

◆ iterationFormat()

std::string openPMD::Series::iterationFormat ( ) const
Returns
String describing a pattern describing how to access single iterations in the raw file.

◆ machine()

std::string openPMD::Series::machine ( ) const
Exceptions
no_such_attribute_errorIf optional attribute is not present.
Returns
String indicating the machine or relevant hardware that created the file.

◆ meshesPath()

std::string openPMD::Series::meshesPath ( ) const
Exceptions
no_such_attribute_errorIf optional attribute is not present.
Returns
String representing the path to mesh records, relative(!) to basePath.

◆ name()

std::string openPMD::Series::name ( ) const
Returns
String of a pattern for file names.

◆ openPMD()

std::string openPMD::Series::openPMD ( ) const
Returns
String representing the current enforced version of the openPMD standard.

◆ openPMDextension()

uint32_t openPMD::Series::openPMDextension ( ) const
Returns
32-bit mask of applied extensions to the openPMD standard.

◆ operator bool()

openPMD::Series::operator bool ( ) const

Is this a usable Series object?

Returns
true If a Series has been opened for reading and/or writing.
false If the object has been default-constructed.

◆ parseBase()

void openPMD::Series::parseBase ( )

Parse the Series.

Only necessary in linear read mode. In linear read mode, the Series constructor does not do any IO accesses. This call effectively triggers the side effects of Series::readIterations(), for use cases where data needs to be accessed before iterating through the iterations.

The reason for introducing this restricted alias to Series::readIterations() is that the name "readIterations" is misleading for that use case: When using IO steps, this call only ensures that the first step is parsed.

◆ parseJsonOptions()

template<typename TracingJSON >
void openPMD::Series::parseJsonOptions ( TracingJSON &  options,
ParsedInput input 
)

Parse non-backend-specific configuration in JSON config.

Currently this parses the keys defer_iteration_parsing, backend and iteration_encoding.

Template Parameters
TracingJSONtemplate parameter so we don't have to include the JSON lib here

◆ particlesPath()

std::string openPMD::Series::particlesPath ( ) const
Exceptions
no_such_attribute_errorIf optional attribute is not present.
Returns
String representing the path to particle species, relative(!) to basePath.

◆ readGorVBased()

auto openPMD::Series::readGorVBased ( bool  do_always_throw_errors,
bool  init,
std::set< IterationIndex_t > const &  ignoreIterations = {} 
)

Note on re-parsing of a Series: If init == false, the parsing process will seek for new Iterations/Records/Record Components etc.

If series.iterations contains the attribute snapshot, returns its value. If do_always_throw_errors is false, this method will try to handle errors and turn them into a warning (useful when parsing a Series, since parsing should succeed without issue). If true, the error will always be re-thrown (useful when using ReadIterations since those methods should be aware when the current step is broken).

◆ readIterations()

ReadIterations openPMD::Series::readIterations ( )

Entry point to the reading end of the streaming API.

Creates and returns an instance of the ReadIterations class which can be used for iterating over the openPMD iterations in a C++11-style for loop. Series::readIterations() is an intentionally restricted API that ensures a workflow which also works in streaming setups, e.g. an iteration cannot be opened again once it has been closed. For a less restrictive API in non-streaming situations, Series::iterations can be accessed directly. Look for the ReadIterations class for further documentation.

Returns
ReadIterations

◆ setAuthor()

Series & openPMD::Series::setAuthor ( std::string const &  author)

Indicate the author and contact for the information in the file.

Parameters
authorString indicating author and contact for the information in the file.
Returns
Reference to modified series.

◆ setBasePath()

Series & openPMD::Series::setBasePath ( std::string const &  basePath)

Set the common prefix for all data sets and sub-groups of a specific iteration.

Parameters
basePathString of the common prefix for all data sets and sub-groups of a specific iteration.
Returns
Reference to modified series.

◆ setDate()

Series & openPMD::Series::setDate ( std::string const &  date)

Indicate the date of creation.

Parameters
dateString indicating the date of creation.
Returns
Reference to modified series.

◆ setIterationEncoding()

Series & openPMD::Series::setIterationEncoding ( IterationEncoding  iterationEncoding)

Set the encoding style for multiple iterations in this series.

A preview on the openPMD 2.0 variable-based iteration encoding can be activated with this call. Making full use of the variable-based iteration encoding requires (1) explicit support by the backend (available only in ADIOS2) and (2) use of the openPMD streaming API. In other backends and without the streaming API, only one iteration/snapshot may be written in the variable-based encoding, making this encoding a good choice for single-snapshot data dumps.

Parameters
iterationEncodingDesired encoding style for multiple iterations in this series.
Returns
Reference to modified series.

◆ setIterationFormat()

Series & openPMD::Series::setIterationFormat ( std::string const &  iterationFormat)

Set a pattern describing how to access single iterations in the raw file.

Parameters
iterationFormatString with the iteration regex %T defining either the series of files (fileBased) or the series of groups within a single file (groupBased) that allows to extract the iteration from it. For fileBased formats the iteration must be included in the file name. The format depends on the selected iterationEncoding method.
Returns
Reference to modified series.

◆ setMachine()

Series & openPMD::Series::setMachine ( std::string const &  newMachine)

Indicate the machine or relevant hardware that created the file.

Parameters
newMachineString indicating the machine or relevant hardware that created the file (semicolon-separated list if needed)..
Returns
Reference to modified series.

◆ setMeshesPath()

Series & openPMD::Series::setMeshesPath ( std::string const &  meshesPath)

Set the path to mesh records, relative(!) to basePath.

Parameters
meshesPathString of the path to mesh records, relative(!) to basePath.
Returns
Reference to modified series.

◆ setName()

Series & openPMD::Series::setName ( std::string const &  name)

Set the pattern for file names.

Parameters
nameString of the pattern for file names. Must include iteration regex %T for fileBased data.
Returns
Reference to modified series.

◆ setOpenPMD()

Series & openPMD::Series::setOpenPMD ( std::string const &  openPMD)

Set the version of the enforced openPMD standard.

Parameters
openPMDString MAJOR.MINOR.REVISION of the desired version of the openPMD standard.
Returns
Reference to modified series.

◆ setOpenPMDextension()

Series & openPMD::Series::setOpenPMDextension ( uint32_t  openPMDextension)

Set a 32-bit mask of applied extensions to the openPMD standard.

Parameters
openPMDextensionUnsigned 32-bit integer used as a bit-mask of applied extensions.
Returns
Reference to modified series.

◆ setParticlesPath()

Series & openPMD::Series::setParticlesPath ( std::string const &  particlesPath)

Set the path to groups for each particle species, relative(!) to basePath.

Parameters
particlesPathString of the path to groups for each particle species, relative(!) to basePath.
Returns
Reference to modified series.

◆ setSoftware()

Series & openPMD::Series::setSoftware ( std::string const &  newName,
std::string const &  newVersion = std::string("unspecified") 
)

Indicate the software/code/simulation that created the file.

Parameters
newNameString indicating the software/code/simulation that created the file.
newVersionString indicating the version of the software/code/simulation that created the file.
Returns
Reference to modified series.

◆ setSoftwareDependencies()

Series & openPMD::Series::setSoftwareDependencies ( std::string const &  newSoftwareDependencies)

Indicate dependencies of software that were used to create the file.

Parameters
newSoftwareDependenciesString indicating dependencies of software that were used to create the file (semicolon-separated list if needed).
Returns
Reference to modified series.

◆ setSoftwareVersion()

Series & openPMD::Series::setSoftwareVersion ( std::string const &  softwareVersion)

Indicate the version of the software/code/simulation that created the file.

Deprecated:
Set the version with the second argument of setSoftware()
Parameters
softwareVersionString indicating the version of the software/code/simulation that created the file.
Returns
Reference to modified series.

◆ software()

std::string openPMD::Series::software ( ) const
Exceptions
no_such_attribute_errorIf optional attribute is not present.
Returns
String indicating the software/code/simulation that created the file;

◆ softwareDependencies()

std::string openPMD::Series::softwareDependencies ( ) const
Exceptions
no_such_attribute_errorIf optional attribute is not present.
Returns
String indicating dependencies of software that were used to create the file.

◆ softwareVersion()

std::string openPMD::Series::softwareVersion ( ) const
Exceptions
no_such_attribute_errorIf optional attribute is not present.
Returns
String indicating the version of the software/code/simulation that created the file.

◆ writeIterations()

WriteIterations openPMD::Series::writeIterations ( )

Entry point to the writing end of the streaming API.

Creates and returns an instance of the WriteIterations class which is an intentionally restricted container of iterations that takes care of streaming semantics, e.g. ensuring that an iteration cannot be reopened once closed. For a less restrictive API in non-streaming situations, Series::iterations can be accessed directly. The created object is stored as member of the Series object, hence this method may be called as many times as a user wishes. There is only one shared iterator state per Series, even when calling this method twice. Look for the WriteIterations class for further documentation.

Returns
WriteIterations

The documentation for this class was generated from the following files: