#include <hdf5_reader.h>

Public Types
using	supported_1D_types = std::tuple< uint8_t, uint16_t, uint32_t, uint64_t, int8_t, int16_t, int32_t, int64_t, float, double, std::string >

using	supported_2D_types = std::tuple< std::array< uint64_t, 2 > >

Public Member Functions
	Hdf5Reader ()
	Create a valid Hdf5Reader with the default plugin.

	Hdf5Reader (std::shared_ptr< Hdf5PluginInterface< supported_1D_types, supported_2D_types >> impl)
	Create an Hdf5Reader with a user supplied plugin.

template<class T >
std::vector< T >	readSelection (const HighFive::DataSet &dset, const Selection &selection) const

HighFive::File	openFile (const std::string &filename) const

template<class T >
std::vector< T >	readSelection (const HighFive::DataSet &dset, const Selection &xsel, const Selection &ysel) const

Detailed Description

Abstraction for reading HDF5 datasets.

The Hdf5Reader provides an interface for reading canonical selections from datasets. Selections are canonical if they are sorted and don't overlap. This allows implementing different optimization strategies, such as minimizing bytes read, aggregating nearby reads or using MPI collective I/O.

The design uses virtual inheritance, which enables users to inject their own reader if needed. This class is the interface used within libsonata. It simply delegates to a "plugin", that satisfies the interface Hdf5PluginInterface.

To enable MPI collective I/O, libsonata must call all methods in an MPI-collective manner. This implies that the number of times any function in libsonata calls any of the Hdf5Reader methods must not depend on the arguments to the function.

Examples:

void wrong(Selection selection) { // Wrong because some MPI ranks might return without // calling readSelection. if(selection.empty()) { return; } hdf5_reader.readSelection(dset, selection); }

void also_wrong(Selection selection) { // Wrong because hdf5_reader is called selection.ranges().size() // number of times. Which could be different on each MPI rank. for(auto range : selection.ranges()) { hdf5_reader.readSelection(dset, Selection(std::vector{range})); } }

void subtle(Selection selection, bool flag) { // If the flag can differ between MPI ranks, this is wrong because // readSelection is called with different dsets. If the flag must // be the same on all MPI ranks, this is correct. If this happens in // the libsonata API, then passing the same flag on all MPI ranks becomes // a requirement for the users, when using a collective reader. Example: // pop.get_attribute(attr_name, selection) if(flag) { hdf5_reader.readSelection(dset1, selection); } else { hdf5_reader.readSelection(dset2, selection); } }

void correct(Selection selection) { // Correct because no matter which branch is taken // hdf5_reader.readSelection is called exactly once. if(selection.size % 2 == 0) { hdf5_reader.readSelection(dset, selection); } else { hdf5_reader.readSelection(dset, {}); } }

Member Function Documentation

◆ openFile()

HighFive::File bbp::sonata::Hdf5Reader::openFile ( const std::string & filename ) const

Open the HDF5.

The dataset passed to readSelection must be obtained from a file open via this method.

◆ readSelection() [1/2]

template<class T >

std::vector<T> bbp::sonata::Hdf5Reader::readSelection	(	const HighFive::DataSet &	dset,
		const Selection &	selection
	)		const

inline

Read the selected subset of the one-dimensional array.

Both selections are canonical, i.e. sorted and non-overlapping. The dataset is obtained from a HighFive::File opened via this->openFile.

◆ readSelection() [2/2]