labcore.measurement.storage#

plottr.data.datadict_storage

Provides file-storage tools for the DataDict class.

Description of the HDF5 storage format#

We use a simple mapping from DataDict to the HDF5 file. Within the file, a single DataDict is stored in a (top-level) group of the file. The data fields are datasets within that group.

Global meta data of the DataDict are attributes of the group; field meta data are attributes of the dataset (incl., the unit and axes values). The meta data keys are given exactly like in the DataDict, i.e., incl the double underscore pre- and suffix.

Functions

run_and_save_sweep(sweep, data_dir, name[, ...])

Iterates through a sweep, saving the data coming through it into a file called <name> at <data_dir> directory.

Classes

NumpyEncoder(*[, skipkeys, ensure_ascii, ...])

class labcore.measurement.storage.NumpyEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]#

Bases: JSONEncoder

default(obj)[source]#

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)
labcore.measurement.storage.run_and_save_sweep(sweep: Sweep, data_dir: str, name: str, ignore_all_None_results: bool = True, save_action_kwargs: bool = False, add_timestamps=False, archive_files: List[str] | None = None, return_data: bool = False, safe_write_mode: bool = False, **extra_saving_items) → Tuple[str | Path, DataDict | None][source]#

Iterates through a sweep, saving the data coming through it into a file called <name> at <data_dir> directory.

Parameters:

  • sweep – Sweep object to iterate through.

  • data_dir – Directory of file location.

  • name – Name of the file.

  • ignore_all_None_results – if True, don’t save any records that contain a None. if False, only do not save records that are all-None.

  • save_action_kwargs – If True, the action_kwargs of the sweep will be saved as a json file named after the first key of the kwargs dctionary followed by ‘_action_kwargs’ in the same directory as the data.

  • archive_files –

    List of files to copy into a folder called ‘archived_files’ in the same directory that the data is saved. It should be a list of paths (str), regular expressions are supported. If a folder is passed, it will copy the entire folder and all of its subdirectories and files into the archived_files folder. If one of the arguments could not be found, a message will be printed and the measurement will be performed without the file being archived. An exception is raised if the type is invalid.

    e.g. archive_files=[’.txt’, ‘calibration_files’, ‘../test_file.py’]. ‘.txt’ will copy every txt file located in the working directory. ‘calibration_files’ will copy the entire folder called calibration_files from the working directory into the archived_files folder. ‘../test_file.py’ will copy the script test_file.py from one directory above the working directory.

  • extra_saving_items – Kwargs for extra objects that should be saved. If the kwarg is a dictionary, the function will try and save it as a JSON file. If the dictionary contains objects that are not JSON serializable it will be pickled. Any other kind of object will be pickled too. The files will have their keys as names.

  • safe_write_mode – Indicates if the data should be written in safe mode or not. Look into ddh5 writer for more info.

Raises:

TypeError – A Typerror is raised if the object passed for archive_files is not correct