Welcome to hickle’s documentation!
Hickle is an HDF5-based clone of pickle, with a twist: instead of serializing to a pickle file, Hickle dumps to an HDF5 file (Hierarchical Data Format). It is designed to be a “drop-in” replacement for pickle (for common data objects), but is really an amalgam of h5py and dill/pickle with extended functionality.
That is: hickle is a neat little way of dumping python variables to HDF5 files that can be read in most programming languages, not just Python. Hickle is fast, and allows for transparent compression of your data (LZF / GZIP).
- hickle.dump(py_obj, file_obj, mode='w', path='/', *, filename=None, options={}, **kwargs)
Write a hickled representation of py_obj to the provided file_obj.
- Parameters:
(object) (py_obj) – Python object to hickle to HDF5.
(file (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
file-like (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
h5py.File (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
str (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
(file –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
str) (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
{'file' (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
(str) (filename) –
string indicating how the file shall be opened. For details see Python open.
- Note: The ‘b’ flag is optional as all files are and have to be opened in
binary mode.
(str) – Path within HDF5-file or group to dump to/load from.
(str) – The name of the file. Ignored when f is str or h5py.File object.
(dict) (options) –
Each entry in this dict modifies how hickle dumps data to file. For example
{ custom = True }
would enforce use of custom loaders on all classes registered with this kind of loader.
{ custom = False }
would disable custom loaders for dumped data even if globally turned on. More options may follow.
kwargs (keyword arguments) – Additional keyword arguments that must be provided to the
create_dataset()method. For example compression=True
- Raises:
CloseFileError: – If passed h5py.File, h5py.Group or h5py.Dataset object is not accessible. This in most cases indicates that underlying HDF5 was closed or if file or file or file-like object has already been closed.
FileError – If passed file or file-like object is not opened for reading or in addition for writing in case mode corresponds to any of ‘w’, ‘w+’, ‘x’, ‘x+’ or a.
ValueError: – If anything else than str, bytes or None specified for filename or for mode is anything else specified than ‘w’,’w+’,’x’,’x+’,’r’,’r+’,’a’ or contains any optional open flag other than ‘b’
- hickle.load(file_obj, path='/', safe=True, filename=None)
Load the Python object stored in file_obj at path and return it.
- Parameters:
(file (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
file-like (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
h5py.File (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
str (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
(file –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
str) (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
{'file' (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
- path (str): optional
Path within HDF5-file or group to dump to/load from.
- safe (bool): optional
Disable automatic depickling of arbitrary python objects. DO NOT set this to False unless the file is from a trusted source. (See https://docs.python.org/3/library/pickle.html for an explanation)
Note: ignored when loading hickle 4.x and newer files
- filename (str): optional
The name of the file. Ignored when f is str or h5py.File object.
- Returns:
py_obj – The unhickled Python object.
- Return type:
object
- Raises:
CloseFileError: – If passed h5py.File, h5py.Group or h5py.Dataset object is not accessible. This in most cases indicates that underlying HDF5 was closed or if file or file or file-like object has already been closed.
FileError – If passed file or file-like object is not opened for reading
ValueError: – If anything else than str, bytes or None specified for filename