Welcome to hickle’s documentation!
Hickle is an HDF5-based clone of pickle, with a twist: instead of serializing to a pickle file, Hickle dumps to an HDF5 file (Hierarchical Data Format). It is designed to be a “drop-in” replacement for pickle (for common data objects), but is really an amalgam of h5py and dill/pickle with extended functionality.
That is: hickle is a neat little way of dumping python variables to HDF5 files that can be read in most programming languages, not just Python. Hickle is fast, and allows for transparent compression of your data (LZF / GZIP).
- hickle.dump(py_obj, file_obj, mode='w', path='/', *, filename=None, options={}, **kwargs)
Write a hickled representation of py_obj to the provided file_obj.
- Parameters:
(object) (py_obj) – Python object to hickle to HDF5.
(file (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
file-like (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
h5py.File (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
str (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
(file –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
str) (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
{'file' (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
(str) (filename) –
string indicating how the file shall be opened. For details see Python open.
- Note: The ‘b’ flag is optional as all files are and have to be opened in
binary mode.
(str) – Path within HDF5-file or group to dump to/load from.
(str) – The name of the file. Ignored when f is str or h5py.File object.
(dict) (options) –
Each entry in this dict modifies how hickle dumps data to file. For example
{ custom = True }
would enforce use of custom loaders on all classes registered with this kind of loader.
{ custom = False }
would disable custom loaders for dumped data even if globally turned on. More options may follow.
kwargs (keyword arguments) – Additional keyword arguments that must be provided to the
create_dataset()method. For example compression=True
- Raises:
CloseFileError: – If passed h5py.File, h5py.Group or h5py.Dataset object is not accessible. This in most cases indicates that underlying HDF5 was closed or if file or file or file-like object has already been closed.
FileError – If passed file or file-like object is not opened for reading or in addition for writing in case mode corresponds to any of ‘w’, ‘w+’, ‘x’, ‘x+’ or a.
ValueError: – If anything else than str, bytes or None specified for filename or for mode is anything else specified than ‘w’,’w+’,’x’,’x+’,’r’,’r+’,’a’ or contains any optional open flag other than ‘b’
- hickle.load(file_obj, path='/', safe=True, filename=None)
Load the Python object stored in file_obj at path and return it.
- Parameters:
(file (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
file-like (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
h5py.File (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
str (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
(file –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
str) (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
{'file' (file,'name':str} ):) –
File to open for dumping or loading purposes. str:
the path of the HDF5-file that must be used.
- ~h5py.Group:
the group (or file) in an open HDF5-file that must be used.
- file, file-like:
file or like object which provides read, seek, tell and write methods
- tuple:
two element tuple with the first being the file or file like object to dump to and the second the filename to be used instead of ‘filename’ parameter
- dict:
dictionary with ‘file’ and ‘name’ items
- path (str): optional
Path within HDF5-file or group to dump to/load from.
- safe (bool): optional
When True (default), raise UnsafeFileError if the file contains any objects that were serialized with pickle. Pickle payloads can execute arbitrary code on load, so never set this to False unless the file originates from a completely trusted source. (See https://docs.python.org/3/library/pickle.html)
Prefer hickle.load_unsafe() over passing safe=False explicitly; it makes the unsafe intent visible at the call site.
- filename (str): optional
The name of the file. Ignored when f is str or h5py.File object.
- Returns:
py_obj – The unhickled Python object.
- Return type:
object
- Raises:
CloseFileError: – If passed h5py.File, h5py.Group or h5py.Dataset object is not accessible. This in most cases indicates that underlying HDF5 was closed or if file or file or file-like object has already been closed.
FileError – If passed file or file-like object is not opened for reading
UnsafeFileError: – If safe=True (the default) and the file contains pickle-serialized objects. Use hickle.load_unsafe() to suppress this for trusted files.
ValueError: – If anything else than str, bytes or None specified for filename