neuraxle.steps.caching

Module-level documentation for neuraxle.steps.caching. Here is an inheritance diagram, including dependencies to other base modules of Neuraxle:

Inheritance diagram of neuraxle.steps.caching

Pipeline Steps For Caching

Classes

BaseValueHasher

JoblibValueCachingWrapper(wrapped, …)

Value Caching Wrapper class that caches the wrapped step transformed data inputs using python joblib library.

Md5Hasher

ValueCachingWrapper(wrapped, cache_folder, …)

Value caching wrapper wraps a step to cache the values.

Examples using neuraxle.steps.caching.JoblibValueCachingWrapper


class neuraxle.steps.caching.ValueCachingWrapper(wrapped: neuraxle.base.BaseStep, cache_folder: str = 'cache', value_hasher: Optional[neuraxle.steps.caching.BaseValueHasher] = None)[source]

Bases: neuraxle.base.MetaStep

Value caching wrapper wraps a step to cache the values.

__init__(wrapped: neuraxle.base.BaseStep, cache_folder: str = 'cache', value_hasher: Optional[neuraxle.steps.caching.BaseValueHasher] = None)[source]

Initialize self. See help(type(self)) for accurate signature.

_fit_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseStep', <class 'neuraxle.data_container.DataContainer'>)[source]

Fit transform data container.

Parameters
Returns

tuple(fitted pipeline, data_container)

_transform_data_container(data_container, context)[source]

Transform data container.

Parameters
Returns

transformed data container

_hash_value(data_input)[source]
_transform_with_cache(data_container: neuraxle.data_container.DataContainer) → Iterable[T_co][source]

Transform data container using value caching.

Parameters

data_container (neuraxle.data_container.DataContainer) – the data container to transform

Returns

iterable

create_checkpoint_path() → str[source]

Create checkpoint path.

Returns

checkpoint path

flush_cache()[source]

Flush all cached values :return:

read_cache(data_input) → Any[source]

Read cache for a given data input.

Parameters

data_input (Any) – data input to get cache for

Returns

write_cache(data_input, output)[source]

Write cache for a given data input and output.

Parameters
  • data_input (Any) – data input to write cache for

  • output (Any) – output to write cache for

Returns

contains_cache_for(data_input) → bool[source]

Returns true if the data input transform output is cached.

Return type

bool

Parameters

data_input – to get cache from

Returns

boolean to indicate if a cache is present for the given data input

get_cache_path_for(data_input) → str[source]

Get the cache path for the given data input.

Return type

str

Parameters

data_input – data input to get cache path for

Returns

str for cache path

_abc_impl = <_abc_data object>
class neuraxle.steps.caching.JoblibValueCachingWrapper(wrapped: neuraxle.base.BaseStep, cache_folder: str = 'cache', value_hasher: Optional[neuraxle.steps.caching.BaseValueHasher] = None)[source]

Bases: neuraxle.steps.caching.ValueCachingWrapper

Value Caching Wrapper class that caches the wrapped step transformed data inputs using python joblib library.

create_checkpoint_path() → str[source]

Create checkpoint path.

Returns

checkpoint path

flush_cache()[source]

Flush all cached values :return:

read_cache(data_input)[source]

Read cache for a given data input.

Parameters

data_input (Any) – data input to get cache for

Returns

write_cache(data_input, output)[source]

Write cache for a given data input and output.

Parameters
  • data_input (Any) – data input to write cache for

  • output (Any) – output to write cache for

Returns

contains_cache_for(data_input) → bool[source]

Returns true if the data input transform output is cached.

Return type

bool

Parameters

data_input – to get cache from

Returns

boolean to indicate if a cache is present for the given data input

get_cache_path_for(data_input)[source]

Get the cache path for the given data input.

Parameters

data_input – data input to get cache path for

Returns

str for cache path

_abc_impl = <_abc_data object>
class neuraxle.steps.caching.BaseValueHasher[source]

Bases: abc.ABC

hash(data_input)[source]
_abc_impl = <_abc_data object>
class neuraxle.steps.caching.Md5Hasher[source]

Bases: neuraxle.steps.caching.BaseValueHasher

_abc_impl = <_abc_data object>
hash(data_input)[source]