neuraxle.steps.caching

Pipeline Steps For Caching

Classes

BaseValueHasher

JoblibValueCachingWrapper(wrapped, …)

Joblib Value Caching Wrapper class that caches the wrapped step transformed data inputs using python pickle library.

Md5Hasher

PickleValueCachingWrapper(wrapped, …)

Value Caching Wrapper class that caches the wrapped step transformed data inputs using python pickle library.

ValueCachingWrapper(wrapped, cache_folder, …)

Value caching wrapper wraps a step to cache the values.

class neuraxle.steps.caching.BaseValueHasher[source]
hash(data_input)[source]
class neuraxle.steps.caching.JoblibValueCachingWrapper(wrapped: neuraxle.base.BaseStep, cache_folder: str = 'cache', value_hasher: Optional[neuraxle.steps.caching.BaseValueHasher] = None)[source]

Joblib Value Caching Wrapper class that caches the wrapped step transformed data inputs using python pickle library.

contains_cache_for(data_input) → bool[source]

Returns true if the data input transform output is cached.

Parameters

data_input – to get cache from

Returns

boolean to indicate if a cache is present for the given data input

create_checkpoint_path() → str[source]

Create checkpoint path.

Returns

checkpoint path

flush_cache()[source]

Flush all cached values :return:

get_cache_path_for(data_input)[source]

Get the cache path for the given data input.

Parameters

data_input – data input to get cache path for

Returns

str for cache path

read_cache(data_input)[source]

Read cache for a given data input.

Parameters

data_input (Any) – data input to get cache for

Returns

write_cache(data_input, output)[source]

Write cache for a given data input and output.

Parameters
  • data_input (Any) – data input to write cache for

  • output (Any) – output to write cache for

Returns

class neuraxle.steps.caching.Md5Hasher[source]
hash(data_input)[source]
class neuraxle.steps.caching.PickleValueCachingWrapper(wrapped: neuraxle.base.BaseStep, cache_folder: str = 'cache', value_hasher: Optional[neuraxle.steps.caching.BaseValueHasher] = None)[source]

Value Caching Wrapper class that caches the wrapped step transformed data inputs using python pickle library.

contains_cache_for(data_input) → bool[source]

Returns true if the data input transform output is cached.

Parameters

data_input – to get cache from

Returns

boolean to indicate if a cache is present for the given data input

create_checkpoint_path() → str[source]

Create checkpoint path.

Returns

checkpoint path

flush_cache()[source]

Flush all cached values :return:

get_cache_path_for(data_input)[source]

Get the cache path for the given data input.

Parameters

data_input – data input to get cache path for

Returns

str for cache path

read_cache(data_input)[source]

Read cache for a given data input.

Parameters

data_input (Any) – data input to get cache for

Returns

write_cache(data_input, output)[source]

Write cache for a given data input and output.

Parameters
  • data_input (Any) – data input to write cache for

  • output (Any) – output to write cache for

Returns

class neuraxle.steps.caching.ValueCachingWrapper(wrapped: neuraxle.base.BaseStep, cache_folder: str = 'cache', value_hasher: Optional[neuraxle.steps.caching.BaseValueHasher] = None)[source]

Value caching wrapper wraps a step to cache the values.

contains_cache_for(data_input) → bool[source]

Returns true if the data input transform output is cached.

Parameters

data_input – to get cache from

Returns

boolean to indicate if a cache is present for the given data input

create_checkpoint_path() → str[source]

Create checkpoint path.

Returns

checkpoint path

flush_cache()[source]

Flush all cached values :return:

get_cache_path_for(data_input) → str[source]

Get the cache path for the given data input.

Parameters

data_input – data input to get cache path for

Returns

str for cache path

read_cache(data_input) → Any[source]

Read cache for a given data input.

Parameters

data_input (Any) – data input to get cache for

Returns

write_cache(data_input, output)[source]

Write cache for a given data input and output.

Parameters
  • data_input (Any) – data input to write cache for

  • output (Any) – output to write cache for

Returns