neuraxle.steps.loop

Module-level documentation for neuraxle.steps.loop. Here is an inheritance diagram, including dependencies to other base modules of Neuraxle:

Inheritance diagram of neuraxle.steps.loop

Pipeline Steps For Looping

Classes

Break()

BreakIf(condition_function)

Continue()

ContinueIf(condition_function)

FlattenForEach(wrapped, then_unflatten)

Step that reduces a dimension instead of manually looping on it.

ForEach(wrapped[, cache_folder_when_no_handle])

Truncable step that fits/transforms each step for each of the data inputs, and expected outputs.

StepClonerForEachDataInput(wrapped[, …])

Examples using neuraxle.steps.loop.FlattenForEach

Examples using neuraxle.steps.loop.ForEach


Exceptions

BreakInterrupt

This exception is used to signal the interruption of a minibatch

ContinueInterrupt

This exception is used to signal to the minibatch iterator to skip the rest of the execution of the current iteration.

class neuraxle.steps.loop.ForEach(wrapped: neuraxle.base.BaseTransformer, cache_folder_when_no_handle=None)[source]

Bases: neuraxle.base.ForceHandleOnlyMixin, neuraxle.base.ResumableStepMixin, neuraxle.base.MetaStep

Truncable step that fits/transforms each step for each of the data inputs, and expected outputs.

__init__(wrapped: neuraxle.base.BaseTransformer, cache_folder_when_no_handle=None)[source]

Initialize self. See help(type(self)) for accurate signature.

_fit_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.base.BaseStep[source]

Fit each step for each data inputs, and expected outputs

Return type

BaseStep

Parameters
Returns

self

_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Transform each step for each data inputs.

Return type

DataContainer

Parameters
Returns

self

_fit_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → Tuple[neuraxle.base.BaseStep, neuraxle.data_container.DataContainer][source]

Fit transform each step for each data inputs, and expected outputs

Parameters
Returns

self, transformed_data_container

hash_data_container(data_container: neuraxle.data_container.DataContainer) → neuraxle.data_container.DataContainer[source]

Hash data container using self.hashers.

  1. Hash current ids with hyperparams.

  2. Hash summary id with hyperparams.

Return type

DataContainer

Parameters

data_container (DataContainer) – the data container to transform

Returns

transformed data container

should_resume(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → bool[source]

Returns True if a step can be resumed with the given the data container and execution context. See Checkpoint class documentation for more details on how a resumable checkpoint works.

Return type

bool

Parameters
Returns

if we can resume

resume(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
_abc_impl = <_abc_data object>
exception neuraxle.steps.loop.ContinueInterrupt[source]

Bases: Exception

This exception is used to signal to the minibatch iterator to skip the rest of the execution of the current iteration.

exception neuraxle.steps.loop.BreakInterrupt[source]

Bases: Exception

This exception is used to signal the interruption of a minibatch

class neuraxle.steps.loop.Break[source]

Bases: neuraxle.base.ForceHandleMixin, neuraxle.base.Identity

__init__()[source]

Initialize self. See help(type(self)) for accurate signature.

_did_process(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Apply side effects after any step method.

Return type

DataContainer

Parameters
Returns

(data container, execution context)

_abc_impl = <_abc_data object>
class neuraxle.steps.loop.BreakIf(condition_function: Callable)[source]

Bases: neuraxle.steps.flow.ExecuteIf

__init__(condition_function: Callable)[source]

Initialize self. See help(type(self)) for accurate signature.

_abc_impl = <_abc_data object>
class neuraxle.steps.loop.Continue[source]

Bases: neuraxle.base.ForceHandleMixin, neuraxle.base.Identity

__init__()[source]

Initialize self. See help(type(self)) for accurate signature.

_did_process(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Apply side effects after any step method.

Return type

DataContainer

Parameters
Returns

(data container, execution context)

_abc_impl = <_abc_data object>
class neuraxle.steps.loop.ContinueIf(condition_function: Callable)[source]

Bases: neuraxle.steps.flow.ExecuteIf

__init__(condition_function: Callable)[source]

Initialize self. See help(type(self)) for accurate signature.

_abc_impl = <_abc_data object>
class neuraxle.steps.loop.StepClonerForEachDataInput(wrapped: neuraxle.base.BaseTransformer, copy_op=<function deepcopy>, cache_folder_when_no_handle=None)[source]

Bases: neuraxle.base.ForceHandleOnlyMixin, neuraxle.base.MetaStep

__init__(wrapped: neuraxle.base.BaseTransformer, copy_op=<function deepcopy>, cache_folder_when_no_handle=None)[source]

Initialize self. See help(type(self)) for accurate signature.

get_children() → List[neuraxle.base.BaseStep][source]

Get the list of all the children for that step.

Returns

list of children

_will_process(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseStep', <class 'neuraxle.data_container.DataContainer'>)[source]

Apply side effects before any step method. :type context: ExecutionContext :type data_container: DataContainer :param data_container: data container :param context: execution context :return: (data container, execution context)

_copy_one_step_per_data_input(data_container)[source]
_fit_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseStep', <class 'neuraxle.data_container.DataContainer'>)[source]

Fit transform data container.

Parameters
Returns

(fitted self, data container)

_fit_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseStep', <class 'neuraxle.data_container.DataContainer'>)[source]

Fit data container.

Parameters
Returns

(fitted self, data container)

_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseStep', <class 'neuraxle.data_container.DataContainer'>)[source]

Transform data container.

Parameters
Returns

data container

_inverse_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]
inverse_transform(processed_outputs)[source]

Inverse Transform the given transformed data inputs.

mutate() or reverse() can be called to change the default transform behavior :

p = Pipeline([MultiplyBy()])
_in = np.array([1, 2])
_out = p.transform(_in)
_regenerated_in = reversed(p).transform(_out)
assert np.array_equal(_regenerated_in, _in)
Parameters

processed_outputs – processed data inputs

Returns

inverse transformed processed outputs

_abc_impl = <_abc_data object>
class neuraxle.steps.loop.FlattenForEach(wrapped: neuraxle.base.BaseTransformer, then_unflatten: bool = True)[source]

Bases: neuraxle.base.ForceHandleMixin, neuraxle.base.ResumableStepMixin, neuraxle.base.MetaStep

Step that reduces a dimension instead of manually looping on it.

__init__(wrapped: neuraxle.base.BaseTransformer, then_unflatten: bool = True)[source]

Initialize self. See help(type(self)) for accurate signature.

_will_process(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseTransformer', <class 'neuraxle.data_container.DataContainer'>)[source]

Flatten data container before any processing is done on the wrapped step.

Parameters
Returns

(data container, execution context)

Return type

(‘BaseTransformer’, DataContainer)

_flatten_list(list_to_flatten)[source]

Flatten the first dimension of a list.

Parameters

list_to_flatten – list to flatten

Returns

flattened list, len flattened lists

_did_process(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Reaugment the flattened data container.

Return type

DataContainer

Parameters
Returns

data container

_reaugment_list(list_to_reaugment, flattened_dimension_lengths)[source]

Reaugment list with the flattened dimension lengths.

Parameters

list_to_reaugment – list to then_unflatten

Returns

reaugmented numpy array

should_resume(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]

Returns True if a step can be resumed with the given the data container and execution context. See Checkpoint class documentation for more details on how a resumable checkpoint works.

Parameters
Returns

if we can resume

_abc_impl = <_abc_data object>