neuraxle.steps.flow

Neuraxle’s Flow Steps

Pipeline wrapper steps that only implement the handle methods, and don’t apply any transformation to the data.

Classes

ChooseOneOrManyStepsOf(steps[, hyperparams])

A pipeline to allow choosing many steps using an hyperparameter.

ChooseOneStepOf(steps[, hyperparams])

A pipeline to allow choosing one step using an hyperparameter.

ExpandDim(wrapped)

Similar to numpys expand_dim function, ExpandDim step expands the dimension of all the data inside the data container.

ForceMustHandleMixin

A pipeline step that only requires the implementation of handler methods :

Optional(wrapped, enabled[, …])

A wrapper to nullify a step : nullify its hyperparams, and also nullify all of his behavior.

ReversiblePreprocessingWrapper(…)

TruncableSteps with a preprocessing step(1), and a postprocessing step(2) that inverse transforms with the preprocessing step at the end (1, 2, reversed(1)).

TestOnlyWrapper(wrapped)

A wrapper to run wrapped step only in test mode

TrainOnlyWrapper(wrapped)

A wrapper to run wrapped step only in train mode

TrainOrTestOnlyWrapper(wrapped[, is_train_only])

A wrapper to run wrapped step only in test mode, or only in train mode.

class neuraxle.steps.flow.ChooseOneOrManyStepsOf(steps, hyperparams=None)[source]

A pipeline to allow choosing many steps using an hyperparameter.

Example usage :

p = Pipeline([
    ChooseOneOrManyStepsOf([
        ('a', Identity()),
        ('b', Identity())
    ])
])
p.set_hyperparams({
    'ChooseOneOrManyStepsOf__a__enabled': True,
    'ChooseOneOrManyStepsOf__b__enabled': False
})
# or
p.set_hyperparams({
    'ChooseOneOrManyStepsOf': {
        'a': { 'enabled': True },
        'b': { 'enabled': False }
    }
})

See also

Pipeline Optional

class neuraxle.steps.flow.ChooseOneStepOf(steps, hyperparams=None)[source]

A pipeline to allow choosing one step using an hyperparameter.

Example usage :

p = Pipeline([
    ChooseOneStepOf([
        ('a', Identity()),
        ('b', Identity())
    ])
])
p.set_hyperparams({
    'ChooseOneOrManyStepsOf__choice': 'a',
})
# or
p.set_hyperparams({
    'ChooseOneStepOf': {
        'a': { 'enabled': True },
        'b': { 'enabled': False }
    }
})

See also

Pipeline Optional

set_hyperparams(hyperparams: dict)[source]

Set chosen step hyperparams.

Parameters

hyperparams (HyperparameterSamples) – hyperparams

Returns

class neuraxle.steps.flow.ExpandDim(wrapped: neuraxle.base.BaseStep)[source]

Similar to numpys expand_dim function, ExpandDim step expands the dimension of all the data inside the data container. ExpandDim sends the expanded data container to the wrapped step. ExpandDim returns the transformed expanded dim reduced to its original shape (see reduce_dim()).

The wrapped step will receive a single current_id, data_input, and expected output:
  • The current_id is a list of one element that contains a single summary hash for all of the current ids.

  • The data_inputs is a list of one element that contains the original expected outputs list.

  • The expected_outputs is a list of one element that contains the original expected outputs list.

See also

ForceHandleMixin, MetaStepMixin, BaseStep BaseHasher ExpandedDataContainer

should_resume(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → bool[source]

Returns True if a step can be resumed with the given the data container, and execution context. See Checkpoint class documentation for more details on how a resumable checkpoint works.

Parameters
  • data_container – data container to resume from

  • context – execution context to resume from

Returns

if we can resume

Return type

bool

class neuraxle.steps.flow.ForceMustHandleMixin[source]
A pipeline step that only requires the implementation of handler methods :
  • handle_transform

  • handle_fit_transform

  • handle_fit

If forbids only implementing fit or transform or fit_transform without the handles. So it forces the handles.

See also

BaseStep

fit(data_inputs, expected_outputs=None) → neuraxle.steps.flow.ForceMustHandleMixin[source]
fit_transform(data_inputs, expected_outputs=None) → neuraxle.steps.flow.ForceMustHandleMixin[source]
handle_fit(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseStep', <class 'neuraxle.data_container.DataContainer'>)[source]
handle_fit_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseStep', <class 'neuraxle.data_container.DataContainer'>)[source]
handle_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]
transform(data_inputs) → neuraxle.steps.flow.ForceMustHandleMixin[source]
class neuraxle.steps.flow.Optional(wrapped: neuraxle.base.BaseStep, enabled: bool = True, nullified_return_value=None)[source]

A wrapper to nullify a step : nullify its hyperparams, and also nullify all of his behavior.

Example usage :

p = Pipeline([
    Optional(Identity(), enabled=True)
])
handle_fit(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseStep', <class 'neuraxle.data_container.DataContainer'>)[source]

Nullify wrapped step hyperparams, and don’t fit the wrapped step.

Parameters
Returns

step, data_container

Type

(BaseStep, DataContainer)

handle_fit_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseStep', <class 'neuraxle.data_container.DataContainer'>)[source]

Nullify wrapped step hyperparams, and don’t fit_transform the wrapped step.

Parameters
Returns

step, data_container

Type

(BaseStep, DataContainer)

handle_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Nullify wrapped step hyperparams, and don’t transform the wrapped step.

Parameters
Returns

step, data_container

Type

DataContainer

class neuraxle.steps.flow.ReversiblePreprocessingWrapper(preprocessing_step, postprocessing_step)[source]

TruncableSteps with a preprocessing step(1), and a postprocessing step(2) that inverse transforms with the preprocessing step at the end (1, 2, reversed(1)).

Example usage :

step = ReversiblePreprocessingWrapper(
    preprocessing_step=MultiplyBy2(),
    postprocessing_step=Add10()
)

outputs = step.transform(np.array(range(5)))

assert np.array_equal(outputs, np.array([5, 6, 7, 8, 9]))
handle_fit(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.steps.flow.ReversiblePreprocessingWrapper[source]

Handle fit by fitting preprocessing step, and postprocessing step.

Parameters
Returns

self, data_container

Return type

(ReversiblePreprocessingWrapper, DataContainer)

handle_fit_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('ReversiblePreprocessingWrapper', <class 'neuraxle.data_container.DataContainer'>)[source]

According to the idiom of (1, 2, reversed(1)), we do this, in order:

  • 1. Fit Transform preprocessing step

  • 2. Fit Transform postprocessing step

  • reversed(1). Inverse transform preprocessing step

Parameters
Returns

(self, data_container)

Return type

(ReversiblePreprocessingWrapper, DataContainer)

handle_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

According to the idiom of (1, 2, reversed(1)), we do this, in order:

  • 1. Transform preprocessing step

  • 2. Transform postprocessing step

  • reversed(1). Inverse transform preprocessing step

Parameters
Returns

data_container

Return type

DataContainer

class neuraxle.steps.flow.TestOnlyWrapper(wrapped: neuraxle.base.BaseStep)[source]

A wrapper to run wrapped step only in test mode

Execute only in train mode:

p = Pipeline([
    TestOnlyWrapper(Identity())
])
class neuraxle.steps.flow.TrainOnlyWrapper(wrapped: neuraxle.base.BaseStep)[source]

A wrapper to run wrapped step only in train mode

Execute only in train mode:

p = Pipeline([
    TrainOnlyWrapper(Identity())
])
class neuraxle.steps.flow.TrainOrTestOnlyWrapper(wrapped: neuraxle.base.BaseStep, is_train_only=True)[source]

A wrapper to run wrapped step only in test mode, or only in train mode.

Execute only in test mode:

p = Pipeline([
    TrainOrTestOnlyWrapper(Identity(), is_train_only=True)
])

Execute only in train mode:

p = Pipeline([
    TrainOnlyWrapper(Identity(), test_only=False)
])

See also

TrainOnlyWrapper, TestOnlyWrapper, ForceMustHandleMixin, MetaStepMixin, BaseStep

handle_fit(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseStep', <class 'neuraxle.data_container.DataContainer'>)[source]
Parameters
Returns

step, data_container

Type

(BaseStep, DataContainer)

handle_fit_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseStep', <class 'neuraxle.data_container.DataContainer'>)[source]
Parameters
Returns

step, data_container

Type

(BaseStep, DataContainer)

handle_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]
Parameters
Returns

step, data_container

Type

DataContainer