neuraxle.steps.misc

Module-level documentation for neuraxle.steps.misc. Here is an inheritance diagram, including dependencies to other base modules of Neuraxle:

Miscelaneous Pipeline Steps

You can find here misc. pipeline steps, for example, callbacks useful for debugging, testing, and so forth.

Classes

AssertFalseStep(message)	Assert False upon _transform_data_container and _fit_data_container.
BaseCallbackStep(callback_function, …[, …])	Base class for callback steps.
CallbackWrapper(wrapped, …[, …])	A step that calls a callback function for each of his methods : transform, fit, fit_transform, and even inverse_transform.
FitCallbackStep(callback_function, …[, …])	Call a callback method on fit.
FitTransformCallbackStep([…])
FitTransformCounterLoggingStep()
HandleCallbackStep(handle_fit_callback, …)
Sleep(sleep_time, add_random_quantity)
TapeCallbackFunction()	This class’s purpose is to be sent to the callback to accumulate information.
TransformCallbackStep(callback_function, …)	Call a callback method on transform and inverse transform.
TransformOnlyCounterLoggingStep()

Examples using neuraxle.steps.misc.Sleep

Parallel processing in Neuraxle

class neuraxle.steps.misc.AssertFalseStep(message: str = 'This step should not fit nor transform.')

Bases: neuraxle.base.HandleOnlyMixin, neuraxle.base.BaseStep

Assert False upon _transform_data_container and _fit_data_container.

__init__(message: str = 'This step should not fit nor transform.')

Initialize self. See help(type(self)) for accurate signature.

_transform_data_container(data_container, context)

Transform data container.

Parameters

• data_container – data container

• context – execution context

Returns

data container

_fit_data_container(data_container, context)

Fit data container.

Parameters

• data_container – data container

• context – execution context

Returns

(fitted self, data container)

_abc_impl = <_abc_data object>

class neuraxle.steps.misc.BaseCallbackStep(callback_function, more_arguments: List[T] = (), hyperparams=None, fit_callback_function=None, transform_function=None)

Bases: neuraxle.base.BaseStep, abc.ABC

Base class for callback steps.

__init__(callback_function, more_arguments: List[T] = (), hyperparams=None, fit_callback_function=None, transform_function=None)

Create the callback step with a function and extra arguments to send to the function

Parameters

• callback_function – The function that will be called on events.

• more_arguments – Extra arguments that will be sent to the callback after the processed data (optional).

_fit_callback(data_inputs, expected_outputs)

Will call the self.fit_callback_function() with the data being processed and the extra arguments specified. It has no other effect.

Parameters

• data_inputs – data inputs to fit

• expected_outputs – expected outputs to fit

Returns

self

_callback(data)

Will call the self.callback_function() with the data being processed and the extra arguments specified. It has no other effect.

Parameters

data_inputs – the data to process

Returns

None

fit(data_inputs, expected_outputs=None)

Fit data inputs on the given expected outputs.

Parameters

• data_inputs – data inputs

• expected_outputs – expected outputs to fit on.

Returns

self

transform(data_inputs)

Transform given data inputs.

Parameters

data_inputs – data inputs

Returns

transformed data inputs

inverse_transform(processed_outputs)

Inverse Transform the given transformed data inputs.

p = Pipeline([MultiplyByN(2)])
_in = np.array([1, 2])
_out = p.transform(_in)
_regenerated_in = p.inverse_transform(_out)
assert np.array_equal(_regenerated_in, _in)
assert np.array_equal(_out, _in * 2)

Parameters

processed_outputs – processed data inputs

Returns

inverse transformed processed outputs

_abc_impl = <_abc_data object>

class neuraxle.steps.misc.FitCallbackStep(callback_function, more_arguments: List[T] = (), hyperparams=None, fit_callback_function=None, transform_function=None)

Bases: neuraxle.steps.misc.BaseCallbackStep

Call a callback method on fit.

fit(data_inputs, expected_outputs=None) → neuraxle.steps.misc.FitCallbackStep

Will call the self._callback() with the data being processed and the extra arguments specified. Note that here, the data to process is packed into a tuple of (data_inputs, expected_outputs). It has no other effect.

Parameters

• data_inputs – the data to process

• expected_outputs – the data to process

Returns

self

_abc_impl = <_abc_data object>

class neuraxle.steps.misc.TransformCallbackStep(callback_function, more_arguments: List[T] = (), hyperparams=None, fit_callback_function=None, transform_function=None)

Bases: neuraxle.steps.misc.BaseCallbackStep

Call a callback method on transform and inverse transform.

fit_transform(data_inputs, expected_outputs=None) → Tuple[neuraxle.base.BaseStep, Any]

Fit, and transform step with the given data inputs, and expected outputs.

Parameters

• data_inputs – data inputs

• expected_outputs – expected outputs to fit on

Returns

(fitted self, tranformed data inputs)

transform(data_inputs)

Will call the self._callback() with the data being processed and the extra arguments specified. It has no other effect.

Parameters

data_inputs – the data to process

Returns

the same data as input, unchanged (like the Identity class).

inverse_transform(processed_outputs)

Will call the self._callback() with the data being processed and the extra arguments specified. It has no other effect.

Parameters

processed_outputs – the data to process

Returns

the same data as input, unchanged (like the Identity class).

_abc_impl = <_abc_data object>

class neuraxle.steps.misc.FitTransformCallbackStep(transform_callback_function=None, fit_callback_function=None, more_arguments: List[T] = (), transform_function=None, hyperparams=None)

Bases: neuraxle.base.BaseStep

__init__(transform_callback_function=None, fit_callback_function=None, more_arguments: List[T] = (), transform_function=None, hyperparams=None)

Initialize self. See help(type(self)) for accurate signature.

fit(data_inputs, expected_outputs=None)

Fit data inputs on the given expected outputs.

Parameters

• data_inputs – data inputs

• expected_outputs – expected outputs to fit on.

Returns

self

transform(data_inputs)

Transform given data inputs.

Parameters

data_inputs – data inputs

Returns

transformed data inputs

fit_transform(data_inputs, expected_outputs=None) → Tuple[neuraxle.base.BaseStep, Any]

Fit, and transform step with the given data inputs, and expected outputs.

Parameters

• data_inputs – data inputs

• expected_outputs – expected outputs to fit on

Returns

(fitted self, tranformed data inputs)

inverse_transform(processed_outputs)

Inverse Transform the given transformed data inputs.

p = Pipeline([MultiplyByN(2)])
_in = np.array([1, 2])
_out = p.transform(_in)
_regenerated_in = p.inverse_transform(_out)
assert np.array_equal(_regenerated_in, _in)
assert np.array_equal(_out, _in * 2)

Parameters

processed_outputs – processed data inputs

Returns

inverse transformed processed outputs

clear_callbacks()

_abc_impl = <_abc_data object>

class neuraxle.steps.misc.CallbackWrapper(wrapped, transform_callback_function, fit_callback_function, inverse_transform_callback_function=None, more_arguments: List[T] = (), hyperparams=None)

Bases: neuraxle.base.HandleOnlyMixin, neuraxle.base.MetaStep

A step that calls a callback function for each of his methods : transform, fit, fit_transform, and even inverse_transform. To be used with TapeCallbackFunction.

tape_fit = TapeCallbackFunction()
tape_transform = TapeCallbackFunction()
tape_inverse_transform = TapeCallbackFunction()

callback_wrapper = CallbackWrapper(MultiplyByN(2), tape_transform_preprocessing, tape_fit_preprocessing, tape_inverse_transform_preprocessing)

See also

HandleOnlyMixin, MetaStepMixin, BaseStep

__init__(wrapped, transform_callback_function, fit_callback_function, inverse_transform_callback_function=None, more_arguments: List[T] = (), hyperparams=None)

Initialize self. See help(type(self)) for accurate signature.

_fit_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → Tuple[neuraxle.base.BaseStep, neuraxle.data_container.DataContainer]

Parameters

• data_container (DataContainer) – data container

• context (ExecutionContext) – execution context

Returns

step, data_container

_fit_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → Tuple[neuraxle.base.BaseStep, neuraxle.data_container.DataContainer]

Parameters

• data_container (DataContainer) – data container

• context (ExecutionContext) – execution context

Returns

step, data_container

_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer

Return type

DataContainer

Parameters

• data_container (DataContainer) – data container

• context (ExecutionContext) – execution context

Returns

step, data_container

Type

DataContainer

handle_inverse_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer

Parameters

• context (ExecutionContext) – execution context

• data_container (DataContainer) – data containerj

Returns

data container

Return type

DataContainer

_abc_impl = <_abc_data object>

class neuraxle.steps.misc.TapeCallbackFunction

Bases: object

This class’s purpose is to be sent to the callback to accumulate information.

Example usage:

expected_tape = ["1", "2", "3", "a", "b", "4"]
tape = TapeCallbackFunction()

p = Pipeline([
    Identity(),
    TransformCallbackStep(tape.callback, ["1"]),
    TransformCallbackStep(tape.callback, ["2"]),
    TransformCallbackStep(tape.callback, ["3"]),
    AddFeatures([
        TransformCallbackStep(tape.callback, ["a"]),
        TransformCallbackStep(tape.callback, ["b"]),
    ]),
    TransformCallbackStep(tape.callback, ["4"]),
    Identity()
])
p.fit_transform(np.ones((1, 1)))

assert expected_tape == tape.get_name_tape()

__init__()

Initialize the tape (cache lists).

callback(data, name: str = '')

Will stick the data and name to the tape.

Parameters

• data – data to save

• name (str) – name to save (string)

Returns

None

reset()

Reset callback data. :return: None

get_data() → List[T]

Get the data tape

Returns

The list of data.

get_name_tape() → List[str]

Get the data tape

Returns

The list of names.

class neuraxle.steps.misc.HandleCallbackStep(handle_fit_callback, handle_transform_callback, handle_fit_transform_callback)

Bases: neuraxle.base.ForceHandleOnlyMixin, neuraxle.base.BaseStep

__init__(handle_fit_callback, handle_transform_callback, handle_fit_transform_callback)

Initialize self. See help(type(self)) for accurate signature.

_fit_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → Tuple[neuraxle.base.BaseStep, neuraxle.data_container.DataContainer]

Fit data container.

Parameters

• data_container (DataContainer) – data container

• context (ExecutionContext) – execution context

Returns

(fitted self, data container)

_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer

Transform data container.

Return type

DataContainer

Parameters

• data_container (DataContainer) – data container

• context (ExecutionContext) – execution context

Returns

data container

_fit_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → Tuple[neuraxle.base.BaseStep, neuraxle.data_container.DataContainer]

Fit and transform data container with the given execution context. Will do:

data_container, context = self._fit_data_container(data_container, context)
data_container = self._transform_data_container(data_container, context)
return self, data_container

Parameters

• data_container (DataContainer) – data container

• context (ExecutionContext) – execution context

Returns

transformed data container

_abc_impl = <_abc_data object>

class neuraxle.steps.misc.Sleep(sleep_time: float = 0.1, add_random_quantity: float = 0.0)

Bases: neuraxle.base.BaseTransformer

__init__(sleep_time: float = 0.1, add_random_quantity: float = 0.0)

Sleep for a given time, given in seconds.

transform(data_inputs)

Transform given data inputs.

Parameters

data_inputs – data inputs

Returns

transformed data inputs

_abc_impl = <_abc_data object>

class neuraxle.steps.misc.FitTransformCounterLoggingStep

Bases: neuraxle.base.HandleOnlyMixin, neuraxle.base.BaseStep

__init__()

Initialize self. See help(type(self)) for accurate signature.

_fit_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.base.BaseStep

Fit data container.

Return type

BaseStep

Parameters

• data_container (DataContainer) – data container

• context (ExecutionContext) – execution context

Returns

(fitted self, data container)

_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer

Transform data container.

Return type

DataContainer

Parameters

• data_container (DataContainer) – data container

• context (ExecutionContext) – execution context

Returns

data container

_fit_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer

Fit and transform data container with the given execution context. Will do:

data_container, context = self._fit_data_container(data_container, context)
data_container = self._transform_data_container(data_container, context)
return self, data_container

Return type

DataContainer

Parameters

• data_container (DataContainer) – data container

• context (ExecutionContext) – execution context

Returns

transformed data container

_log(context, func_name)

_abc_impl = <_abc_data object>

class neuraxle.steps.misc.TransformOnlyCounterLoggingStep

Bases: neuraxle.base.NonFittableMixin, neuraxle.steps.misc.FitTransformCounterLoggingStep

__init__()

Initialize self. See help(type(self)) for accurate signature.

_abc_impl = <_abc_data object>