neuraxle.base

Neuraxle’s Base Classes

This is the core of Neuraxle. Most pipeline steps derive (inherit) from those classes. They are worth noticing.

Classes

AssertExpectedOutputIsNone()

AssertExpectedOutputIsNoneMixin()

AssertExpectedOutputIsNoneStep()

AssertExpectedOutputIsNotNoneMixin()

AssertionMixin()

BaseHasher

Base class to hash hyperparamters, and data input ids together.

BaseSaver

Any saver must inherit from this one.

BaseStep(hyperparams, hyperparams_space, …)

Base class for a transformer step that can also be fitted.

BaseTransformer(hyperparams, …)

Base class for a pipeline step that can only be transformed.

DidProcessAssertionMixin()

EvaluableStepMixin()

A step that can be evaluated with the scoring functions.

ExecutionContext(root, execution_phase, …)

Execution context object containing all of the pipeline hierarchy steps.

ExecutionMode

An enumeration.

ExecutionPhase

An enumeration.

ForceHandleIdentity()

ForceHandleMixin([cache_folder])

A step that automatically calls handle methods in the transform, fit, and fit_transform methods.

ForceHandleOnlyMixin([cache_folder])

A step that automatically calls handle methods in the transform, fit, and fit_transform methods.

FullDumpLoader(name[, stripped_saver])

Identity step that can load the full dump of a pipeline step.

GlobalServiceAssertionExecutorMixin()

Any step which inherit of this class will test globaly retrievable service assertion of itself and all its children on a will_process call.

GlobalyRetrievableServiceAssertionWrapper(…)

Is used to assert the presence of service at the start of the pipeline AND at execution time for a given step.

HandleOnlyMixin()

A pipeline step that only requires the implementation of handler methods :

HashlibMd5Hasher

Class to hash hyperparamters, and data input ids together using md5 algorithm from hashlib : https://docs.python.org/3/library/hashlib.html

HashlibMd5ValueHasher

Identity([savers, name])

A pipeline step that has no effect at all but to return the same data without changes.

IdentityHandlerMethodsMixin([cache_folder])

A step that has a default implementation for all handler methods.

JoblibStepSaver

Saver that can save, or load a step with joblib.load, and joblib.dump.

LocalServiceAssertionWrapper(wrapped, …)

Is used to assert the presence of service at execution time for a given step

MetaStep(wrapped, hyperparams, …)

MetaStepJoblibStepSaver()

Custom saver for meta step mixin.

MetaStepMixin(wrapped, savers)

A class to represent a step that wraps another step.

MixinForBaseTransformer()

Any steps/transformers within a pipeline that inherits of this class should implement BaseStep/BaseTransformer and initialize it before any mixin.

NonFittableMixin()

A pipeline step that requires no fitting: fitting just returns self when called to do no action.

NonTransformableMixin()

A pipeline step that has no effect at all but to return the same data without changes.

ResumableStepMixin()

Mixin to add resumable function to a step, or a class that can be resumed, for example a checkpoint on disk.

StepWithContext(wrapped, context, …)

TransformHandlerOnlyMixin()

A pipeline step that only requires the implementation of _transform_data_container.

TruncableJoblibStepSaver()

Step saver for a TruncableSteps.

TruncableSteps(steps_as_tuple, …)

Step that contains multiple steps.

WillProcessAssertionMixin()

class neuraxle.base.AssertExpectedOutputIsNone[source]
_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
class neuraxle.base.AssertExpectedOutputIsNoneMixin[source]
_assert(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
class neuraxle.base.AssertExpectedOutputIsNoneStep[source]
_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
class neuraxle.base.AssertExpectedOutputIsNotNoneMixin[source]
_assert(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
class neuraxle.base.AssertionMixin[source]
_assert(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
class neuraxle.base.BaseHasher[source]

Base class to hash hyperparamters, and data input ids together. The DataContainer class uses the hashed values for its current ids. BaseStep uses many BaseHasher objects to hash hyperparameters, and data inputs ids together after each transform.

See also

DataContainer

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
hash(current_ids: List[str], hyperparameters: neuraxle.hyperparams.space.HyperparameterSamples, data_inputs: Iterable) → List[str][source]

Hash DataContainer.current_ids, data inputs, and hyperparameters together.

Parameters
  • current_ids – current hashed ids (can be None if this function has not been called yet)

  • hyperparameters – step hyperparameters to hash with current ids

  • data_inputs – data inputs to hash current ids for

Returns

the new hashed current ids

single_hash(current_id: str, hyperparameters: neuraxle.hyperparams.space.HyperparameterSamples) → List[str][source]

Hash summary id, and hyperparameters together.

Parameters
  • current_id – current hashed id

  • hyperparameters (HyperparameterSamples) – step hyperparameters to hash with current ids

Returns

the new hashed current id

class neuraxle.base.BaseSaver[source]

Any saver must inherit from this one. Some savers just save parts of objects, some save it all or what remains. Each :class`BaseStep` can potentially have multiple savers to make serialization possible.

See also

save(), load()

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
can_load(step: neuraxle.base.BaseTransformer, context: neuraxle.base.ExecutionContext)[source]

Returns true if we can load the given step with the given execution context.

Parameters
  • step – step to load

  • context – execution context to load from

Returns

load_step(step: neuraxle.base.BaseTransformer, context: neuraxle.base.ExecutionContext) → neuraxle.base.BaseTransformer[source]

Load step with execution context.

Parameters
  • step – step to load

  • context – execution context to load from

Returns

loaded base step

save_step(step: neuraxle.base.BaseTransformer, context: neuraxle.base.ExecutionContext) → neuraxle.base.BaseTransformer[source]

Save step with execution context.

Parameters
  • step – step to save

  • context – execution context

  • save_savers

Returns

class neuraxle.base.BaseStep(hyperparams: neuraxle.hyperparams.space.HyperparameterSamples = None, hyperparams_space: neuraxle.hyperparams.space.HyperparameterSpace = None, name: str = None, savers: List[neuraxle.base.BaseSaver] = None, hashers: List[neuraxle.base.BaseHasher] = None)[source]

Base class for a transformer step that can also be fitted.

If a step is not fittable, you can inherit from BaseTransformer instead. If a step is not transformable, you can inherit from NonTransformableMixin. A step should only change its state inside fit() or fit_transform() (see _FittableStep for more info). Every step can be saved using its savers of type BaseSaver (see _HasHashers for more info). Most step hash data inputs with hyperparams after every transformations to update the current ids inside the DataContainer (see _HasHashers for more info). Every step has hyperparemeters, and hyperparameters spaces that can be set before the learning process begins (see _HasHyperparams, and _HasHyperparamsSpace for more info).

Example usage :

class Normalize(BaseStep):
    def __init__(self):
        BaseStep.__init__(self)
        self.mean = None
        self.std = None

    def fit(self, data_inputs, expected_outputs=None) -> 'BaseStep':
        self._calculate_mean_std(data_inputs)
        return self

    def _calculate_mean_std(self, data_inputs):
        self.mean = np.array(data_inputs).mean(axis=0)
        self.std = np.array(data_inputs).std(axis=0)

    def fit_transform(self, data_inputs, expected_outputs=None):
        self.fit(data_inputs, expected_outputs)
        return self, (np.array(data_inputs) - self.mean) / self.std

    def transform(self, data_inputs):
        if self.mean is None or self.std is None:
            self._calculate_mean_std(data_inputs)
        return (np.array(data_inputs) - self.mean) / self.std

p = Pipeline([
    Normalize()
])

p, outputs = p.fit_transform(data_inputs, expected_outputs)

Note

All heavy initialization logic should be done inside the setup method (e.g.: things inside GPU), and NOT in the constructor.

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
class neuraxle.base.BaseTransformer(hyperparams: neuraxle.hyperparams.space.HyperparameterSamples = None, hyperparams_space: neuraxle.hyperparams.space.HyperparameterSpace = None, name: str = None, savers: List[neuraxle.base.BaseSaver] = None, hashers: List[neuraxle.base.BaseHasher] = None)[source]

Base class for a pipeline step that can only be transformed.

Every step can be saved using its savers of type BaseSaver (see _HasHashers for more info). Most step hash data inputs with hyperparams after every transformations to update the current ids inside the DataContainer (see _HasHashers for more info). Every step has hyperparemeters, and hyperparameters spaces that can be set before the learning process begins (see _HasHyperparams, and _HasHyperparamsSpace for more info).

Example usage :

class AddN(BaseTransformer):
    def __init__(self, add=1):
        super().__init__(hyperparams=HyperparameterSamples({ 'add': add }))

    def transform(self, data_inputs):
        if not isinstance(data_inputs, np.ndarray):
            data_inputs = np.array(data_inputs)

        return data_inputs + self.hyperparams['add']

    def inverse_transform(self, data_inputs):
        if not isinstance(data_inputs, np.ndarray):
            data_inputs = np.array(data_inputs)

        return data_inputs - self.hyperparams['add']

Note

All heavy initialization logic should be done inside the setup method (e.g.: things inside GPU), and NOT in the constructor.

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
_invalidate()[source]
_set_train(is_train) → neuraxle.hyperparams.space.RecursiveDict[source]
_teardown() → neuraxle.base.BaseTransformer[source]

Teardown step after program execution. Inverse of setup, and it should clear memory. Override this method if you need to clear memory.

Returns

self

get_name() → str[source]

Get the name of the pipeline step.

Returns

the name, a string.

Note

A step name is the same value as the one in the keys of Pipeline.steps_as_tuple

get_step_by_name(name)[source]
invalidate() → neuraxle.base.BaseTransformer[source]

Invalidate a step, and all of its children. Invalidating a step makes it eligible to be saved again.

A step is invalidated when any of the following things happen :
  • a mutation has been performed on the step: func:~neuraxle.base._HasMutations.mutate

  • an hyperparameter has changed func: ~neuraxle.base._HasHyperparams.set_hyperparams

  • an hyperparameter space has changed func: ~neuraxle.base._HasHyperparamsSpace.set_hyperparams_space

  • a call to the fit method func:~neuraxle.base._FittableStep.handle_fit

  • a call to the fit_transform method func:~neuraxle.base._FittableStep.handle_fit_transform

  • the step name has changed func:~neuraxle.base.BaseStep.set_name

Returns

self

Note

This is a recursive method used in :class:̀_HasChildrenMixin`.

See also

apply(), _apply()

reverse() → neuraxle.base.BaseTransformer[source]

The object will mutate itself such that the .transform method (and of all its underlying objects if applicable) be replaced by the .inverse_transform method.

Note: the reverse may fail if there is a pending mutate that was set earlier with .will_mutate_to.

Returns

a copy of self, reversed. Each contained object will also have been reversed if self is a pipeline.

set_name(name: str)[source]

Set the name of the pipeline step.

Parameters

name – a string.

Returns

self

Note

A step name is the same value as the one in the keys of steps_as_tuple

set_train(is_train: bool = True)[source]

This method overrides the method of BaseStep to also consider the wrapped step as well as self. Set pipeline step mode to train or test.

Note

This is a recursive method used in :class:̀_HasChildrenMixin`.

See also

apply(), _apply() _set_train()

setup(context: neuraxle.base.ExecutionContext = None) → neuraxle.base.BaseTransformer[source]

Initialize the step before it runs. Only from here and not before that heavy things should be created (e.g.: things inside GPU), and NOT in the constructor.

Warning

The setup method is called once for each step when handle_fit, handle_fit_transform or handle_transform is called. The setup method is executed only if is self.is_initialized is False A setup function should set the self.is_initialized to True when called.

Parameters

context – execution context

Returns

self

class neuraxle.base.DidProcessAssertionMixin[source]
_did_process(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> (<class 'neuraxle.data_container.DataContainer'>, <class 'neuraxle.base.ExecutionContext'>)[source]

Calls self._assert(data_container,context)

class neuraxle.base.EvaluableStepMixin[source]

A step that can be evaluated with the scoring functions.

See also

BaseStep :class:’MixinForBaseTransformer’

get_score()[source]
class neuraxle.base.ExecutionContext(root: str = '/home/gui/Documents/GIT/www.neuraxle.org-builder/docs/cache', execution_phase: neuraxle.base.ExecutionPhase = <ExecutionPhase.UNSPECIFIED: None>, execution_mode: neuraxle.base.ExecutionMode = <ExecutionMode.FIT_OR_FIT_TRANSFORM_OR_TRANSFORM: 'fit_or_fit_transform_or_transform'>, stripped_saver: neuraxle.base.BaseSaver = None, parents: List[BaseStep] = None, services: Dict[Type, object] = None, logger: logging.Logger = None)[source]

Execution context object containing all of the pipeline hierarchy steps. First item in execution context parents is root, second is nested, and so on. This is like a stack.

The execution context is used for fitted step saving, and caching :
copy()[source]
empty()[source]

Return True if the context doesn’t have parent steps.

Returns

if parents len is 0

get_execution_mode() → neuraxle.base.ExecutionMode[source]
get_names()[source]

Returns a list of the parent names.

Returns

list of parents step names

get_path(is_absolute: bool = True)[source]

Creates the directory path for the current execution context.

Parameters

is_absolute – bool to say if we want to add root to the path or not

Returns

current context path

get_service(service_abstract_class_type: Type) → object[source]

Get the registered instance for the given abstract class type.

Parameters

service_abstract_class_type – base type

Returns

self

get_services() → object[source]

Get the registered instances in the services.

Returns

self

has_service(service_abstract_class_type: Type) → bool[source]

Return a bool indicating if the service has been registered.

Parameters

service_abstract_class_type – base type

Returns

if the service registered or not

load(path: str) → neuraxle.base.BaseTransformer[source]

Load full dump at the given path.

Parameters

path – pipeline step path

Returns

loaded step

mkdir()[source]

Creates the directory to save the last parent step.

Returns

peek() → neuraxle.base.BaseTransformer[source]

Get last parent.

Returns

the last parent base step

pop() → bool[source]

Pop the context. Returns True if it successfully popped an item from the parents list.

Returns

if an item has been popped

pop_item() → neuraxle.base.BaseTransformer[source]

Change the execution context to be the same as the latest parent context.

Returns

push(step: neuraxle.base.BaseTransformer) → neuraxle.base.ExecutionContext[source]

Pushes a step in the parents of the execution context.

Parameters

step – step to add to the execution context

Returns

self

register_service(service_abstract_class_type: Type, service_instance: object) → neuraxle.base.ExecutionContext[source]

Register base class instance inside the services.

Parameters
  • service_abstract_class_type – base type

  • service_instance – instance

Returns

self

save(full_dump=False)[source]

Save all unsaved steps in the parents of the execution context using save(). This method is called from a step checkpointer inside a Checkpoint.

Parameters

full_dump – save full pipeline dump to be able to load everything without source code (false by default).

Returns

See also

BaseStep, save()

save_last()[source]

Save only the last step in the execution context.

See also

save()

set_execution_phase(phase: neuraxle.base.ExecutionPhase) → neuraxle.base.ExecutionContext[source]

Set the instance’s execution phase to given phase.

Parameters

phase

Returns

set_logger(logger)[source]
set_service_locator(services: Dict[Type, object]) → neuraxle.base.ExecutionContext[source]

Register abstract class type instances.

Parameters

services – instance

Returns

self

should_save_last_step() → bool[source]

Returns True if the last step should be saved.

Returns

if the last step should be saved

to_identity() → neuraxle.base.ExecutionContext[source]

Create a fake execution context containing only identity steps. Create the parents by using the path of the current execution context.

Returns

fake identity execution context

class neuraxle.base.ExecutionMode[source]

An enumeration.

FIT = 'fit'[source]
FIT_OR_FIT_TRANSFORM = 'fit_or_fit_transform'[source]
FIT_OR_FIT_TRANSFORM_OR_TRANSFORM = 'fit_or_fit_transform_or_transform'[source]
FIT_TRANSFORM = 'fit_transform'[source]
INVERSE_TRANSFORM = 'inverse_transform'[source]
TRANSFORM = 'transform'[source]
class neuraxle.base.ExecutionPhase[source]

An enumeration.

PRETRAIN = 'pretraining'[source]
PROD = 'production'[source]
TEST = 'test'[source]
TRAIN = 'training'[source]
UNSPECIFIED = None[source]
VALIDATION = 'validation'[source]
class neuraxle.base.ForceHandleIdentity[source]
_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
class neuraxle.base.ForceHandleMixin(cache_folder=None)[source]

A step that automatically calls handle methods in the transform, fit, and fit_transform methods. A class which inherits from ForceHandleMixin can’t use BaseStep’s _fit_data_container, _fit_transform_data_container and _transform_data_container. They must be redefined; failure to do so will trigger an Exception on initialisation (and would create infinite loop if these checks were not there).

_encapsulate_data(data_inputs, expected_outputs=None, execution_mode=None) → Tuple[neuraxle.base.ExecutionContext, neuraxle.data_container.DataContainer][source]

Encapsulate data with DataContainer.

Parameters
  • data_inputs – data inputs

  • expected_outputs – expected outputs

  • execution_mode – execution mode

Returns

execution context, data container

_ensure_method_overriden(method_name, original_cls)[source]

Asserts that a given method of current instance overrides the default one defined in a given class. We assume that current instance inherits from the given class but do not test it (MixinForBaseTransformer test inheritance to BaseTransfromer already).

Parameters
  • method_name

  • original_cls

Returns

fit(data_inputs, expected_outputs=None) → neuraxle.base.HandleOnlyMixin[source]

Using handle_fit(), fit step with the given data inputs, and expected outputs.

Parameters

data_inputs – data inputs

Returns

fitted self

fit_transform(data_inputs, expected_outputs=None) → Tuple[neuraxle.base.HandleOnlyMixin, Iterable][source]

Using handle_fit_transform(), fit and transform step with the given data inputs, and expected outputs.

Parameters

data_inputs – data inputs

Returns

fitted self, outputs

transform(data_inputs) → Iterable[source]

Using handle_transform(), transform data inputs.

Parameters

data_inputs – data inputs

Returns

outputs

class neuraxle.base.ForceHandleOnlyMixin(cache_folder=None)[source]

A step that automatically calls handle methods in the transform, fit, and fit_transform methods.

It also requires the implementation of handler methods :
  • _transform_data_container

  • _fit_transform_data_container

  • _fit_data_container

class neuraxle.base.FullDumpLoader(name, stripped_saver=None)[source]

Identity step that can load the full dump of a pipeline step. Used by load().

Usage example:

saved_step = FullDumpLoader(
    name=path,
    stripped_saver=self.stripped_saver
).load(context_for_loading, True)
_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
load(context: neuraxle.base.ExecutionContext, full_dump=True) → neuraxle.base.BaseStep[source]

Load the full dump of a pipeline step.

Parameters
  • context – execution context

  • full_dump – load full dump or not (always true, inherited from BaseStep

Returns

loaded step

class neuraxle.base.GlobalServiceAssertionExecutorMixin[source]

Any step which inherit of this class will test globaly retrievable service assertion of itself and all its children on a will_process call.

_assert(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]

Calls _global_assert_has_services on GlobalyRetrievableServiceAssertionWrapper instances that are (recursively) children of this node. :param context: The ExecutionContext for which we test the presence of service.

class neuraxle.base.GlobalyRetrievableServiceAssertionWrapper(wrapped: neuraxle.base.BaseTransformer = None, service_assertions: List[Type] = None, savers: List[neuraxle.base.BaseSaver] = None)[source]

Is used to assert the presence of service at the start of the pipeline AND at execution time for a given step.

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
_global_assert_has_services(context: neuraxle.base.ExecutionContext) → neuraxle.hyperparams.space.RecursiveDict[source]

Intended to be used in a .apply(‘_global_assert_has_services’) call from the outside. Is used to test the presence of services at the root of the pipeline.

See also GlobalServiceAssertionExecutorMixin._apply_service_assertions :params context : the execution context

class neuraxle.base.HandleOnlyMixin[source]
A pipeline step that only requires the implementation of handler methods :
  • _transform_data_container

  • _fit_transform_data_container

  • _fit_data_container

If forbids only implementing fit or transform or fit_transform without the handles. So it forces the handles.

_fit_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.base.BaseTransformer[source]
_fit_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseTransformer', <class 'neuraxle.data_container.DataContainer'>)[source]
_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]
fit(data_inputs, expected_outputs=None) → neuraxle.base.HandleOnlyMixin[source]
fit_transform(data_inputs, expected_outputs=None) → neuraxle.base.HandleOnlyMixin[source]
transform(data_inputs) → neuraxle.base.HandleOnlyMixin[source]
class neuraxle.base.HashlibMd5Hasher[source]

Class to hash hyperparamters, and data input ids together using md5 algorithm from hashlib : https://docs.python.org/3/library/hashlib.html

The DataContainer class uses the hashed values for its current ids. BaseStep uses many BaseHasher objects to hash hyperparameters, and data inputs ids together after each transform.

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
hash(current_ids, hyperparameters, data_inputs: Any = None) → List[str][source]

Hash DataContainer.current_ids, data inputs, and hyperparameters together using hashlib.md5

Parameters
  • current_ids – current hashed ids (can be None if this function has not been called yet)

  • hyperparameters – step hyperparameters to hash with current ids

  • data_inputs – data inputs to hash current ids for

Returns

the new hashed current ids

single_hash(current_id: str, hyperparameters: neuraxle.hyperparams.space.HyperparameterSamples) → List[str][source]

Hash summary id, and hyperparameters together.

Parameters
  • current_id – current hashed id

  • hyperparameters – step hyperparameters to hash with current ids

Returns

the new hashed current id

class neuraxle.base.HashlibMd5ValueHasher[source]
_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
hash(current_ids, hyperparameters, data_inputs: Any = None) → List[str][source]

Hash DataContainer.current_ids, data inputs, and hyperparameters together using hashlib.md5

Parameters
  • current_ids – current hashed ids (can be None if this function has not been called yet)

  • hyperparameters – step hyperparameters to hash with current ids

  • data_inputs – data inputs to hash current ids for

Returns

the new hashed current ids

class neuraxle.base.Identity(savers=None, name=None)[source]

A pipeline step that has no effect at all but to return the same data without changes.

This can be useful to concatenate new features to existing features, such as what AddFeatures do.

Identity inherits from NonTransformableMixin and from NonFittableMixin which makes it a class that has no effect in the pipeline: it doesn’t require fitting, and at transform-time, it returns the same data it received.

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
class neuraxle.base.IdentityHandlerMethodsMixin(cache_folder=None)[source]

A step that has a default implementation for all handler methods.

It is useful for steps that only the following methods :
  • will_fit()

  • will_transform()

  • will_fit_transform()

  • will_process()

  • did_process()

  • did_fit()

  • did_transform()

  • did_fit_transform()

_fit_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseTransformer', <class 'neuraxle.data_container.DataContainer'>)[source]
_fit_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseTransformer', <class 'neuraxle.data_container.DataContainer'>)[source]
_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]
class neuraxle.base.JoblibStepSaver[source]

Saver that can save, or load a step with joblib.load, and joblib.dump.

This saver is a good default saver when the object is already stripped out of things that would make it unserializable.

It is the default stripped_saver for the ExecutionContext. The stripped saver is the first to load the step, and the last to save the step. The saver receives a stripped version of the step so that it can be saved by joblib.

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
_create_step_path(context, step)[source]

Create step path for the given context.

Parameters
  • context – execution context

  • step – step to save, or load

Returns

path

can_load(step: neuraxle.base.BaseTransformer, context: neuraxle.base.ExecutionContext) → bool[source]

Returns true if the given step has been saved with the given execution context.

Parameters
  • step – step that might have been saved

  • context – execution context

Returns

if we can load the step with the given context

load_step(step: neuraxle.base.BaseTransformer, context: neuraxle.base.ExecutionContext) → neuraxle.base.BaseTransformer[source]

Load stripped step.

Parameters
  • step – stripped step to load

  • context – execution context to load from

Returns

save_step(step: neuraxle.base.BaseTransformer, context: neuraxle.base.ExecutionContext) → neuraxle.base.BaseTransformer[source]

Saved step stripped out of things that would make it unserializable.

Parameters
  • step – stripped step to save

  • context – execution context to save from

Returns

class neuraxle.base.LocalServiceAssertionWrapper(wrapped: neuraxle.base.BaseTransformer = None, service_assertions: List[Type] = None, savers: List[neuraxle.base.BaseSaver] = None)[source]

Is used to assert the presence of service at execution time for a given step

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
_assert(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]

Assert self.local_service_assertions are present in the context.

do_assert_has_services(context: neuraxle.base.ExecutionContext)[source]

Assert that all the necessary services are provided in the execution context.

Parameters

context – The ExecutionContext for which we test the presence of service.

class neuraxle.base.MetaStep(wrapped: neuraxle.base.BaseTransformer = None, hyperparams: neuraxle.hyperparams.space.HyperparameterSamples = None, hyperparams_space: neuraxle.hyperparams.space.HyperparameterSpace = None, name: str = None, savers: List[neuraxle.base.BaseSaver] = None, hashers: List[neuraxle.base.BaseHasher] = None)[source]
_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
class neuraxle.base.MetaStepJoblibStepSaver[source]

Custom saver for meta step mixin.

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
load_step(step: neuraxle.base.MetaStep, context: neuraxle.base.ExecutionContext) → neuraxle.base.MetaStep[source]

Load MetaStepMixin.

  1. Loop through all of the sub steps savers, and only load the sub steps that have been saved.

  2. Refresh steps

Parameters
  • step – step to load

  • context – execution context

Returns

loaded truncable steps

save_step(step: neuraxle.base.MetaStep, context: neuraxle.base.ExecutionContext) → neuraxle.base.MetaStep[source]

Save MetaStepMixin.

  1. Save wrapped step.

  2. Strip wrapped step form the meta step mixin.

  3. Save meta step with wrapped step savers.

Parameters
  • step – meta step to save

  • context – execution context

Returns

class neuraxle.base.MetaStepMixin(wrapped: neuraxle.base.BaseTransformer = None, savers: List[neuraxle.base.BaseSaver] = None)[source]

A class to represent a step that wraps another step. It can be used for many things. For example, ForEachDataInput adds a loop before any calls to the wrapped step :

class ForEachDataInput(MetaStepMixin, BaseStep):
    def __init__(
        self,
        wrapped: BaseStep
    ):
        BaseStep.__init__(self)
        MetaStepMixin.__init__(self, wrapped)

    def fit(self, data_inputs, expected_outputs=None):
        if expected_outputs is None:
            expected_outputs = [None] * len(data_inputs)

        for di, eo in zip(data_inputs, expected_outputs):
            self.wrapped = self.wrapped.fit(di, eo)

        return self

    def transform(self, data_inputs):
        outputs = []
        for di in data_inputs:
            output = self.wrapped.transform(di)
            outputs.append(output)

    return outputs

    def fit_transform(self, data_inputs, expected_outputs=None):
        if expected_outputs is None:
            expected_outputs = [None] * len(data_inputs)

        outputs = []
        for di, eo in zip(data_inputs, expected_outputs):
            self.wrapped, output = self.wrapped.fit_transform(di, eo)
        outputs.append(output)

        return self, outputs
_fit_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
_fit_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
_inverse_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
fit(data_inputs, expected_outputs=None)[source]
fit_transform(data_inputs, expected_outputs=None)[source]
get_children() → List[neuraxle.base.BaseStep][source]

Get the list of all the childs for that step. _HasChildrenMixin calls this method to apply methods to all of the childs for that step.

Returns

list of child steps

get_step() → neuraxle.base.BaseStep[source]

Get wrapped step

Returns

self.wrapped

get_step_by_name(name)[source]
handle_fit_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
handle_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
inverse_transform(data_inputs)[source]
mutate(new_method='inverse_transform', method_to_assign_to='transform', warn=False) → neuraxle.base.BaseTransformer[source]

Mutate self, and self.wrapped. Please refer to mutate() for more information.

Parameters
  • new_method – the method to replace transform with, if there is no pending will_mutate_to call.

  • method_to_assign_to – the method to which the new method will be assigned to, if there is no pending will_mutate_to call.

  • warn – (verbose) wheter or not to warn about the inexistence of the method.

Returns

self, a copy of self, or even perhaps a new or different BaseStep object.

resume(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
set_step(step: neuraxle.base.BaseTransformer) → neuraxle.base.BaseStep[source]

Set wrapped step to the given step.

Parameters

step – new wrapped step

Returns

self

should_resume(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
transform(data_inputs)[source]
will_mutate_to(new_base_step: Optional[neuraxle.base.BaseTransformer] = None, new_method: str = None, method_to_assign_to: str = None) → neuraxle.base.BaseTransformer[source]

Add pending mutate self, self.wrapped. Please refer to will_mutate_to() for more information.

Parameters
  • new_base_step – if it is not None, upon calling mutate, the object it will mutate to will be this provided new_base_step.

  • method_to_assign_to – if it is not None, upon calling mutate, the method_to_affect will be the one that is used on the provided new_base_step.

  • new_method – if it is not None, upon calling mutate, the new_method will be the one that is used on the provided new_base_step.

Returns

self

class neuraxle.base.MixinForBaseTransformer[source]

Any steps/transformers within a pipeline that inherits of this class should implement BaseStep/BaseTransformer and initialize it before any mixin. This class checks that its the case at initialization.

_ensure_basetransformer_init_called()[source]

Assert that BaseTransformer’s init method has been called.

class neuraxle.base.NonFittableMixin[source]

A pipeline step that requires no fitting: fitting just returns self when called to do no action. Note: fit methods are not implemented

_fit_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
_fit_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
fit(data_inputs, expected_outputs=None) → neuraxle.base.NonFittableMixin[source]

Don’t fit.

Parameters
  • data_inputs – the data that would normally be fitted on.

  • expected_outputs – the data that would normally be fitted on.

Returns

self

class neuraxle.base.NonTransformableMixin[source]

A pipeline step that has no effect at all but to return the same data without changes. Transform method is automatically implemented as changing nothing.

Example :

class PrintOnFit(NonTransformableMixin, BaseStep):
    def __init__(self):
        BaseStep.__init__(self)

    def fit(self, data_inputs, expected_outputs=None) -> 'FitCallbackStep':
        print((data_inputs, expected_outputs))
        return self

Note

fit methods are not implemented

_fit_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Do nothing - return the same data.

Parameters
  • data_container – data container

  • context – execution context

Returns

data container

inverse_transform(processed_outputs)[source]

Do nothing - return the same data.

Parameters

processed_outputs – the data to process

Returns

the processed_outputs, unchanged.

transform(data_inputs)[source]

Do nothing - return the same data.

Parameters

data_inputs – the data to process

Returns

the data_inputs, unchanged.

class neuraxle.base.ResumableStepMixin[source]

Mixin to add resumable function to a step, or a class that can be resumed, for example a checkpoint on disk.

resume(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]
should_resume(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → bool[source]

Returns True if a step can be resumed with the given the data container and execution context. See Checkpoint class documentation for more details on how a resumable checkpoint works.

Parameters
  • data_container – data container to resume from

  • context – execution context to resume from

Returns

if we can resume

class neuraxle.base.StepWithContext(wrapped: neuraxle.base.BaseTransformer, context: neuraxle.base.ExecutionContext, raise_if_not_root: bool = True)[source]
_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
_will_process(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> (<class 'neuraxle.data_container.DataContainer'>, <class 'neuraxle.base.ExecutionContext'>)[source]

Inject the given context and test service assertions (if any are appliable) before processing the wrapped step.

Parameters

data_container – data container to process

Returns

data container, execution context

class neuraxle.base.TransformHandlerOnlyMixin[source]

A pipeline step that only requires the implementation of _transform_data_container.

_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Transform data container with the given execution context.

Parameters
  • data_container – data container

  • context – execution context

Returns

transformed data container

transform(data_inputs) → neuraxle.base.HandleOnlyMixin[source]
class neuraxle.base.TruncableJoblibStepSaver[source]

Step saver for a TruncableSteps. TruncableJoblibStepSaver saves, and loads all of the sub steps using their savers.

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
load_step(step: neuraxle.base.TruncableSteps, context: neuraxle.base.ExecutionContext) → neuraxle.base.TruncableSteps[source]
  1. Loop through all of the sub steps savers, and only load the sub steps that have been saved.

  2. Refresh steps

Parameters
  • step – step to load

  • context – execution context

Returns

loaded truncable steps

save_step(step: neuraxle.base.TruncableSteps, context: neuraxle.base.ExecutionContext)[source]
  1. Loop through all the steps, and save the ones that need to be saved.

  2. Add a new property called sub step savers inside truncable steps to be able to load sub steps when loading.

  3. Strip steps from truncable steps at the end.

Parameters
  • step – step to save

  • context – execution context

Returns

class neuraxle.base.TruncableSteps(steps_as_tuple: List[Union[Tuple[str, neuraxle.base.BaseTransformer], neuraxle.base.BaseTransformer]], hyperparams: neuraxle.hyperparams.space.HyperparameterSamples = {}, hyperparams_space: neuraxle.hyperparams.space.HyperparameterSpace = {}, mute_step_renaming_warning: bool = True)[source]

Step that contains multiple steps. Pipeline inherits form this class. It is possible to truncate this step * __getitem__()

  • self.steps contains the actual steps

  • self.steps_as_tuple contains a list of tuple of step name, and step

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
_load_saved_pipeline_steps_before_index(saved_pipeline: neuraxle.base.TruncableSteps, index: int)[source]

Load the cached pipeline steps before the index into the current steps

Parameters
  • saved_pipeline – saved pipeline

  • index – step index

Returns

_patch_missing_names(steps_as_tuple: List[Union[Tuple[str, neuraxle.base.BaseTransformer], neuraxle.base.BaseTransformer]]) → List[Union[Tuple[str, neuraxle.base.BaseTransformer], neuraxle.base.BaseTransformer]][source]

Make sure that each sub step has a unique name, and add a name to the sub steps that don’t have one already.

Parameters

steps_as_tuple – a NamedTupleList

Returns

a NamedTupleList with fixed names

_refresh_steps()[source]

Private method to refresh inner state after having edited self.steps_as_tuple (recreate self.steps from self.steps_as_tuple).

_rename_step(step_name, class_name, names_yet: set)[source]

Rename step by adding a number suffix after the class name. Ensure uniqueness with the names yet parameter.

Parameters
  • step_name – step name

  • class_name – class name

  • names_yet – names already taken

Returns

new step name

_step_index_to_name(step_index)[source]
_wrap_non_base_steps(steps_as_tuple: List) → List[Union[Tuple[str, neuraxle.base.BaseTransformer], neuraxle.base.BaseTransformer]][source]

If some steps are not of type BaseStep, we’ll try to make them of this type. For instance, sklearn objects will be wrapped by a SKLearnWrapper here.

Parameters

steps_as_tuple – a list of steps or of named tuples of steps (e.g.: NamedTupleList)

Returns

a NamedTupleList

append(item: Tuple[str, BaseTransformer]) → neuraxle.base.TruncableSteps[source]

Add an item to steps as tuple.

Parameters

item – item tuple (step name, step)

Returns

self

are_steps_before_index_the_same(other: neuraxle.base.TruncableSteps, index: int) → bool[source]

Returns true if self.steps before index are the same as other.steps before index.

Parameters
  • other – other truncable steps to compare

  • index – max step index to compare

Returns

bool

ends_with(step_type: type)[source]

Returns true if truncable steps end with a step of the given type.

Parameters

step_type – step type

Returns

if truncable steps ends with the given step type

get_children() → List[neuraxle.base.BaseStep][source]

Get the list of sub step inside the step with children.

Returns

children steps

get_step_by_name(name)[source]
items() → ItemsView[source]

Returns all of the steps as tuples items (step_name, step).

Returns

step items tuple : (step name, step)

keys() → KeysView[source]

Returns the step names.

Returns

list of step names

mutate(new_method='inverse_transform', method_to_assign_to='transform', warn=False) → neuraxle.base.BaseTransformer[source]

Call mutate on every steps the the present truncable step contains.

Parameters
  • new_method – the method to replace transform with.

  • method_to_assign_to – the method to which the new method will be assigned to.

  • warn – (verbose) wheter or not to warn about the inexistence of the method.

Returns

self, a copy of self, or even perhaps a new or different BaseStep object.

See also

reverse() inverse_transform()

pop() → neuraxle.base.BaseTransformer[source]

Pop the last step.

Returns

last step

popfront() → neuraxle.base.BaseTransformer[source]

Pop the first step.

Returns

first step

popfrontitem() → Tuple[str, neuraxle.base.BaseTransformer][source]

Pop the first step.

Returns

first step item

popitem(key=None) → Tuple[str, neuraxle.base.BaseTransformer][source]

Pop the last step, or the step with the given key

Parameters

key – step name to pop, or None

Returns

last step item

set_steps(steps_as_tuple: List[Union[Tuple[str, neuraxle.base.BaseTransformer], neuraxle.base.BaseTransformer]])[source]

Set steps as tuple.

Parameters

steps_as_tuple – list of tuple containing step name and step

Returns

should_save()[source]

Returns if the step needs to be saved or not.

Returns

If self or any of his sub steps should be saved, returns True.

split(step_type: type) → List[neuraxle.base.TruncableSteps][source]

Split truncable steps by a step class name.

Parameters

step_type – step class type to split from.

Returns

list of truncable steps containing the splitted steps

values() → ValuesView[source]

Get step values.

Returns

all of the steps

class neuraxle.base.WillProcessAssertionMixin[source]
_will_process(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> (<class 'neuraxle.data_container.DataContainer'>, <class 'neuraxle.base.ExecutionContext'>)[source]

Calls self._assert(data_container,context)

class neuraxle.base._CouldHaveContext[source]

Step that can have a context. It has “has service assertions” to ensure that the context has registered all the necessary services.

A context can be injected with the with_context method:

context = ExecutionContext(root=tmpdir)
service = SomeService()
context.set_service_locator({SomeBaseService: service})

p = Pipeline([
    SomeStep().assert_has_services(SomeBaseService)
]).with_context(context=context)

Or alternatively,

p = Pipeline([
    RegisterSomeService(),
    SomeStep().assert_has_services_at_execution(SomeBaseService)
])

Context services can be used inside any step with handler methods:

class SomeStep(ForceHandleMixin, Identity):
    def __init__(self):
        Identity.__init__(self)
        ForceHandleMixin.__init__(self)

    def _transform_data_container(self, data_container: DataContainer, context: ExecutionContext):
        service: SomeBaseService = context.get_service(SomeBaseService)
        service.service_method(data_container.data_inputs)
        return data_container
assert_has_services(*service_assertions) → neuraxle.base.GlobalyRetrievableServiceAssertionWrapper[source]

Set all service assertions to be made at the root of the pipeline and before processing the step.

Parameters

service_assertions – base types that need to be available in the execution context at the root of the pipeline

assert_has_services_at_execution(*service_assertions) → neuraxle.base.LocalServiceAssertionWrapper[source]

Set all service assertions to be made before processing the step.

Parameters

service_assertions – base types that need to be available in the execution context

with_context(context: neuraxle.base.ExecutionContext)[source]

An higher order step to inject a context inside a step. A step with a context forces the pipeline to use that context through handler methods. This is useful for dependency injection because you can register services inside the ExecutionContext. It also ensures that the context has registered all the necessary services.

context = ExecutionContext(root=tmpdir)
context.set_service_locator(ServiceLocator().services) # where services is of type Dict[Type, object]

p = WithContext(Pipeline([
    SomeStep().with_assertion_has_services(SomeBaseService)
]), context)
class neuraxle.base._CustomHandlerMethods[source]

A class to represent a step that needs to add special behavior on top of the normal handler methods. It allows the step to apply side effects before calling the real handler method.

Apply additional behavior (mini-batching, parallel processing, etc.) before calling the internal handler methods :
fit_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]

Custom fit data container method.

Parameters
  • data_container – data container to fit on

  • context – execution context

Returns

fitted self

fit_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]

Custom fit transform data container method.

Parameters
  • data_container – data container to fit on

  • context – execution context

Returns

fitted self, transformed data container

handle_fit(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.base.BaseStep[source]

Handle fit with a custom handler method for fitting the data container. The custom method to override is fit_data_container. The custom method fit_data_container replaces _fit_data_container.

Parameters
  • data_container – the data container to transform

  • context – execution context

Returns

tuple(fitted pipeline, data_container)

See also

DataContainer, ExecutionContext

handle_fit_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseStep', <class 'neuraxle.data_container.DataContainer'>)[source]

Handle fit_transform with a custom handler method for fitting, and transforming the data container. The custom method to override is fit_transform_data_container. The custom method fit_transform_data_container replaces _fit_transform_data_container().

Parameters
  • data_container – the data container to transform

  • context – execution context

Returns

tuple(fitted pipeline, data_container)

See also

DataContainer, ExecutionContext

handle_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Handle transform with a custom handler method for transforming the data container. The custom method to override is transform_data_container. The custom method transform_data_container replaces _transform_data_container().

Parameters
  • data_container – the data container to transform

  • context – execution context

Returns

transformed data container

See also

DataContainer, ExecutionContext

transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext)[source]

Custom transform data container method.

Parameters
  • data_container – data container to transform

  • context – execution context

Returns

transformed data container

class neuraxle.base._FittableStep[source]

An internal class to represent a step that can be fitted. See BaseStep, for a complete step that can be transformed, and fitted inside a neuraxle.pipeline.Pipeline.

_did_fit(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Apply side effects before fit is called.

Parameters
  • data_container – data container

  • context – execution context

Returns

(data container, execution context)

_did_fit_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Apply side effects after fit transform.

Parameters
  • data_container – data container

  • context – execution context

Returns

(fitted self, data container)

_fit_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.base._FittableStep[source]

Fit data container.

Parameters
  • data_container – data container

  • context – execution context

Returns

(fitted self, data container)

_fit_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseStep', <class 'neuraxle.data_container.DataContainer'>)[source]

Fit transform data container.

Parameters
  • data_container – data container

  • context – execution context

Returns

(fitted self, data container)

_will_fit(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> (<class 'neuraxle.data_container.DataContainer'>, <class 'neuraxle.base.ExecutionContext'>)[source]

Before fit is called, apply side effects on the step, the data container, or the execution context.

Parameters
  • data_container – data container

  • context – execution context

Returns

(data container, execution context)

_will_fit_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> (<class 'neuraxle.data_container.DataContainer'>, <class 'neuraxle.base.ExecutionContext'>)[source]

Apply side effects before fit_transform is called.

Parameters
  • data_container – data container

  • context – execution context

Returns

(data container, execution context)

fit(data_inputs, expected_outputs) → neuraxle.base._FittableStep[source]

Fit data inputs on the given expected outputs.

Parameters
  • data_inputs – data inputs

  • expected_outputs – expected outputs to fit on.

Returns

self

fit_transform(data_inputs, expected_outputs=None) -> ('BaseStep', typing.Any)[source]

Fit, and transform step with the given data inputs, and expected outputs.

Parameters
  • data_inputs – data inputs

  • expected_outputs – expected outputs to fit on

Returns

(fitted self, tranformed data inputs)

handle_fit(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.base.BaseStep[source]

Override this to add side effects or change the execution flow before (or after) calling fit(). The default behavior is to rehash current ids with the step hyperparameters.

Parameters
  • data_container – the data container to transform

  • context – execution context

Returns

tuple(fitted pipeline, data_container)

handle_fit_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseStep', <class 'neuraxle.data_container.DataContainer'>)[source]

Override this to add side effects or change the execution flow before (or after) calling * fit_transform(). The default behavior is to rehash current ids with the step hyperparameters.

Parameters
  • data_container – the data container to transform

  • context – execution context

Returns

tuple(fitted pipeline, data_container)

meta_fit(X_train, y_train, metastep: neuraxle.base.MetaStep)[source]

Uses a meta optimization technique (AutoML) to find the best hyperparameters in the given hyperparameter space.

Usage: p = p.meta_fit(X_train, y_train, metastep=RandomSearch(n_iter=10, scoring_function=r2_score, higher_score_is_better=True))

Call .mutate(new_method="inverse_transform", method_to_assign_to="transform"), and the current estimator will become

Parameters
  • X_train – data_inputs.

  • y_train – expected_outputs.

  • metastep – a metastep, that is, a step that can sift through the hyperparameter space of another estimator.

Returns

your best self.

class neuraxle.base._HasChildrenMixin[source]

Mixin to add behavior to the steps that have children (sub steps).

_apply_childrens(results: neuraxle.hyperparams.space.RecursiveDict, method: Union[str, Callable], ra: neuraxle.base._RecursiveArguments) → neuraxle.hyperparams.space.RecursiveDict[source]
_apply_self(method: Union[str, Callable], ra: neuraxle.base._RecursiveArguments)[source]
apply(method: Union[str, Callable], ra: neuraxle.base._RecursiveArguments = None, *args, **kwargs) → neuraxle.hyperparams.space.RecursiveDict[source]

Apply method to root, and children steps. Split the root, and children values inside the arguments of type RecursiveDict.

Parameters
  • method – str or callable function to apply

  • ra – recursive arguments

Returns

get_children() → List[neuraxle.base.BaseStep][source]

Get the list of all the childs for that step.

Returns

class neuraxle.base._HasHashers(hashers: List[neuraxle.base.BaseHasher] = None)[source]

An internal class to represent a step that has hashers. Most step rehash after every transformations to update the summary id, and the current ids inside the DataContainer. Hashers hash by hyperparameters, and source code.

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
hash(data_container: neuraxle.data_container.DataContainer) → List[str][source]

Hash data inputs, current ids, and hyperparameters together using self.hashers. This is used to create unique ids for the data checkpoints.

Parameters

data_container – data container

Returns

hashed current ids

See also

Checkpoint

hash_data_container(data_container)[source]

Hash data container using self.hashers.

  1. Hash current ids with hyperparams.

  2. Hash summary id with hyperparams.

Parameters

data_container – the data container to transform

Returns

transformed data container

set_hashers(hashers: List[neuraxle.base.BaseHasher]) → neuraxle.base._HasHashers[source]
summary_hash(data_container: neuraxle.data_container.DataContainer) → str[source]

Hash data inputs, current ids, and hyperparameters together using self.hashers. This is used to create unique ids for the data checkpoints.

Parameters

data_container – data container

Returns

hashed current ids

See also

Checkpoint

class neuraxle.base._HasHyperparams(hyperparams: neuraxle.hyperparams.space.HyperparameterSamples = None)[source]

An internal class to represent a step that has hyperparameters of type HyperparameterSamples. See BaseStep, for a complete step that can be transformed, and fitted inside a neuraxle.pipeline.Pipeline.

Every step has hyperparemeters, and hyperparameters spaces that can be set before the learning process begins. Hyperparameters can not only be passed in the constructor, but also be set by the pipeline that contains all of the steps :

pipeline = Pipeline([
    SomeStep()
])

pipeline.set_hyperparams(HyperparameterSamples({
    'learning_rate': 0.1,
    'SomeStep__learning_rate': 0.05
}))
_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
_get_hyperparams() → neuraxle.hyperparams.space.HyperparameterSamples[source]
_get_params() → neuraxle.hyperparams.space.HyperparameterSamples[source]
_set_hyperparams(hyperparams: Dict) → neuraxle.hyperparams.space.HyperparameterSamples[source]
_set_params(params: dict) → neuraxle.hyperparams.space.HyperparameterSamples[source]
_update_hyperparams(hyperparams: Dict) → neuraxle.hyperparams.space.HyperparameterSamples[source]
get_hyperparams() → neuraxle.hyperparams.space.HyperparameterSamples[source]

Get step hyperparameters as HyperparameterSamples.

Returns

step hyperparameters

Note

This is a recursive method that will call _get_hyperparams().

See also

HyperparameterSamples, :class:̀_HasChildrenMixin`, BaseStep.apply(), _HasChildrenMixin._apply(), _HasChildrenMixin._get_hyperparams()

get_params() → dict[source]

Get step hyperparameters as a flat primitive dict.

Example :

s.set_params(learning_rate=0.1)
hyperparams = s.get_params()
assert hyperparams == {"learning_rate": 0.1}
Returns

hyperparameters

See also

HyperparameterSamples :class:̀_HasChildrenMixin`, apply(), apply(), _get_params()

set_hyperparams(hyperparams: neuraxle.hyperparams.space.HyperparameterSamples) → neuraxle.base.BaseTransformer[source]

Set the step hyperparameters.

Example :

step.set_hyperparams(HyperparameterSamples({
    'learning_rate': 0.10
}))
Parameters

hyperparams – hyperparameters

Returns

self

Note

This is a recursive method that will call _set_hyperparams().

See also

HyperparameterSamples, :class:̀_HasChildrenMixin`, BaseStep.apply(), _HasChildrenMixin._apply(), _HasChildrenMixin._set_train()

set_params(**params) → neuraxle.base.BaseTransformer[source]

Set step hyperparameters with a dictionary.

Example :

s.set_params(learning_rate=0.1)
hyperparams = s.get_params()
assert hyperparams == {"learning_rate": 0.1}

:param arbitrary number of arguments for hyperparameters

Note

This is a recursive method that will call _set_params() in the end.

See also

HyperparameterSamples, :class:̀_HasChildrenMixin`, apply(), _apply(), _set_params()

update_hyperparams(hyperparams: neuraxle.hyperparams.space.HyperparameterSamples) → neuraxle.base.BaseTransformer[source]

Update the step hyperparameters without removing the already-set hyperparameters. This can be useful to add more hyperparameters to the existing ones without flushing the ones that were already set.

Example :

step.set_hyperparams(HyperparameterSamples({
    'learning_rate': 0.10
    'weight_decay': 0.001
}))

step.update_hyperparams(HyperparameterSamples({
    'learning_rate': 0.01
}))

assert step.get_hyperparams()['learning_rate'] == 0.01
assert step.get_hyperparams()['weight_decay'] == 0.001
Parameters

hyperparams – hyperparameters

Returns

self

See also

HyperparameterSamples, :class:̀_HasChildrenMixin`, BaseStep.apply(), _HasChildrenMixin._apply(), _HasChildrenMixin._update_hyperparams()

class neuraxle.base._HasHyperparamsSpace(hyperparams_space: neuraxle.hyperparams.space.HyperparameterSpace = None)[source]

An internal class to represent a step that has hyperparameter spaces of type HyperparameterSpace. See BaseStep, for a complete step that can be transformed, and fitted inside a neuraxle.pipeline.Pipeline.

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
_get_hyperparams_space() → neuraxle.hyperparams.space.HyperparameterSpace[source]
_set_hyperparams_space(hyperparams_space: Dict) → neuraxle.hyperparams.space.HyperparameterSpace[source]
_update_hyperparams_space(hyperparams_space: Dict) → neuraxle.hyperparams.space.HyperparameterSpace[source]
get_hyperparams_space() → neuraxle.hyperparams.space.HyperparameterSpace[source]

Get step hyperparameters space.

Example :

step.get_hyperparams_space()
Returns

step hyperparams space

Note

This is a recursive method that will call BaseStep._get_hyperparams_space() in the end.

set_hyperparams_space(hyperparams_space: neuraxle.hyperparams.space.HyperparameterSpace) → neuraxle.base.BaseTransformer[source]

Set step hyperparameters space.

Example :

step.set_hyperparams_space(HyperparameterSpace({
    'hp': RandInt(0, 10)
}))
Parameters

hyperparams_space – hyperparameters space

Returns

self

Note

This is a recursive method that will call BaseStep._set_hyperparams_space() in the end.

See also

HyperparameterSamples, HyperparameterSpace, HyperparameterDistribution :class:̀_HasChildrenMixin`, BaseStep.apply(), _HasChildrenMixin._apply(), _HasChildrenMixin._get_params()

update_hyperparams_space(hyperparams_space: neuraxle.hyperparams.space.HyperparameterSpace) → neuraxle.base.BaseTransformer[source]

Update the step hyperparameter spaces without removing the already-set hyperparameters. This can be useful to add more hyperparameter spaces to the existing ones without flushing the ones that were already set.

Example :

step.set_hyperparams_space(HyperparameterSpace({
    'learning_rate': LogNormal(0.5, 0.5)
    'weight_decay': LogNormal(0.001, 0.0005)
}))

step.update_hyperparams_space(HyperparameterSpace({
    'learning_rate': LogNormal(0.5, 0.1)
}))

assert step.get_hyperparams_space()['learning_rate'] == LogNormal(0.5, 0.1)
assert step.get_hyperparams_space()['weight_decay'] == LogNormal(0.001, 0.0005)
Parameters

hyperparams_space – hyperparameters space

Returns

self

Note

This is a recursive method that will call BaseStep._update_hyperparams_space() in the end.

See also

update_hyperparams(), HyperparameterSpace

class neuraxle.base._HasMutations[source]

An internal class to represent a step that can be mutated. A step can replace some of its method by others. For example, you might want to reverse a step, and replace the transform method by the inverse transform method.

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
mutate(new_method='inverse_transform', method_to_assign_to='transform', warn=True) → neuraxle.base.BaseStep[source]

Replace the “method_to_assign_to” method by the “new_method” method, IF the present object has no pending calls to .will_mutate_to() waiting to be applied. If there is a pending call, the pending call will override the methods specified in the present call. If the change fails (such as if the new_method doesn’t exist), then a warning is printed (optional). By default, there is no pending will_mutate_to call.

This could for example be useful within a pipeline to apply inverse_transform to every pipeline steps, or to assign predict_probas to predict, or to assign “inverse_transform” to “transform” to a reversed pipeline.

Parameters
  • new_method – the method to replace transform with, if there is no pending will_mutate_to call.

  • method_to_assign_to – the method to which the new method will be assigned to, if there is no pending will_mutate_to call.

  • warn – (verbose) wheter or not to warn about the inexistence of the method.

Returns

self, a copy of self, or even perhaps a new or different BaseStep object.

will_mutate_to(new_base_step: Optional[neuraxle.base.BaseTransformer] = None, new_method: str = None, method_to_assign_to: str = None) → neuraxle.base.BaseTransformer[source]

This will change the behavior of self.mutate(<...>) such that when mutating, it will return the presently provided new_base_step BaseStep (can be left to None for self), and the .mutate method will also apply the new_method and the method_to_affect, if they are not None, and after changing the object to new_base_step.

This can be useful if your pipeline requires unsupervised pretraining. For example:

X_pretrain = ...
X_train = ...

p = Pipeline(
    SomePreprocessing(),
    SomePretrainingStep().will_mutate_to(new_base_step=SomeStepThatWillUseThePretrainingStep),
    Identity().will_mutate_to(new_base_step=ClassifierThatWillBeUsedOnlyAfterThePretraining)
)
# Pre-train the pipeline
p = p.fit(X_pretrain, y=None)

# This will leave `SomePreprocessing()` untouched and will affect the two other steps.
p = p.mutate(new_method="transform", method_to_affect="transform")

# Pre-train the pipeline
p = p.fit(X_train, y_train)  # Then fit the classifier and other new things
Parameters
  • new_base_step – if it is not None, upon calling mutate, the object it will mutate to will be this provided new_base_step.

  • method_to_assign_to – if it is not None, upon calling mutate, the method_to_affect will be the one that is used on the provided new_base_step.

  • new_method – if it is not None, upon calling mutate, the new_method will be the one that is used on the provided new_base_step.

Returns

self

class neuraxle.base._HasRecursiveMethods[source]

An internal class to represent a step that has recursive methods. The apply apply() function is used to apply a method to a step and its children.

Example usage :

class _HasHyperparams:
    # ...
    def set_hyperparams(self, hyperparams: Union[HyperparameterSamples, Dict]) -> HyperparameterSamples:
        self.apply(method='_set_hyperparams', hyperparams=HyperparameterSamples(hyperparams))
        return self

    def _set_hyperparams(self, hyperparams: Union[HyperparameterSamples, Dict]) -> HyperparameterSamples:
        self._invalidate()
        hyperparams = HyperparameterSamples(hyperparams)
        self.hyperparams = hyperparams if len(hyperparams) > 0 else self.hyperparams
        return self.hyperparams

pipeline = Pipeline([
    SomeStep()
])

pipeline.set_hyperparams(HyperparameterSamples({
    'learning_rate': 0.1,
    'SomeStep__learning_rate': 0.05
}))
apply(method: Union[str, Callable], ra: neuraxle.base._RecursiveArguments = None, *args, **kwargs) → neuraxle.hyperparams.space.RecursiveDict[source]

Apply a method to a step and its children.

Parameters
  • method – method name that need to be called on all steps

  • ra – recursive arguments

  • args – any additional arguments to be passed to the method

  • kwargs – any additional positional arguments to be passed to the method

Returns

method outputs, or None if no method has been applied

class neuraxle.base._HasSavers(savers: List[neuraxle.base.BaseSaver] = None)[source]

An internal class to represent a step that can be saved. A step with savers is saved using its list of savers. Each saver saves some parts of the step.

A pipeline can save the step that need to be saved (see save()) can be saved :

step = Pipeline([
    Identity()
])
step.save()
step = step.load()

Or, it can also save a full dump that can be reloaded without any source code :

step = Identity().set_name('step_name')
step.save(full_dump=True)
step = ExecutionContext().load('step_name')
_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
_load_step(context, savers)[source]
add_saver(saver: neuraxle.base.BaseSaver) → neuraxle.base.BaseTransformer[source]

Add a step saver of a pipeline step.

Returns

self

See also

BaseSaver

get_savers() → List[neuraxle.base.BaseSaver][source]

Get the step savers of a pipeline step.

Returns

step savers

See also

BaseSaver

load(context: neuraxle.base.ExecutionContext, full_dump=False) → neuraxle.base.BaseTransformer[source]

Load step using the execution context to create the directory of the saved step. Warning:

Parameters
  • context – execution context to load step from

  • full_dump – save full dump bool

Returns

loaded step

Warning

Please do not override this method because on loading it is an identity step that will load whatever step you coded.

save(context: neuraxle.base.ExecutionContext, full_dump=False) → neuraxle.base.BaseTransformer[source]

Save step using the execution context to create the directory to save the step into. The saving happens by looping through all of the step savers in the reversed order.

Some savers just save parts of objects, some save it all or what remains. The ExecutionContext.stripped_saver has to be called last because it needs a stripped version of the step.

Parameters
  • context – context to save from

  • full_dump – save full pipeline dump to be able to load everything without source code (false by default).

Returns

self

set_savers(savers: List[neuraxle.base.BaseSaver]) → neuraxle.base.BaseTransformer[source]

Set the step savers of a pipeline step.

Returns

self

See also

BaseSaver

should_save() → bool[source]

Returns true if the step should be saved. If the step has been initialized and invalidated, then it must be saved.

A step is invalidated when any of the following things happen :

  • a mutation has been performed on the step mutate()

  • an hyperparameter has changed func:~neuraxle.base._HasHyperparams.set_hyperparams

  • an hyperparameter space has changed func:~neuraxle.base._HasHyperparamsSpace.set_hyperparams_space

  • a call to the fit method func:~neuraxle.base._FittableStep.handle_fit

  • a call to the fit_transform method func:~neuraxle.base._FittableStep.handle_fit_transform

  • the step name has changed func:~neuraxle.base.BaseStep.set_name

Returns

if the step should be saved

class neuraxle.base._RecursiveArguments(ra=None, *args, **kwargs)[source]

This class is used by apply(), and _HasChildrenMixin to pass the right arguments to steps with children.

class neuraxle.base._TransformerStep[source]

An internal class to represent a step that can be transformed, or inverse transformed. See BaseTransformer, for the complete transformer step that can be used inside a neuraxle.pipeline.Pipeline. See BaseStep, for a step that can also be fitted inside a neuraxle.pipeline.Pipeline.

Every step must implement transform(). If a step is not transformable, you can inherit from NonTransformableMixin.

Every transformer step has handle methods that can be overridden to add side effects or change the execution flow based on the execution context, and the data container :
_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
_did_process(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Apply side effects after any step method.

Parameters
  • data_container – data container

  • context – execution context

Returns

(data container, execution context)

_did_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Apply side effects after transform.

Parameters
  • data_container – data container

  • context – execution context

Returns

data container

_inverse_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]
_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Transform data container.

Parameters
  • data_container – data container

  • context – execution context

Returns

data container

_will_process(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> (<class 'neuraxle.data_container.DataContainer'>, <class 'neuraxle.base.ExecutionContext'>)[source]

Apply side effects before any step method. :param data_container: data container :param context: execution context :return: (data container, execution context)

_will_transform_data_container(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> (<class 'neuraxle.data_container.DataContainer'>, <class 'neuraxle.base.ExecutionContext'>)[source]

Apply side effects before transform.

Parameters
  • data_container – data container

  • context – execution context

Returns

(data container, execution context)

fit(data_inputs, expected_outputs) → neuraxle.base._TransformerStep[source]

Fit given data inputs. By default, a step only transforms in the fit transform method. To add fitting to your step, see class:_FittableStep for more info. :param data_inputs: data inputs :param expected_outputs: expected outputs to fit on :return: transformed data inputs

fit_transform(data_inputs, expected_outputs=None)[source]

Fit transform given data inputs. By default, a step only transforms in the fit transform method. To add fitting to your step, see class:_FittableStep for more info.

Parameters
  • data_inputs – data inputs

  • expected_outputs – expected outputs to fit on

Returns

transformed data inputs

handle_fit(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.base.BaseTransformer[source]

Override this to add side effects or change the execution flow before (or after) calling fit(). The default behavior is to rehash current ids with the step hyperparameters.

Parameters
  • data_container – the data container to transform

  • context – execution context

Returns

tuple(fitted pipeline, data_container)

handle_fit_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) -> ('BaseTransformer', <class 'neuraxle.data_container.DataContainer'>)[source]

Override this to add side effects or change the execution flow before (or after) calling * fit_transform(). The default behavior is to rehash current ids with the step hyperparameters.

Parameters
  • data_container – the data container to transform

  • context – execution context

Returns

tuple(fitted pipeline, data_container)

handle_inverse_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Override this to add side effects or change the execution flow before (or after) calling inverse_transform(). The default behavior is to rehash current ids with the step hyperparameters.

Parameters
  • data_container – the data container to inverse transform

  • context – execution context

Returns

data_container

handle_predict(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Handle_transform in test mode.

Parameters
  • data_container – the data container to transform

  • context – execution context

Returns

transformed data container

handle_transform(data_container: neuraxle.data_container.DataContainer, context: neuraxle.base.ExecutionContext) → neuraxle.data_container.DataContainer[source]

Override this to add side effects or change the execution flow before (or after) calling * transform(). The default behavior is to rehash current ids with the step hyperparameters.

Parameters
  • data_container – the data container to transform

  • context – execution context

Returns

transformed data container

inverse_transform(processed_outputs)[source]

Inverse Transform the given transformed data inputs.

mutate() or reverse() can be called to change the default transform behavior :

p = Pipeline([MultiplyBy()])
_in = np.array([1, 2])
_out = p.transform(_in)
_regenerated_in = reversed(p).transform(_out)
assert np.array_equal(_regenerated_in, _in)
Parameters

processed_outputs – processed data inputs

Returns

inverse transformed processed outputs

predict(data_input)[source]

Predict the expected output in test mode using func:~neuraxle.base._TransformerStep.transform, but by setting self to test mode first and then reverting the mode.

Parameters

data_input – data input to predict

Returns

prediction

teardown() → neuraxle.base.BaseTransformer[source]

Applies _teardown on the step and, if applicable, its children. :return: self

transform(data_inputs)[source]

Transform given data inputs.

Parameters

data_inputs – data inputs

Returns

transformed data inputs

class neuraxle.base._WithContextStepSaver[source]

Custom saver for steps that have an ExecutionContext. Loading will inject the saved dependencies inside the :class`ExecutionContext`.

See also

_HasContext, BaseSaver, ExecutionContext

_abc_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache = <_weakrefset.WeakSet object>[source]
_abc_negative_cache_version = 60
_abc_registry = <_weakrefset.WeakSet object>[source]
can_load(step: neuraxle.base.StepWithContext, context: neuraxle.base.ExecutionContext)[source]

Returns true if we can load the given step with the given execution context.

Parameters
  • step – step to load

  • context – execution context to load from

Returns

load_step(step: neuraxle.base.StepWithContext, context: neuraxle.base.ExecutionContext) → neuraxle.base.StepWithContext[source]

Load a step with a context by setting the context as the loading context.

Parameters
  • step – step with context

  • context – execution context to load from

Returns

loaded step with context

save_step(step: neuraxle.base.StepWithContext, context: neuraxle.base.ExecutionContext) → neuraxle.base.StepWithContext[source]

If needed, remove parents of a step with context before saving.

Parameters
  • step – step with context

  • context – execution context to load from

Returns

saved step with context

neuraxle.base._sklearn_to_neuraxle_step(step) → neuraxle.base.BaseTransformer[source]