neuraxle.metaopt.callbacks

Module-level documentation for neuraxle.metaopt.callbacks. Here is an inheritance diagram, including dependencies to other base modules of Neuraxle:

Neuraxle’s training callbacks classes.

Training callback classes.

Classes

BaseCallback	Base class for a training callback.
BestModelCheckpoint()	Saves the pipeline model in a folder named “best” when the a new best validation score is reached.
CallbackList(callbacks)	Callback list that be executed.
EarlyStoppingCallback(…[, metric_name])	Perform early stopping when there is multiple epochs in a row that didn’t improve the performance of the model.
IfBestScore(wrapped_callback)	Meta callback that only execute when the trial is a new best score.
IfLastStep(wrapped_callback)	Meta callback that only execute when the training is finished or fitted, or when it is the last epoch.
MetaCallback(wrapped_callback)	Meta callback wraps another callback.
MetricCallback(name, metric_function, DIT, …)	Callback that calculates metric results.
ScoringCallback(metric_function, DIT, …[, …])	Metric Callback that calculates metric results for the main scoring metric.
StepSaverCallback(label)	Callback that saves the trial model.

Examples using neuraxle.metaopt.callbacks.MetricCallback

Boston Housing Regression with Meta Optimization

Examples using neuraxle.metaopt.callbacks.ScoringCallback

Usage of AutoML loop, and hyperparams with sklearn models.

class neuraxle.metaopt.callbacks.BaseCallback

Bases: abc.ABC

Base class for a training callback. Callbacks are called after each epoch inside the fit function of the Trainer.

See also

MetaCallback, AutoML, Trainer,

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool

_abc_impl = <_abc_data object>

class neuraxle.metaopt.callbacks.EarlyStoppingCallback(max_epochs_without_improvement, metric_name=None)

Bases: neuraxle.metaopt.callbacks.BaseCallback

Perform early stopping when there is multiple epochs in a row that didn’t improve the performance of the model.

__init__(max_epochs_without_improvement, metric_name=None)

Parameters

• max_epochs_without_improvement – The number of step without improvement on the validation score before an early stopping is triggered.

• metric_name – The name of the metric on which we want to condition the early stopping. If None, the main metric will be used.

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool

_abc_impl = <_abc_data object>

class neuraxle.metaopt.callbacks.MetaCallback(wrapped_callback: neuraxle.metaopt.callbacks.BaseCallback)

Bases: neuraxle.metaopt.callbacks.BaseCallback

Meta callback wraps another callback. It can be useful to test conditions before executing certain callbacks.

See also

BaseCallback, IfBestScore, IfLastStep, StepSaverCallback,

__init__(wrapped_callback: neuraxle.metaopt.callbacks.BaseCallback)

Initialize self. See help(type(self)) for accurate signature.

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool

_abc_impl = <_abc_data object>

class neuraxle.metaopt.callbacks.IfBestScore(wrapped_callback: neuraxle.metaopt.callbacks.BaseCallback)

Bases: neuraxle.metaopt.callbacks.MetaCallback

Meta callback that only execute when the trial is a new best score.

See also

MetaCallback,

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool

_abc_impl = <_abc_data object>

class neuraxle.metaopt.callbacks.IfLastStep(wrapped_callback: neuraxle.metaopt.callbacks.BaseCallback)

Bases: neuraxle.metaopt.callbacks.MetaCallback

Meta callback that only execute when the training is finished or fitted, or when it is the last epoch.

See also

MetaCallback,

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool

_abc_impl = <_abc_data object>

class neuraxle.metaopt.callbacks.StepSaverCallback(label)

Bases: neuraxle.metaopt.callbacks.BaseCallback

Callback that saves the trial model.

See also

BaseCallback, AutoML, Trainer,

__init__(label)

Initialize self. See help(type(self)) for accurate signature.

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool

_abc_impl = <_abc_data object>

class neuraxle.metaopt.callbacks.BestModelCheckpoint

Bases: neuraxle.metaopt.callbacks.IfBestScore

Saves the pipeline model in a folder named “best” when the a new best validation score is reached. It is important to note that when refit=True, an AutoML loop will overwrite the best model after refitting.

__init__()

Initialize self. See help(type(self)) for accurate signature.

_abc_impl = <_abc_data object>

class neuraxle.metaopt.callbacks.CallbackList(callbacks: List[neuraxle.metaopt.callbacks.BaseCallback])

Bases: neuraxle.metaopt.callbacks.BaseCallback

Callback list that be executed.

See also

AutoML, Trainer, BaseCallback,

__init__(callbacks: List[neuraxle.metaopt.callbacks.BaseCallback])

Initialize self. See help(type(self)) for accurate signature.

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool

append(callback: neuraxle.metaopt.callbacks.BaseCallback) → neuraxle.metaopt.callbacks.CallbackList

_abc_impl = <_abc_data object>

class neuraxle.metaopt.callbacks.MetricCallback(name: str, metric_function: Callable[[EOT, DIT, Optional[neuraxle.base.ExecutionContext]], float], higher_score_is_better: bool, log_metrics=True, pass_context_to_metric_function: bool = False)

Bases: neuraxle.metaopt.callbacks.BaseCallback

Callback that calculates metric results. Adds the results into the trial repository.

See also

BaseCallback, MetaCallback, EarlyStoppingCallback, IfBestScore, IfLastStep, StepSaverCallback, CallbackList,

__init__(name: str, metric_function: Callable[[EOT, DIT, Optional[neuraxle.base.ExecutionContext]], float], higher_score_is_better: bool, log_metrics=True, pass_context_to_metric_function: bool = False)

Initialize self. See help(type(self)) for accurate signature.

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool

_abc_impl = <_abc_data object>

class neuraxle.metaopt.callbacks.ScoringCallback(metric_function: Callable[[EOT, DIT, Optional[neuraxle.base.ExecutionContext]], float], name='main', higher_score_is_better: bool = True, log_metrics: bool = True, pass_context_to_metric_function: bool = False)

Bases: neuraxle.metaopt.callbacks.MetricCallback

Metric Callback that calculates metric results for the main scoring metric. Adds the results into the trial repository.

See also

MetricCallback,

__init__(metric_function: Callable[[EOT, DIT, Optional[neuraxle.base.ExecutionContext]], float], name='main', higher_score_is_better: bool = True, log_metrics: bool = True, pass_context_to_metric_function: bool = False)

Initialize self. See help(type(self)) for accurate signature.

_abc_impl = <_abc_data object>