neuraxle.metaopt.callbacks

Module-level documentation for neuraxle.metaopt.callbacks. Here is an inheritance diagram, including dependencies to other base modules of Neuraxle:

Inheritance diagram of neuraxle.metaopt.callbacks

Neuraxle’s training callbacks classes.

Training callback classes.

Classes

BaseCallback

Base class for a training callback.

BestModelCheckpoint()

Saves the pipeline model in a folder named “best” when the a new best validation score is reached.

CallbackList(callbacks)

Callback list that be executed.

EarlyStoppingCallback(…[, metric_name])

Perform early stopping when there is multiple epochs in a row that didn’t improve the performance of the model.

IfBestScore(wrapped_callback)

Meta callback that only execute when the trial is a new best score.

IfLastStep(wrapped_callback)

Meta callback that only execute when the training is finished or fitted, or when it is the last epoch.

MetaCallback(wrapped_callback)

Meta callback wraps another callback.

MetricCallback(name, metric_function, DIT, …)

Callback that calculates metric results.

ScoringCallback(metric_function, DIT, …[, …])

Metric Callback that calculates metric results for the main scoring metric.

StepSaverCallback(label)

Callback that saves the trial model.

Examples using neuraxle.metaopt.callbacks.MetricCallback

Examples using neuraxle.metaopt.callbacks.ScoringCallback


class neuraxle.metaopt.callbacks.BaseCallback[source]

Bases: abc.ABC

Base class for a training callback. Callbacks are called after each epoch inside the fit function of the Trainer.

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool[source]
_abc_impl = <_abc_data object>
class neuraxle.metaopt.callbacks.EarlyStoppingCallback(max_epochs_without_improvement, metric_name=None)[source]

Bases: neuraxle.metaopt.callbacks.BaseCallback

Perform early stopping when there is multiple epochs in a row that didn’t improve the performance of the model.

__init__(max_epochs_without_improvement, metric_name=None)[source]
Parameters
  • max_epochs_without_improvement – The number of step without improvement on the validation score before an early stopping is triggered.

  • metric_name – The name of the metric on which we want to condition the early stopping. If None, the main metric will be used.

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool[source]
_abc_impl = <_abc_data object>
class neuraxle.metaopt.callbacks.MetaCallback(wrapped_callback: neuraxle.metaopt.callbacks.BaseCallback)[source]

Bases: neuraxle.metaopt.callbacks.BaseCallback

Meta callback wraps another callback. It can be useful to test conditions before executing certain callbacks.

__init__(wrapped_callback: neuraxle.metaopt.callbacks.BaseCallback)[source]

Initialize self. See help(type(self)) for accurate signature.

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool[source]
_abc_impl = <_abc_data object>
class neuraxle.metaopt.callbacks.IfBestScore(wrapped_callback: neuraxle.metaopt.callbacks.BaseCallback)[source]

Bases: neuraxle.metaopt.callbacks.MetaCallback

Meta callback that only execute when the trial is a new best score.

See also

MetaCallback,

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool[source]
_abc_impl = <_abc_data object>
class neuraxle.metaopt.callbacks.IfLastStep(wrapped_callback: neuraxle.metaopt.callbacks.BaseCallback)[source]

Bases: neuraxle.metaopt.callbacks.MetaCallback

Meta callback that only execute when the training is finished or fitted, or when it is the last epoch.

See also

MetaCallback,

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool[source]
_abc_impl = <_abc_data object>
class neuraxle.metaopt.callbacks.StepSaverCallback(label)[source]

Bases: neuraxle.metaopt.callbacks.BaseCallback

Callback that saves the trial model.

__init__(label)[source]

Initialize self. See help(type(self)) for accurate signature.

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool[source]
_abc_impl = <_abc_data object>
class neuraxle.metaopt.callbacks.BestModelCheckpoint[source]

Bases: neuraxle.metaopt.callbacks.IfBestScore

Saves the pipeline model in a folder named “best” when the a new best validation score is reached. It is important to note that when refit=True, an AutoML loop will overwrite the best model after refitting.

__init__()[source]

Initialize self. See help(type(self)) for accurate signature.

_abc_impl = <_abc_data object>
class neuraxle.metaopt.callbacks.CallbackList(callbacks: List[neuraxle.metaopt.callbacks.BaseCallback])[source]

Bases: neuraxle.metaopt.callbacks.BaseCallback

Callback list that be executed.

__init__(callbacks: List[neuraxle.metaopt.callbacks.BaseCallback])[source]

Initialize self. See help(type(self)) for accurate signature.

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool[source]
append(callback: neuraxle.metaopt.callbacks.BaseCallback) → neuraxle.metaopt.callbacks.CallbackList[source]
_abc_impl = <_abc_data object>
class neuraxle.metaopt.callbacks.MetricCallback(name: str, metric_function: Callable[[EOT, DIT, Optional[neuraxle.base.ExecutionContext]], float], higher_score_is_better: bool, log_metrics=True, pass_context_to_metric_function: bool = False)[source]

Bases: neuraxle.metaopt.callbacks.BaseCallback

Callback that calculates metric results. Adds the results into the trial repository.

__init__(name: str, metric_function: Callable[[EOT, DIT, Optional[neuraxle.base.ExecutionContext]], float], higher_score_is_better: bool, log_metrics=True, pass_context_to_metric_function: bool = False)[source]

Initialize self. See help(type(self)) for accurate signature.

call(trial_split: neuraxle.metaopt.data.aggregates.TrialSplit, dact_train: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], dact_valid: neuraxle.data_container.DataContainer[~IDT, ~DIT, ~EOT][IDT, DIT, EOT], is_finished_and_fitted: bool = False) → bool[source]
_abc_impl = <_abc_data object>
class neuraxle.metaopt.callbacks.ScoringCallback(metric_function: Callable[[EOT, DIT, Optional[neuraxle.base.ExecutionContext]], float], name='main', higher_score_is_better: bool = True, log_metrics: bool = True, pass_context_to_metric_function: bool = False)[source]

Bases: neuraxle.metaopt.callbacks.MetricCallback

Metric Callback that calculates metric results for the main scoring metric. Adds the results into the trial repository.

See also

MetricCallback,

__init__(metric_function: Callable[[EOT, DIT, Optional[neuraxle.base.ExecutionContext]], float], name='main', higher_score_is_better: bool = True, log_metrics: bool = True, pass_context_to_metric_function: bool = False)[source]

Initialize self. See help(type(self)) for accurate signature.

_abc_impl = <_abc_data object>