neuraxle.metaopt.data.aggregates

Module-level documentation for neuraxle.metaopt.data.aggregates. Here is an inheritance diagram, including dependencies to other base modules of Neuraxle:

Inheritance diagram of neuraxle.metaopt.data.aggregates

Neuraxle’s AutoML Scope Manager Classes

Scope manager objects used by AutoML algorithm classes.

Although the name “manager” for a class is disliked by the Neuraxle community, it is used as these are Pythonic context managers, as described in Python’s contextlib documentation.

Classes are splitted like this for the AutoML: - Projects - Clients - Rounds (runs) - Trials - TrialSplits - MetricResults

Classes

BaseAggregate(_dataclass, context[, is_deep])

Base class for aggregated objects using the repo and the dataclasses to manipulate them.

Client(_dataclass, context[, is_deep])

MetricResults(_dataclass, context[, is_deep])

Project(_dataclass, context[, is_deep])

Root(_dataclass, context[, is_deep])

Round(_dataclass, context[, is_deep])

Trial(_dataclass, context[, is_deep])

This class is a sub-contextualization of the HyperparameterRepository class for holding a trial and manipulating it within its context.

TrialSplit(_dataclass, context[, is_deep])

One split of a trial.


neuraxle.metaopt.data.aggregates._with_method_as_context_manager(func: Callable[[BaseAggregate], BaseAggregate]) → Callable[[neuraxle.metaopt.data.aggregates.BaseAggregate], AbstractContextManager[SubAggregateT]][source]

This is a method to be used as a context manager. This will sync items with the repos. Example:

See also

neuraxle.metaopt.data.aggregates.BaseAggregate neuraxle.metaopt.data.aggregates.BaseAggregate.__enter__() neuraxle.metaopt.data.aggregates.BaseAggregate.__exit__() neuraxle.metaopt.auto_ml.Trainer neuraxle.metaopt.auto_ml.Trainer.train()

class neuraxle.metaopt.data.aggregates.BaseAggregate(_dataclass: SubDataclassT, context: neuraxle.metaopt.data.vanilla.AutoMLContext, is_deep=False, parent: ParentAggregateT = None)[source]

Bases: neuraxle.base._CouldHaveContext, neuraxle.base.BaseService, contextlib.AbstractContextManager, typing.Generic

Base class for aggregated objects using the repo and the dataclasses to manipulate them.

For more information, read this article by Martin Fowler on DDD Aggregates.

__init__(_dataclass: SubDataclassT, context: neuraxle.metaopt.data.vanilla.AutoMLContext, is_deep=False, parent: ParentAggregateT = None)[source]

Initialize self. See help(type(self)) for accurate signature.

static from_context(context: neuraxle.metaopt.data.vanilla.AutoMLContext, is_deep=True) → neuraxle.metaopt.data.aggregates.BaseAggregate[source]

Return an aggregate from a context. Requirement: Have already pre-push the attr of dataclass into the context before calling this.

classmethod dummy(context: neuraxle.base.ExecutionContext = None) → neuraxle.metaopt.data.aggregates.BaseAggregate[source]

Create a dummy object of the desired type for testing purposes. The parent subtree will be created in a temporary repository, or in a real repository if the context is provided.

_invariant()[source]
without_context() → neuraxle.metaopt.data.aggregates.BaseAggregate[source]

Return a copy of this aggregate without the context. Useful for initializing a temporary aggregate, such as a filtered or reduced aggregate without all its subaggregates to disallow saving the reduced aggregate.

flow
repo
subaggregate(_dataclass: SubDataclassT, context: neuraxle.metaopt.data.vanilla.AutoMLContext, is_deep=False, parent: ParentAggregateT = None) → SubAggregateT[source]
parent
parentaggregate
dataclass
subdataclass
sanitize_metric_name(metric_name: str = None)[source]

If the argument metric is None, the optimizer’s metric is taken. If the optimizer’s metric is None, the parent metric is taken.

refresh(deep: bool = True)[source]
save(deep: bool = True) → neuraxle.metaopt.data.aggregates.BaseAggregate[source]
save_subaggregate(subagg: SubAggregateT, deep=False) → neuraxle.metaopt.data.aggregates.BaseAggregate[source]
managed_subresource(*args, **kwds) → SubAggregateT[source]

This is a method to be used as a context manager. This will sync items with the repos. Example:

See also

neuraxle.metaopt.data.aggregates.BaseAggregate neuraxle.metaopt.data.aggregates.BaseAggregate.__enter__() neuraxle.metaopt.data.aggregates.BaseAggregate.__exit__() neuraxle.metaopt.auto_ml.Trainer neuraxle.metaopt.auto_ml.Trainer.train()

_managed_subresource(*args, **kwds) → AbstractContextManager[SubAggregateT][source]
_acquire_managed_subresource(*args, **kwds) → SubAggregateT[source]

Acquire a new subaggregate that is managed such that it is deep saved at the beginning.

_release_managed_subresource(resource: SubAggregateT, e: Exception = None) → bool[source]

Release a subaggregate that was acquired with managed_subresource. The subaggregate normally has already saved itself. We may update things again here if needed.

Exceptions may be handled here. If handled, return True, if not, then return False.

_abc_impl = <_abc_data object>
class neuraxle.metaopt.data.aggregates.Root(_dataclass: SubDataclassT, context: neuraxle.metaopt.data.vanilla.AutoMLContext, is_deep=False, parent: ParentAggregateT = None)[source]

Bases: neuraxle.metaopt.data.aggregates.BaseAggregate

save(deep: bool = False)[source]
projects
static from_repo(context: neuraxle.base.ExecutionContext, repo: neuraxle.metaopt.data.vanilla.HyperparamsRepository) → neuraxle.metaopt.data.aggregates.Root[source]
static vanilla(context: neuraxle.base.ExecutionContext = None) → neuraxle.metaopt.data.aggregates.Root[source]
get_project(name: str) → neuraxle.metaopt.data.aggregates.Root[source]

This is a method to be used as a context manager. This will sync items with the repos. Example:

See also

neuraxle.metaopt.data.aggregates.BaseAggregate neuraxle.metaopt.data.aggregates.BaseAggregate.__enter__() neuraxle.metaopt.data.aggregates.BaseAggregate.__exit__() neuraxle.metaopt.auto_ml.Trainer neuraxle.metaopt.auto_ml.Trainer.train()

default_project() → neuraxle.metaopt.data.aggregates.Root[source]

This is a method to be used as a context manager. This will sync items with the repos. Example:

See also

neuraxle.metaopt.data.aggregates.BaseAggregate neuraxle.metaopt.data.aggregates.BaseAggregate.__enter__() neuraxle.metaopt.data.aggregates.BaseAggregate.__exit__() neuraxle.metaopt.auto_ml.Trainer neuraxle.metaopt.auto_ml.Trainer.train()

_acquire_managed_subresource(project_name: str) → neuraxle.metaopt.data.aggregates.Project[source]

Acquire a new subaggregate that is managed such that it is deep saved at the beginning.

_abc_impl = <_abc_data object>
class neuraxle.metaopt.data.aggregates.Project(_dataclass: SubDataclassT, context: neuraxle.metaopt.data.vanilla.AutoMLContext, is_deep=False, parent: ParentAggregateT = None)[source]

Bases: neuraxle.metaopt.data.aggregates.BaseAggregate

get_client(name: str) → neuraxle.metaopt.data.aggregates.Project[source]

This is a method to be used as a context manager. This will sync items with the repos. Example:

See also

neuraxle.metaopt.data.aggregates.BaseAggregate neuraxle.metaopt.data.aggregates.BaseAggregate.__enter__() neuraxle.metaopt.data.aggregates.BaseAggregate.__exit__() neuraxle.metaopt.auto_ml.Trainer neuraxle.metaopt.auto_ml.Trainer.train()

default_client() → neuraxle.metaopt.data.aggregates.Project[source]

This is a method to be used as a context manager. This will sync items with the repos. Example:

See also

neuraxle.metaopt.data.aggregates.BaseAggregate neuraxle.metaopt.data.aggregates.BaseAggregate.__enter__() neuraxle.metaopt.data.aggregates.BaseAggregate.__exit__() neuraxle.metaopt.auto_ml.Trainer neuraxle.metaopt.auto_ml.Trainer.train()

_acquire_managed_subresource(client_name: str) → neuraxle.metaopt.data.aggregates.Client[source]

Acquire a new subaggregate that is managed such that it is deep saved at the beginning.

_abc_impl = <_abc_data object>
class neuraxle.metaopt.data.aggregates.Client(_dataclass: SubDataclassT, context: neuraxle.metaopt.data.vanilla.AutoMLContext, is_deep=False, parent: ParentAggregateT = None)[source]

Bases: neuraxle.metaopt.data.aggregates.BaseAggregate

new_round(main_metric_name: str) → neuraxle.metaopt.data.aggregates.Client[source]

This is a method to be used as a context manager. This will sync items with the repos. Example:

See also

neuraxle.metaopt.data.aggregates.BaseAggregate neuraxle.metaopt.data.aggregates.BaseAggregate.__enter__() neuraxle.metaopt.data.aggregates.BaseAggregate.__exit__() neuraxle.metaopt.auto_ml.Trainer neuraxle.metaopt.auto_ml.Trainer.train()

resume_last_round() → neuraxle.metaopt.data.aggregates.Client[source]

This is a method to be used as a context manager. This will sync items with the repos. Example:

See also

neuraxle.metaopt.data.aggregates.BaseAggregate neuraxle.metaopt.data.aggregates.BaseAggregate.__enter__() neuraxle.metaopt.data.aggregates.BaseAggregate.__exit__() neuraxle.metaopt.auto_ml.Trainer neuraxle.metaopt.auto_ml.Trainer.train()

optim_round(new_round: bool, main_metric_name: str) → neuraxle.metaopt.data.aggregates.Client[source]

This is a method to be used as a context manager. This will sync items with the repos. Example:

See also

neuraxle.metaopt.data.aggregates.BaseAggregate neuraxle.metaopt.data.aggregates.BaseAggregate.__enter__() neuraxle.metaopt.data.aggregates.BaseAggregate.__exit__() neuraxle.metaopt.auto_ml.Trainer neuraxle.metaopt.auto_ml.Trainer.train()

_acquire_managed_subresource(new_round: bool = True, main_metric_name: str = None) → neuraxle.metaopt.data.aggregates.Round[source]

Acquire a new subaggregate that is managed such that it is deep saved at the beginning.

_abc_impl = <_abc_data object>
class neuraxle.metaopt.data.aggregates.Round(_dataclass: SubDataclassT, context: neuraxle.metaopt.data.vanilla.AutoMLContext, is_deep=False, parent: ParentAggregateT = None)[source]

Bases: neuraxle.metaopt.data.aggregates.BaseAggregate

with_optimizer(hp_optimizer: neuraxle.metaopt.data.vanilla.BaseHyperparameterOptimizer, hp_space: neuraxle.hyperparams.space.HyperparameterSpace) → neuraxle.metaopt.data.aggregates.Round[source]
with_metric(metric_name: str) → neuraxle.metaopt.data.aggregates.Round[source]
main_metric_name
new_rvs_trial(continue_on_error: bool = False) → neuraxle.metaopt.data.aggregates.Round[source]

This is a method to be used as a context manager. This will sync items with the repos. Example:

See also

neuraxle.metaopt.data.aggregates.BaseAggregate neuraxle.metaopt.data.aggregates.BaseAggregate.__enter__() neuraxle.metaopt.data.aggregates.BaseAggregate.__exit__() neuraxle.metaopt.auto_ml.Trainer neuraxle.metaopt.auto_ml.Trainer.train()

last_trial(continue_on_error: bool = False) → neuraxle.metaopt.data.aggregates.Round[source]

This is a method to be used as a context manager. This will sync items with the repos. Example:

See also

neuraxle.metaopt.data.aggregates.BaseAggregate neuraxle.metaopt.data.aggregates.BaseAggregate.__enter__() neuraxle.metaopt.data.aggregates.BaseAggregate.__exit__() neuraxle.metaopt.auto_ml.Trainer neuraxle.metaopt.auto_ml.Trainer.train()

refitting_best_trial() → neuraxle.metaopt.data.aggregates.Round[source]

This is a method to be used as a context manager. This will sync items with the repos. Example:

See also

neuraxle.metaopt.data.aggregates.BaseAggregate neuraxle.metaopt.data.aggregates.BaseAggregate.__enter__() neuraxle.metaopt.data.aggregates.BaseAggregate.__exit__() neuraxle.metaopt.auto_ml.Trainer neuraxle.metaopt.auto_ml.Trainer.train()

_trials
get_id
_acquire_managed_subresource(new_trial=True, continue_on_error: bool = False) → neuraxle.metaopt.data.aggregates.Trial[source]

Get a trial.

Parameters
  • new_trial (bool) – If True, will create a new trial. If false, will load the last trial. If None, will load the best trial.

  • continue_on_error (bool) – If True, will continue to the next trial if the current trial fails. Otherwise, will let the exception be raised for the failure (won’t catch).

_release_managed_subresource(resource: neuraxle.metaopt.data.aggregates.Trial, e: Exception = None) → bool[source]

Release a subaggregate that was acquired with managed_subresource. The subaggregate normally has already saved itself. We may update things again here if needed.

Exceptions may be handled here. If handled, return True, if not, then return False.

get_best_hyperparams(metric_name: str = None) → neuraxle.hyperparams.space.HyperparameterSamples[source]

Get best hyperparams from all trials.

Returns

get_best_trial(metric_name: str = None) → Optional[neuraxle.metaopt.data.aggregates.Trial][source]
Returns

trial with best score from all trials

is_higher_score_better(metric_name: str = None) → bool[source]

Return true if higher score is better. If metric_name is None, the optimizer’s metric is taken.

:return

append(trial: neuraxle.metaopt.data.aggregates.Trial)[source]

Add a new trial. Will also save the trial shallowly.

Parameters

trial – new trial

Returns

copy() → neuraxle.metaopt.data.aggregates.Round[source]
get_number_of_split()[source]
get_metric_names() → List[str][source]
best_result_summary(metric_name: str = None) → Tuple[float, int, OrderedDict[str, Any]][source]
summary(metric_name: str = None) → List[Tuple[float, int, OrderedDict[str, Any]]][source]

Get a summary of the round. Best score is first. Values in the returned triplet tuples are: (score, trial_number, hyperparams), sorted by score such that the best score is first.

get_all_hyperparams() → List[OrderedDict[str, Any]][source]

Get all hyperparams from all trials.

Returns

_abc_impl = <_abc_data object>
class neuraxle.metaopt.data.aggregates.Trial(_dataclass: SubDataclassT, context: neuraxle.metaopt.data.vanilla.AutoMLContext, is_deep=False, parent: ParentAggregateT = None)[source]

Bases: neuraxle.metaopt.data.aggregates.BaseAggregate

This class is a sub-contextualization of the HyperparameterRepository class for holding a trial and manipulating it within its context. A Trial contains the results for each validation split. Each trial split contains both the training set results, and the validation set results.

See also

Trial, TrialSplit, HyperparamsRepository, AutoML, ExecutionContext

continue_loop_on_error() → neuraxle.metaopt.data.aggregates.Trial[source]
_validation_splits
main_metric_name
new_validation_split(continue_loop_on_error: bool = False) → neuraxle.metaopt.data.aggregates.Trial[source]

This is a method to be used as a context manager. This will sync items with the repos. Example:

See also

neuraxle.metaopt.data.aggregates.BaseAggregate neuraxle.metaopt.data.aggregates.BaseAggregate.__enter__() neuraxle.metaopt.data.aggregates.BaseAggregate.__exit__() neuraxle.metaopt.auto_ml.Trainer neuraxle.metaopt.auto_ml.Trainer.train()

retrain_split() → neuraxle.metaopt.data.aggregates.Trial[source]
_acquire_managed_subresource(continue_loop_on_error: bool, retrain_split=False) → neuraxle.metaopt.data.aggregates.TrialSplit[source]

Acquire a new subaggregate that is managed such that it is deep saved at the beginning.

_release_managed_subresource(resource: neuraxle.metaopt.data.aggregates.TrialSplit, e: Exception = None) → bool[source]

Release a subaggregate that was acquired with managed_subresource. The subaggregate normally has already saved itself. We may update things again here if needed.

Exceptions may be handled here. If handled, return True, if not, then return False.

is_success()[source]

Checks if the trial is successful from its dataclass record.

get_status() → neuraxle.base.TrialStatus[source]

Get the status of the trial.

are_all_splits_successful() → bool[source]

Return true if all splits are successful.

are_all_splits_failures() → bool[source]

Return true if all splits are failed.

get_avg_validation_score(metric_name: str = None) → Optional[float][source]

Returns the average score for all validation splits’s best validation score for the specified scoring metric.

Returns

validation score

get_avg_n_epoch_to_best_validation_score(metric_name: str = None) → float[source]
get_hyperparams() → neuraxle.hyperparams.space.HyperparameterSamples[source]

Return hyperparams dict.

Returns

hyperparams dict

_set_success() → neuraxle.metaopt.data.aggregates.Trial[source]

Set trial status to success. Must save after to ensure consistency.

Returns

self

_set_failed(error: Exception) → neuraxle.metaopt.data.aggregates.Trial[source]

Set failed trial with exception. Must save after to ensure consistency.

Parameters

error (Exception) – catched exception

Returns

self

get_trial_id(hp_dict: Dict[KT, VT])[source]

Hash hyperparams with blake2s to create a trial hash.

Parameters

hp_dict – hyperparams dict

Returns

is_higher_score_better(metric_name: str = None) → bool[source]

Return if the metric is higher score is better.

Return type

bool

Parameters

metric_name (str) – metric name

Returns

bool

_abc_impl = <_abc_data object>
class neuraxle.metaopt.data.aggregates.TrialSplit(_dataclass: SubDataclassT, context: neuraxle.metaopt.data.vanilla.AutoMLContext, is_deep=False, parent: neuraxle.metaopt.data.aggregates.Trial = None)[source]

Bases: neuraxle.metaopt.data.aggregates.BaseAggregate

One split of a trial. This is where a model is trained and evaluated on a specific dataset split.

__init__(_dataclass: SubDataclassT, context: neuraxle.metaopt.data.vanilla.AutoMLContext, is_deep=False, parent: neuraxle.metaopt.data.aggregates.Trial = None)[source]

Initialize self. See help(type(self)) for accurate signature.

_invariant()[source]
main_metric_name
get_hyperparams() → neuraxle.hyperparams.space.RecursiveDict[source]
with_n_epochs(n_epochs: int) → neuraxle.metaopt.data.aggregates.TrialSplit[source]
next_epoch() → int[source]

Increment epoch. Returns the new epoch id. Epochs are 1-indexed like lengths: first epoch is 1, second is 2, etc.

start() → neuraxle.metaopt.data.aggregates.TrialSplit[source]

Start the trial split.

metric_results
metric_result(metric_name: str = None) → neuraxle.metaopt.data.aggregates.MetricResults[source]

Get a metric result that is not managed with a “with” statement. Access it read-only.

managed_metric(metric_name: str, higher_score_is_better: bool) → neuraxle.metaopt.data.aggregates.TrialSplit[source]

To be used as a with statement to get the managed metric.

This is a method to be used as a context manager. This will sync items with the repos. Example:

See also

neuraxle.metaopt.data.aggregates.BaseAggregate neuraxle.metaopt.data.aggregates.BaseAggregate.__enter__() neuraxle.metaopt.data.aggregates.BaseAggregate.__exit__() neuraxle.metaopt.auto_ml.Trainer neuraxle.metaopt.auto_ml.Trainer.train()

_acquire_managed_subresource(metric_name: str, higher_score_is_better: bool) → neuraxle.metaopt.data.aggregates.MetricResults[source]

Acquire a new subaggregate that is managed such that it is deep saved at the beginning.

_create_or_get_metric_results(name, higher_score_is_better)[source]
_release_managed_subresource(resource: neuraxle.metaopt.data.aggregates.MetricResults, e: Exception = None) → bool[source]

Release a subaggregate that was acquired with managed_subresource. The subaggregate normally has already saved itself. We may update things again here if needed.

Exceptions may be handled here. If handled, return True, if not, then return False.

train_context() → neuraxle.metaopt.data.vanilla.AutoMLContext[source]
validation_context() → neuraxle.metaopt.data.vanilla.AutoMLContext[source]
get_metric_names() → List[str][source]
_set_success() → neuraxle.metaopt.data.aggregates.TrialSplit[source]

Set trial status to success. Must save after to ensure consistency.

Returns

self

is_success()[source]

Set trial status to success.

_set_failed(error: Exception) → neuraxle.metaopt.data.aggregates.TrialSplit[source]

Set failed trial with exception. Must save after to ensure consistency.

Parameters

error (Exception) – catched exception

Returns

self

_abc_impl = <_abc_data object>
class neuraxle.metaopt.data.aggregates.MetricResults(_dataclass: SubDataclassT, context: neuraxle.metaopt.data.vanilla.AutoMLContext, is_deep=False, parent: ParentAggregateT = None)[source]

Bases: neuraxle.metaopt.data.aggregates.BaseAggregate

_invariant()[source]
metric_name
_acquire_managed_subresource(*args, **kwds) → SubAggregateT[source]

Acquire a new subaggregate that is managed such that it is deep saved at the beginning.

add_train_result(score: float)[source]

Add a train metric result.

Parameters
  • name – name of the metric. If None, use the main metric name.

  • score (float) – the value to be logged

  • higher_score_is_better – wheter or not a higher score is better for this metric

add_valid_result(score: float)[source]

Add a validation metric result.

Parameters
  • name – name of the metric. If None, use the main metric name.

  • score (float) – the value to be logged

  • higher_score_is_better – wheter or not a higher score is better for this metric

get_train_scores() → List[float][source]
get_valid_scores() → List[float][source]

Return the validation scores for the given scoring metric.

get_final_validation_score() → float[source]

Return the latest validation score for the given scoring metric.

get_best_validation_score() → float[source]

Return the best validation score for the given scoring metric.

get_n_epochs_to_best_validation_score() → int[source]

Return the number of epochs

is_higher_score_better() → bool[source]

Return True if higher scores are better for the main metric.

Returns

is_new_best_score() → bool[source]

Return True if the latest validation score is the new best score.

Returns

_abc_impl = <_abc_data object>