neuraxle.hyperparams.space

Hyperparameter Dictionary Conversions

Ways to convert from a nested dictionary of hyperparameters to a flat dictionary, and vice versa.

Here is a nested dictionary:

{
    "b": {
        "a": {
            "learning_rate": 7
        },
        "learning_rate": 9
    }
}

Here is an equivalent flat dictionary for the previous nested one:

{
    "b.a.learning_rate": 7,
    "b.learning_rate": 9
}

Notice that if you have a SKLearnWrapper on a sklearn Pipeline object, the hyperparameters past that point will use double underscores __ as a separator rather than a dot in flat dictionaries, and in nested dictionaries the sklearn params will appear as a flat past the sklearn wrapper, which is fine.

By default, hyperparameters are stored inside a HyperparameterSpace or inside a HyperparameterSamples object, which offers methods to do the conversions above, and also using ordered dicts (OrderedDict) to store parameters in-order.

A HyperparameterSpace can be sampled by calling the .rvs() method on it, which will recursively call .rvs() on all the HyperparameterSpace and HyperparameterDistribution that it contains. It will return a HyperparameterSamples object. A HyperparameterSpace can also be narrowed towards an better, finer subspace, which is itself a HyperparameterSpace. This can be done by calling the .narrow_space_from_best_guess method of the HyperparameterSpace which will also recursively apply the changes to its contained HyperparameterSpace and to all its contained HyperparameterDistribution.

The HyperparameterSamples contains sampled hyperparameter, that is, a valued point in the possible space. This is ready to be sent to an instance of the pipeline to try and score it, for example.

Functions

flat_to_nested_dict(flat_hyperparams[, …])

Convert a flat hyperparameter dictionary to a nested one.

nested_dict_to_flat(nested_hyperparams[, …])

Convert a nested hyperparameter dictionary to a flat one.

Classes

HyperparameterSamples

Wraps an hyperparameter nested dict or flat dict, and offer a few more functions.

HyperparameterSpace

Wraps an hyperparameter nested dict or flat dict, and offer a few more functions to process all contained HyperparameterDistribution.

class neuraxle.hyperparams.space.HyperparameterSamples[source]

Wraps an hyperparameter nested dict or flat dict, and offer a few more functions.

This can be set on a Pipeline with the method set_hyperparams.

HyperparameterSamples are often the result of calling .rvs() on an HyperparameterSpace.

to_flat() → neuraxle.hyperparams.space.HyperparameterSamples[source]

Will create an equivalent flat HyperparameterSamples.

Returns

an HyperparameterSamples like self, flattened.

to_flat_as_dict_primitive() → dict[source]

Will create an equivalent flat HyperparameterSpace, as a dict.

Returns

an HyperparameterSpace like self, flattened.

to_flat_as_ordered_dict_primitive() → collections.OrderedDict[source]

Will create an equivalent flat HyperparameterSpace, as a dict.

Returns

an HyperparameterSpace like self, flattened.

to_nested_dict() → neuraxle.hyperparams.space.HyperparameterSamples[source]

Will create an equivalent nested dict HyperparameterSamples.

Returns

an HyperparameterSamples like self, as a nested dict.

to_nested_dict_as_dict_primitive() → dict[source]

Will create an equivalent nested dict HyperparameterSpace, as a dict.

Returns

a nested primitive dict type of self.

to_nested_dict_as_ordered_dict_primitive() → collections.OrderedDict[source]

Will create an equivalent nested dict HyperparameterSpace, as a dict.

Returns

a nested primitive dict type of self.

class neuraxle.hyperparams.space.HyperparameterSpace[source]

Wraps an hyperparameter nested dict or flat dict, and offer a few more functions to process all contained HyperparameterDistribution.

This can be set on a Pipeline with the method set_hyperparams_space.

Calling .rvs() on an HyperparameterSpace results in HyperparameterSamples.

narrow_space_from_best_guess(best_guesses: neuraxle.hyperparams.space.HyperparameterSpace, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.space.HyperparameterSpace[source]

Takes samples estimated to be the best ones of the space as of yet, and restrict the whole space towards that.

Parameters
  • best_guess – sampled HyperparameterSpace (the result of rvs on each parameter, but still stored as a HyperparameterSpace).

  • kept_space_ratio – what proportion of the space is kept. Should be between 0.0 and 1.0. Default is 0.5.

Returns

a new HyperparameterSpace containing the narrowed HyperparameterDistribution objects.

nullify()[source]
rvs() → neuraxle.hyperparams.space.HyperparameterSamples[source]

Sample the space of random variables.

Returns

a random HyperparameterSamples, sampled from a point of the present HyperparameterSpace.

to_flat() → neuraxle.hyperparams.space.HyperparameterSpace[source]

Will create an equivalent flat HyperparameterSpace.

Returns

an HyperparameterSpace like self, flattened.

to_nested_dict() → neuraxle.hyperparams.space.HyperparameterSpace[source]

Will create an equivalent nested dict HyperparameterSpace.

Returns

an HyperparameterSpace like self, as a nested dict.

unnarrow() → neuraxle.hyperparams.space.HyperparameterSpace[source]

Return the original space before narrowing of the distribution. If the distribution was never narrowed, the values in the dict will be copies.

Returns

the original HyperparameterSpace before narrowing.

neuraxle.hyperparams.space.flat_to_nested_dict(flat_hyperparams, dict_ctor=<class 'collections.OrderedDict'>)[source]

Convert a flat hyperparameter dictionary to a nested one.

Parameters
  • flat_hyperparams – a flat hyperparameter dictionary.

  • dict_ctorOrderedDict by default. Will use this as a class to create the new returned dict.

Returns

a nested hyperparameter dictionary.

neuraxle.hyperparams.space.nested_dict_to_flat(nested_hyperparams, dict_ctor=<class 'collections.OrderedDict'>)[source]

Convert a nested hyperparameter dictionary to a flat one.

Parameters
  • nested_hyperparams – a nested hyperparameter dictionary.

  • dict_ctorOrderedDict by default. Will use this as a class to create the new returned dict.

Returns

a flat hyperparameter dictionary.