neuraxle.hyperparams.space

Module-level documentation for neuraxle.hyperparams.space. Here is an inheritance diagram, including dependencies to other base modules of Neuraxle:

Hyperparameter Dictionary Conversions

Ways to convert from a nested dictionary of hyperparameters to a flat dictionary, and vice versa.

Here is a nested dictionary:

{
    "b": {
        "a": {
            "learning_rate": 7
        },
        "learning_rate": 9
    }
}

Here is an equivalent flat dictionary for the previous nested one:

{
    "b.a.learning_rate": 7,
    "b.learning_rate": 9
}

Notice that if you have a SKLearnWrapper on a sklearn Pipeline object, the hyperparameters past that point will use double underscores __ as a separator rather than a dot in flat dictionaries, and in nested dictionaries the sklearn params will appear as a flat past the sklearn wrapper, which is fine.

By default, hyperparameters are stored inside a HyperparameterSpace or inside a HyperparameterSamples object, which offers methods to do the conversions above, and also using ordered dicts (OrderedDict) to store parameters in-order.

A HyperparameterSpace can be sampled by calling the .rvs() method on it, which will recursively call .rvs() on all the HyperparameterSpace and HyperparameterDistribution that it contains. It will return a HyperparameterSamples object. A HyperparameterSpace can also be narrowed towards an better, finer subspace, which is itself a HyperparameterSpace. This can be done by calling the .narrow_space_from_best_guess method of the HyperparameterSpace which will also recursively apply the changes to its contained HyperparameterSpace and to all its contained HyperparameterDistribution.

The HyperparameterSamples contains sampled hyperparameter, that is, a valued point in the possible space. This is ready to be sent to an instance of the pipeline to try and score it, for example.

Classes

HyperparameterSamples(*args[, separator])	Wraps an hyperparameter nested dict or flat dict, and offer a few more functions.
HyperparameterSpace(*args[, separator])	Wraps an hyperparameter nested dict or flat dict, and offer a few more functions to process all contained HyperparameterDistribution.
RecursiveDict(*args[, separator])	A data structure that provides an interface to access nested dictionaries with “flattened keys”, and a few more functions.

Examples using neuraxle.hyperparams.space.HyperparameterSpace

Usage of AutoML loop, and hyperparams with sklearn models.

Boston Housing Regression with Meta Optimization

class neuraxle.hyperparams.space.RecursiveDict(*args, separator=None, **kwds)

Bases: collections.OrderedDict

A data structure that provides an interface to access nested dictionaries with “flattened keys”, and a few more functions.

e.g.

dct = RecursiveDict({‘a’:{‘b’:2}}) assert dct[“a__b”] == 2 dct[“a__b__c”] = 3 assert dct[‘a’][‘b’][‘c’] == dct[“a__b__c”] == dct.to_flat_dict()[“a__b__c”]

This class serves as a base for HyperparameterSamples and HyperparameterSpace

DEFAULT_SEPARATOR = '__'

__init__(*args, separator=None, **kwds)

Initialize self. See help(type(self)) for accurate signature.

_is_only_arg_a_recursive_dict(args, kwds)

_patch_args()

same_class_new_instance(*args, **kwds)

_patch_arg(arg)

Patches argument if needed. :param arg: arg to patch if needed. :return: (patched_arg, did_patch)

get(key) → Union[Any, neuraxle.hyperparams.space.RecursiveDict]

Return the value for key if key is in the dictionary, else default.

_rec_get(key) → Union[Any, neuraxle.hyperparams.space.RecursiveDict]

Split the keys and call getter recursively until we get to the desired element. None returns every non-recursive elements.

get_root_leaf_data() → OrderedDict[str, Any]

Returns a dictionary of all the non-recursive elements. That is, all the elements that are not RecursiveDict in the current root OrderedDict.

iter_flat(pre_key='', values_only=False) → Iterable[T_co]

Returns a generator which yield (flatenned_key, value) pairs. value is never a RecursiveDict instance. Keys are sorted, then values are sorted as well.

to_flat_dict(use_wildcards=False) → OrderedDict[str, Any]

Returns a FlatDict, that is a totally flatened OrderedDict[str, HPSampledValue], with no recursively nested elements, i.e.: {fully__flattened__params: value}.

Parameters

use_wildcards (bool) – If True, wildcards are used in the keys, as if calling self.to_wildcards() directly. See to_wildcards() for more info.

to_nested_dict() → Dict[str, Union[Any, neuraxle.hyperparams.space.RecursiveDict]]

Returns a dictionary counterpart which is still nested but that contains no RecursiveDict.

to_wildcards() → OrderedDict[str, Any]

Returns a FlatDict with wildcards. Shorten the keys to the shortest possible while mainting the same order and value, as well as the ability to differentiate each key.

_wildcards_reduce(rkeys: List[List[str]], depth=0)

with_separator(separator: str)

Create a new recursive dict (from copy) that uses the given separator at each level.

is_empty() → bool

class neuraxle.hyperparams.space.HyperparameterSamples(*args, separator=None, **kwds)

Bases: neuraxle.hyperparams.space.RecursiveDict

Wraps an hyperparameter nested dict or flat dict, and offer a few more functions.

This can be set on a Pipeline with the method set_hyperparams.

HyperparameterSamples are often the result of calling .rvs() on an HyperparameterSpace.

__init__(*args, separator=None, **kwds)

Initialize self. See help(type(self)) for accurate signature.

class neuraxle.hyperparams.space.HyperparameterSpace(*args, separator=None, **kwds)

Bases: neuraxle.hyperparams.space.RecursiveDict

Wraps an hyperparameter nested dict or flat dict, and offer a few more functions to process all contained HyperparameterDistribution.

This can be set on a Pipeline with the method set_hyperparams_space.

Calling .rvs() on an HyperparameterSpace results in HyperparameterSamples.

__init__(*args, separator=None, **kwds)

Initialize self. See help(type(self)) for accurate signature.

_patch_arg(arg)

Override of the RecursiveDict’s default _patch_arg(arg) method.

Parameters

arg – arg to patch if needed.

Returns

(patched_arg, did_patch)

rvs() → neuraxle.hyperparams.space.HyperparameterSamples

Sample the space of random variables.

Returns

a random HyperparameterSamples, sampled from a point of the present HyperparameterSpace.

nullify()

narrow_space_from_best_guess(best_guesses: neuraxle.hyperparams.space.HyperparameterSpace, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.space.HyperparameterSpace

Takes samples estimated to be the best ones of the space as of yet, and restrict the whole space towards that.

Parameters

• best_guesses – sampled HyperparameterSpace (the result of rvs on each parameter, but still stored as a HyperparameterSpace).

• kept_space_ratio (float) – what proportion of the space is kept. Should be between 0.0 and 1.0. Default is 0.5.

Returns

a new HyperparameterSpace containing the narrowed HyperparameterDistribution objects.

unnarrow() → neuraxle.hyperparams.space.HyperparameterSpace

Return the original space before narrowing of the distribution. If the distribution was never narrowed, the values in the dict will be copies.

Returns

the original HyperparameterSpace before narrowing.