neuraxle.hyperparams.space

Module-level documentation for neuraxle.hyperparams.space. Here is an inheritance diagram, including dependencies to other base modules of Neuraxle:

Inheritance diagram of neuraxle.hyperparams.space

Hyperparameter Dictionary Conversions

Ways to convert from a nested dictionary of hyperparameters to a flat dictionary, and vice versa.

Here is a nested dictionary:

{
    "b": {
        "a": {
            "learning_rate": 7
        },
        "learning_rate": 9
    }
}

Here is an equivalent flat dictionary for the previous nested one:

{
    "b.a.learning_rate": 7,
    "b.learning_rate": 9
}

Notice that if you have a SKLearnWrapper on a sklearn Pipeline object, the hyperparameters past that point will use double underscores __ as a separator rather than a dot in flat dictionaries, and in nested dictionaries the sklearn params will appear as a flat past the sklearn wrapper, which is fine.

By default, hyperparameters are stored inside a HyperparameterSpace or inside a HyperparameterSamples object, which offers methods to do the conversions above, and also using ordered dicts (OrderedDict) to store parameters in-order.

A HyperparameterSpace can be sampled by calling the .rvs() method on it, which will recursively call .rvs() on all the HyperparameterSpace and HyperparameterDistribution that it contains. It will return a HyperparameterSamples object. A HyperparameterSpace can also be narrowed towards an better, finer subspace, which is itself a HyperparameterSpace. This can be done by calling the .narrow_space_from_best_guess method of the HyperparameterSpace which will also recursively apply the changes to its contained HyperparameterSpace and to all its contained HyperparameterDistribution.

The HyperparameterSamples contains sampled hyperparameter, that is, a valued point in the possible space. This is ready to be sent to an instance of the pipeline to try and score it, for example.

Classes

HyperparameterSamples(*args[, separator])

Wraps an hyperparameter nested dict or flat dict, and offer a few more functions.

HyperparameterSpace(*args[, separator])

Wraps an hyperparameter nested dict or flat dict, and offer a few more functions to process all contained HyperparameterDistribution.

RecursiveDict(*args[, separator])

A data structure that provides an interface to access nested dictionaries with “flattened keys”, and a few more functions.

Examples using neuraxle.hyperparams.space.HyperparameterSpace


class neuraxle.hyperparams.space.RecursiveDict(*args, separator=None, **kwds)[source]

Bases: collections.OrderedDict

A data structure that provides an interface to access nested dictionaries with “flattened keys”, and a few more functions.

e.g.

dct = RecursiveDict({‘a’:{‘b’:2}}) assert dct[“a__b”] == 2 dct[“a__b__c”] = 3 assert dct[‘a’][‘b’][‘c’] == dct[“a__b__c”] == dct.to_flat_dict()[“a__b__c”]

This class serves as a base for HyperparameterSamples and HyperparameterSpace

DEFAULT_SEPARATOR = '__'
__init__(*args, separator=None, **kwds)[source]

Initialize self. See help(type(self)) for accurate signature.

_is_only_arg_a_recursive_dict(args, kwds)[source]
_patch_args()[source]
same_class_new_instance(*args, **kwds)[source]
_patch_arg(arg)[source]

Patches argument if needed. :param arg: arg to patch if needed. :return: (patched_arg, did_patch)

_rec_get(key)[source]

Split the keys and call getter recursively until we get to the desired element. None returns every non-recursive elements.

get(key)[source]

Return the value for key if key is in the dictionary, else default.

iter_flat(pre_key='', values_only=False)[source]

Returns a generator which yield (flatenned_key, value) pairs. value is never a RecursiveDict instance.

to_flat_dict() → dict[source]

Returns a dictionary with no recursively nested elements, i.e. {flattened_key -> value}.

to_nested_dict() → dict[source]

Returns a dictionary counterpart which is still nested but that contains no RecursiveDict.

with_separator(separator)[source]

Create a new recursive dict that uses the given separator at each level.

Parameters

separator

Returns

class neuraxle.hyperparams.space.HyperparameterSamples(*args, separator=None, **kwds)[source]

Bases: neuraxle.hyperparams.space.RecursiveDict

Wraps an hyperparameter nested dict or flat dict, and offer a few more functions.

This can be set on a Pipeline with the method set_hyperparams.

HyperparameterSamples are often the result of calling .rvs() on an HyperparameterSpace.

__init__(*args, separator=None, **kwds)[source]

Initialize self. See help(type(self)) for accurate signature.

class neuraxle.hyperparams.space.HyperparameterSpace(*args, separator=None, **kwds)[source]

Bases: neuraxle.hyperparams.space.RecursiveDict

Wraps an hyperparameter nested dict or flat dict, and offer a few more functions to process all contained HyperparameterDistribution.

This can be set on a Pipeline with the method set_hyperparams_space.

Calling .rvs() on an HyperparameterSpace results in HyperparameterSamples.

__init__(*args, separator=None, **kwds)[source]

Initialize self. See help(type(self)) for accurate signature.

_patch_arg(arg)[source]

Override of the RecursiveDict’s default _patch_arg(arg) method.

rvs() → neuraxle.hyperparams.space.HyperparameterSamples[source]

Sample the space of random variables.

Returns

a random HyperparameterSamples, sampled from a point of the present HyperparameterSpace.

nullify()[source]
narrow_space_from_best_guess(best_guesses: neuraxle.hyperparams.space.HyperparameterSpace, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.space.HyperparameterSpace[source]

Takes samples estimated to be the best ones of the space as of yet, and restrict the whole space towards that.

Parameters
  • best_guesses – sampled HyperparameterSpace (the result of rvs on each parameter, but still stored as a HyperparameterSpace).

  • kept_space_ratio (float) – what proportion of the space is kept. Should be between 0.0 and 1.0. Default is 0.5.

Returns

a new HyperparameterSpace containing the narrowed HyperparameterDistribution objects.

unnarrow() → neuraxle.hyperparams.space.HyperparameterSpace[source]

Return the original space before narrowing of the distribution. If the distribution was never narrowed, the values in the dict will be copies.

Returns

the original HyperparameterSpace before narrowing.