neuraxle.hyperparams.distributions

Hyperparameter Distributions

Here you’ll find a few hyperparameter distributions. It’s also possible to create yours by inheriting from the base class. Each distribution must override the method rvs, which will return a sampled value from the distribution.

Classes

Boolean([null_default_value])

Get a random boolean hyperparameter.

Choice(choice_list[, null_default_value])

Get a random value from a choice list of possible value for this hyperparameter.

FixedHyperparameter(value[, null_default_value])

This is an hyperparameter that won’t change again, but that is still expressed as a distribution.

HyperparameterDistribution(null_default_value)

Base class for other hyperparameter distributions.

LogNormal(log2_space_mean, log2_space_std, …)

Get a LogNormal distribution.

LogUniform(min_included, max_included[, …])

Get a LogUniform distribution.

Normal(mean, std, hard_clip_min, …)

Get a normal distribution.

PriorityChoice(choice_list[, null_default_value])

Get a random value from a choice list of possible value for this hyperparameter.

Quantized(hd, hds[, null_default_value])

A quantized wrapper for another distribution: will round() the rvs number.

RandInt(min_included, max_included, …)

Get a random integer within a range

Uniform(min_included, max_included[, …])

Get a uniform distribution.

WrappedHyperparameterDistributions(hd, hds)

class neuraxle.hyperparams.distributions.Boolean(null_default_value=False)[source]

Get a random boolean hyperparameter.

cdf(x) → float[source]

Calculate the boolean cumulative distribution function value at position x. :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

pdf(x) → float[source]

Calculate the boolean probability mass function value at position x. :param x: value where the probability mass function is evaluated. :return: value of the probability mass function.

rvs()[source]

Get a random True or False.

Returns

True or False (random).

class neuraxle.hyperparams.distributions.Choice(choice_list: List[T], null_default_value=None)[source]

Get a random value from a choice list of possible value for this hyperparameter.

When narrowed, the choice will only collapse to a single element when narrowed enough. For example, if there are 4 items in the list, only at a narrowing value of 0.25 that the first item will be kept alone.

cdf(x) → float[source]

Calculate the choice probability cumulative distribution function value at position x. The index in the list is used to know how the choice is performed. :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.0) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Will narrow the space. If the cumulative kept_space_ratio gets to be under or equal to 1/len(choice_list), then the list is crunched to a single item as a FixedHyperparameter to reflect this narrowing. So once a small enough kept_space_ratio is reached, the list becomes a fixed unique item from the best guess. Otherwise, a deepcopy of self is returned.

Parameters
  • best_guess – the best item of the list to keep if truly narrowing.

  • kept_space_ratio – the ratio of the space to keep.

Returns

a deepcopy of self, or else a FixedHyperparameter of the best_guess.

pdf(x) → float[source]

Calculate the choice probability mass function value at position x. :param x: value where the probability mass function is evaluated. :return: value of the probability mass function.

rvs()[source]

Get one of the items randomly.

Returns

one of the items of the list.

class neuraxle.hyperparams.distributions.FixedHyperparameter(value, null_default_value=None)[source]

This is an hyperparameter that won’t change again, but that is still expressed as a distribution.

cdf(x) → float[source]

Cumulative distribution function value at x. Since the parameter is fixed, the value return is 1 if x>= value and 0 otherwise.

Parameters

x – value where the cumulative distribution function is evaluated.

Returns

The cumulative distribution function value.

pdf(x) → float[source]

Probability distribution function value at x. Since the parameter is fixed, the value return is 1 when x == value and 0 otherwise.

Parameters

x – value where the probability distribution function is evaluated.

Returns

The probability distribution function value.

rvs()[source]

Sample the non-random anymore value.

Returns

the value given at creation.

class neuraxle.hyperparams.distributions.HyperparameterDistribution(null_default_value)[source]

Base class for other hyperparameter distributions.

cdf(x) → float[source]

Abstract method for cumulative distribution function value at x.

Parameters

x – value where the cumulative distribution function is evaluated.

Returns

The cumulative distribution function value.

get_current_narrowing_value()[source]
narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.0) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Takes a value that is estimated to be the best one of the space, and restrict the space near that value. By default, this function will completely replace the returned value by the new guess if not overriden.

Parameters
  • best_guess – the value towards which we want to narrow down the space.

  • kept_space_ratio – what proportion of the space is kept. Should be between 0.0 and 1.0. Default is to keep only the best_guess (0.0).

Returns

a new HyperparameterDistribution object that has been narrowed down.

nullify()[source]
pdf(x) → float[source]

Abstract method for probability distribution function value at x.

Parameters

x – value where the probability distribution function is evaluated.

Returns

The probability distribution function value.

rvs()[source]

Sample the random variable.

Returns

The randomly sampled value.

unnarrow() → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Return the original distribution before narrowing of the distribution. If the distribution was never narrowed, will return a copy of self.

Returns

the original HyperparameterDistribution before narrowing, or else self if the distribution is virgin.

was_narrowed_from(kept_space_ratio: float, original_hp: neuraxle.hyperparams.distributions.HyperparameterDistribution) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Keep track of the original distribution to restore it.

Parameters
  • kept_space_ratio – the ratio which made the current object narrower than the original_hp.

  • original_hp – The original HyperparameterDistribution, which will be kept in a private variable for an eventual restore.

Returns

self.

class neuraxle.hyperparams.distributions.LogNormal(log2_space_mean: float, log2_space_std: float, hard_clip_min: float = None, hard_clip_max: float = None, null_default_value=None)[source]

Get a LogNormal distribution.

cdf(x) → float[source]

Calculate the LogNormal cumulative distribution value at position x. :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Will narrow the distribution towards the new best_guess. The log2_space_mean (log space mean) will move, in log space, towards the new best guess, and the log2_space_std (log space standard deviation) will be multiplied by the kept_space_ratio.

Parameters
  • best_guess – the value towards which we want to narrow down the space’s mean. Should be between 0.0 and 1.0.

  • kept_space_ratio – what proportion of the space is kept. Default is to keep half the space (0.5).

Returns

a new HyperparameterDistribution that has been narrowed down.

pdf(x) → float[source]

Calculate the LogNormal probability distribution value at position x. :param x: value where the probability distribution function is evaluated. :return: value of the probability distribution function.

rvs() → float[source]

Will return a float value in the specified range as specified at creation. Note: the range at creation was in log space. The return value is after taking an exponent.

Returns

a float.

class neuraxle.hyperparams.distributions.LogUniform(min_included: float, max_included: float, null_default_value=None)[source]

Get a LogUniform distribution.

For example, this is good for neural networks’ learning rates: that vary exponentially.

cdf(x) → float[source]

Calculate the logUniform cumulative distribution value at position x. :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Will narrow, in log space, the distribution towards the new best_guess.

Parameters
  • best_guess – the value towards which we want to narrow down the space. Should be between 0.0 and 1.0.

  • kept_space_ratio – what proportion of the space is kept. Default is to keep half the space (0.5).

Returns

a new HyperparameterDistribution that has been narrowed down.

pdf(x) → float[source]

Calculate the logUniform probability distribution value at position x. :param x: value where the probability distribution function is evaluated. :return: value of the probability distribution function.

rvs() → float[source]

Will return a float value in the specified range as specified at creation.

Returns

a float.

class neuraxle.hyperparams.distributions.Normal(mean: float, std: float, hard_clip_min: float = None, hard_clip_max: float = None, null_default_value: float = None)[source]

Get a normal distribution.

cdf(x) → float[source]

Calculate the Normal cumulative distribution value at position x. :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Will narrow the distribution towards the new best_guess. The mean will move towards the new best guess, and the standard deviation will be multiplied by the kept_space_ratio. The hard clip limit is unchanged.

Parameters
  • best_guess – the value towards which we want to narrow down the space’s mean. Should be between 0.0 and 1.0.

  • kept_space_ratio – what proportion of the space is kept. Default is to keep half the space (0.5).

Returns

a new HyperparameterDistribution that has been narrowed down.

pdf(x) → float[source]

Calculate the Normal probability distribution value at position x. :param x: value where the probability distribution function is evaluated. :return: value of the probability distribution function.

rvs() → float[source]

Will return a float value in the specified range as specified at creation.

Returns

a float.

class neuraxle.hyperparams.distributions.PriorityChoice(choice_list: List[T], null_default_value=None)[source]

Get a random value from a choice list of possible value for this hyperparameter.

The first parameters are kept until the end when the list is narrowed (it is narrowed progressively), unless there is a best guess that surpasses some of the top choices.

cdf(x) → float[source]

Calculate the choice probability cumulative distribution function value at position x. The index in the list is used to know how the choice is performed. :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.0) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Will narrow the space. If the cumulative kept_space_ratio gets to be under or equal to 1-1/len(choice_list), then the list is crunched to discard the last items to reflect this narrowing. After a few narrowing (or a big one), the list may become a FixedHyperparameter. Otherwise if the list is unchanged, a deepcopy of self is returned.

Parameters
  • best_guess – the best item of the list, which will be brought back as the first item.

  • kept_space_ratio – the ratio of the space to keep.

Returns

a deepcopy of self, or a subchoice of self, or else a FixedHyperparameter of the best_guess.

pdf(x) → float[source]

Calculate the choice probability mass function value at position x. :param x: value where the probability mass function is evaluated. :return: value of the probability mass function.

rvs()[source]

Get one of the items randomly.

Returns

one of the items of the list.

class neuraxle.hyperparams.distributions.Quantized(hd: neuraxle.hyperparams.distributions.HyperparameterDistribution = None, hds: List[neuraxle.hyperparams.distributions.HyperparameterDistribution] = None, null_default_value=None)[source]

A quantized wrapper for another distribution: will round() the rvs number.

cdf(x) → float[source]

Calculate the Quantized cumulative distribution function at position x of a continuous distribution. :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.distributions.Quantized[source]

Will narrow the underlying distribution and re-wrap it under a Quantized.

Parameters
  • best_guess – the value towards which we want to narrow down the space.

  • kept_space_ratio – what proportion of the space is kept. Default is to keep half the space (0.5).

Returns

pdf(x) → float[source]

Calculate the Quantized probability mass function value at position x of a continuous distribution. :param x: value where the probability mass function is evaluated. :return: value of the probability mass function.

rvs() → int[source]

Will return an integer, rounded from the output of the previous distribution.

Returns

an integer.

class neuraxle.hyperparams.distributions.RandInt(min_included: int, max_included: int, null_default_value: int = None)[source]

Get a random integer within a range

cdf(x) → float[source]

Calculate the random int cumulative distribution function value at position x. :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Will narrow the underlying distribution towards the best guess.

Parameters
  • best_guess – the value towards which we want to narrow down the space. Should be between 0.0 and 1.0.

  • kept_space_ratio – what proportion of the space is kept. Default is to keep half the space (0.5).

Returns

a new HyperparameterDistribution that has been narrowed down.

pdf(x) → float[source]

Calculate the random int mass function value at position x. :param x: value where the probability mass function is evaluated. :return: value of the probability mass function.

rvs() → int[source]

Will return an integer in the specified range as specified at creation.

Returns

an integer.

class neuraxle.hyperparams.distributions.Uniform(min_included: float, max_included: float, null_default_value=None)[source]

Get a uniform distribution.

cdf(x)[source]

Calculate the Uniform cumulative distribution value at position x. :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

narrow_space_from_best_guess(best_guess, kept_space_ratio: float = 0.5) → neuraxle.hyperparams.distributions.HyperparameterDistribution[source]

Will narrow the underlying distribution towards the best guess.

Parameters
  • best_guess – the value towards which we want to narrow down the space. Should be between 0.0 and 1.0.

  • kept_space_ratio – what proportion of the space is kept. Default is to keep half the space (0.5).

Returns

a new HyperparameterDistribution that has been narrowed down.

pdf(x)[source]

Calculate the Uniform probability distribution value at position x.

Parameters

x – value where the probability distribution function is evaluated.

Returns

value of the probability distribution function.

rvs() → float[source]

Will return a float value in the specified range as specified at creation.

Returns

a float.

class neuraxle.hyperparams.distributions.WrappedHyperparameterDistributions(hd: neuraxle.hyperparams.distributions.HyperparameterDistribution = None, hds: List[neuraxle.hyperparams.distributions.HyperparameterDistribution] = None, null_default_value=None)[source]