# neuraxle.hyperparams.distributions¶

Module-level documentation for neuraxle.hyperparams.distributions. Here is an inheritance diagram, including dependencies to other base modules of Neuraxle:

## Hyperparameter Distributions¶

Here you’ll find a few hyperparameter distributions. It’s also possible to create yours by inheriting from the base class. Each distribution must override the method rvs, which will return a sampled value from the distribution.

Functions

 get_index_in_list_with_bool(choice_list, value)

Classes

 Boolean(proba_is_true[, null_default_value]) Get a random boolean hyperparameter. Choice(choice_list, probas[, null_default_value]) Get a random value from a choice list of possible value for this hyperparameter. TODO docstring TODO replace inheritance TODO docstring DistributionMixture(distributions, …) Get a mixture of multiple distribution FixedHyperparameter(value[, null_default_value]) This is an hyperparameter that won’t change again, but that is still expressed as a distribution. Base class for other hyperparameter distributions. LogNormal(log2_space_mean, log2_space_std, …) Get a LogNormal distribution. LogUniform(min_included, max_included[, …]) Get a LogUniform distribution. Normal(mean, std, hard_clip_min, …) Get a normal distribution. PriorityChoice(choice_list, probas[, …]) Get a random value from a choice list of possible value for this hyperparameter. Quantized(hd, hds[, null_default_value]) A quantized wrapper for another distribution: will round() the rvs number. RandInt(min_included, max_included, …) Get a random integer within a range Uniform(min_included, max_included[, …]) Get a uniform distribution.

### Examples using neuraxle.hyperparams.distributions.RandInt¶

class neuraxle.hyperparams.distributions.HyperparameterDistribution(null_default_value, is_continuous: bool = True)[source]

Bases: object

Base class for other hyperparameter distributions.

__init__(null_default_value, is_continuous: bool = True)[source]

Create a HyperparameterDistribution. This method should still be called with super if it gets overriden.

is_discrete() → bool[source]
rvs()[source]

Sample the random variable.

Returns

The randomly sampled value.

nullify()[source]
pdf(x) → float[source]

Abstract method for probability distribution function value at x.

Return type

float

Parameters

x – value where the probability distribution function is evaluated.

Returns

The probability distribution function value.

cdf(x) → float[source]

Abstract method for cumulative distribution function value at x.

Return type

float

Parameters

x – value where the cumulative distribution function is evaluated.

Returns

The cumulative distribution function value.

min() → float[source]

Abstract method for obtaining minimum value that can sampled in distribution.

Returns

minimal value that can be sampled from distribution.

max() → float[source]

Abstract method for obtaining maximal value that can sampled in distribution.

Returns

maximal value that can be sampled from distribution.

mean() → float[source]

Abstract method for calculating distribution mean value.

Returns

distribution mean value.

std() → float[source]

Base method to calculate distribution std value by taking sqrt of variance value.

Returns

distribution std value.

var() → float[source]

Abstract method for calculate distribution variance value.

: return: distribution variance value.

_abc_impl = <_abc_data object>
class neuraxle.hyperparams.distributions.ContinuousHyperparameterDistrbution(null_default_value=None)[source]

TODO docstring TODO replace inheritance

__init__(null_default_value=None)[source]

Create a HyperparameterDistribution. This method should still be called with super if it gets overriden.

_abc_impl = <_abc_data object>
class neuraxle.hyperparams.distributions.DiscreteHyperparameterDistribution(null_default_value=None)[source]

TODO docstring

__init__(null_default_value=None)[source]

Create a HyperparameterDistribution. This method should still be called with super if it gets overriden.

probabilities() → List[float][source]
values() → List[T][source]
_abc_impl = <_abc_data object>
class neuraxle.hyperparams.distributions.FixedHyperparameter(value, null_default_value=None)[source]

This is an hyperparameter that won’t change again, but that is still expressed as a distribution.

__init__(value, null_default_value=None)[source]

Create a still hyperparameter

Parameters

value – what will be returned by calling .rvs().

rvs()[source]

Sample the non-random anymore value.

Returns

the value given at creation.

pdf(x) → float[source]

Probability distribution function value at x. Since the parameter is fixed, the value return is 1 when x == value and 0 otherwise.

Return type

float

Parameters

x – value where the probability distribution function is evaluated.

Returns

The probability distribution function value.

cdf(x) → float[source]

Cumulative distribution function value at x. Since the parameter is fixed, the value return is 1 if x>= value and 0 otherwise.

Return type

float

Parameters

x – value where the cumulative distribution function is evaluated.

Returns

The cumulative distribution function value.

min()[source]

Calculate minimum value that can be sampled in a fixed distribution.

Returns

minimal value return from distribution.

max()[source]

Calculate maximum value that can be sampled in a fixed distribution.

Returns

maximum value return from distribution.

mean()[source]

Calculate mean value (also called esperance) of the random variable.

Returns

mean value of the random variable.

var()[source]

Calculate variance value of the random variable.

Returns

variance value of the random variable.

_abc_impl = <_abc_data object>
class neuraxle.hyperparams.distributions.Boolean(proba_is_true: Optional[float] = None, null_default_value=False)[source]

Get a random boolean hyperparameter.

__init__(proba_is_true: Optional[float] = None, null_default_value=False)[source]

Create a boolean hyperparameter with given probability.

Boolean distribution is in fact a Bernouilli distribution where given probability set occurance probability of having value 1. (1 - probability) gives occurance probability of having value 0.

Parameters
• proba (float) – a float corresponding to proportion of 1 over 0.

• null_default_value (default choice value. if None, default choice value will be the first choice) – default value for distribution

probabilities() → List[float][source]
values()[source]
rvs()[source]

Get a random True or False.

Returns

True or False (random).

pdf(x) → float[source]

Calculate the boolean probability mass function value at position x. :rtype: float :param x: value where the probability mass function is evaluated. :return: value of the probability mass function.

cdf(x) → float[source]

Calculate the boolean cumulative distribution function value at position x. :rtype: float :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

min()[source]

Calculate minimum value that can be sampled in a boolean distribution.

Returns

minimal value return from distribution.

max()[source]

Calculate maximum value that can be sampled in a boolean distribution.

Returns

maximum value return from distribution.

mean()[source]

Calculate mean value (also called esperance) of the random variable.

Returns

mean value of the random variable.

var()[source]

Calculate variance value of the random variable.

Returns

variance value of the random variable.

_abc_impl = <_abc_data object>
class neuraxle.hyperparams.distributions.Choice(choice_list: List[T], probas: Optional[List[float]] = None, null_default_value=None)[source]

Get a random value from a choice list of possible value for this hyperparameter.

When narrowed, the choice will only collapse to a single element when narrowed enough. For example, if there are 4 items in the list, only at a narrowing value of 0.25 that the first item will be kept alone.

__init__(choice_list: List[T], probas: Optional[List[float]] = None, null_default_value=None)[source]

Create a random choice hyperparameter from the given list.

Parameters
• choice_list (List) – a list of values to sample from.

• null_default_value (default choice value. if None, default choice value will be the first choice) – default value for distribution

probabilities() → List[float][source]
values() → List[float][source]
rvs()[source]

Get one of the items randomly.

Returns

one of the items of the list.

pdf(x) → float[source]

Calculate the choice probability mass function value at position x. :rtype: float :param x: value where the probability mass function is evaluated. :return: value of the probability mass function.

cdf(x) → float[source]

Calculate the choice probability cumulative distribution function value at position x. The index in the list is used to know how the choice is performed. :rtype: float :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

min()[source]

Calculate minimum value that can be sampled in a choice distribution.

Here the minimal index in the list is return. In this case it returns 0.

Returns

minimal value return from distribution.

max()[source]

Calculate maximal value that can be sampled in a choice distribution.

Here the maximal index in the list is return. In this case it returns the length value of the list.

Returns

maximal value return from distribution.

mean()[source]

Calculate mean value (also called esperance) of the random variable.

Returns

mean value of the random variable.

var()[source]

Calculate variance value of the random variable.

Returns

variance value of the random variable.

_abc_impl = <_abc_data object>
class neuraxle.hyperparams.distributions.PriorityChoice(choice_list: List[T], probas: Optional[List[float]] = None, null_default_value=None)[source]

Get a random value from a choice list of possible value for this hyperparameter.

The first parameters are kept until the end when the list is narrowed (it is narrowed progressively), unless there is a best guess that surpasses some of the top choices.

__init__(choice_list: List[T], probas: Optional[List[float]] = None, null_default_value=None)[source]

Create a random choice hyperparameter from the given list (choice_list). The first parameters in the choice_list will be kept longer when narrowing the space.

Parameters
• choice_list (List) – a list of values to sample from.

• null_default_value (default choice value. if None, default choice value will be the first choice) – default value for distribution

probabilities() → List[float][source]
values() → List[T][source]
rvs()[source]

Get one of the items randomly.

Returns

one of the items of the list.

pdf(x) → float[source]

Calculate the choice probability mass function value at position x. :rtype: float :param x: value where the probability mass function is evaluated. :return: value of the probability mass function.

cdf(x) → float[source]

Calculate the choice probability cumulative distribution function value at position x. The index in the list is used to know how the choice is performed. :rtype: float :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

min()[source]

Calculate minimum value that can be sampled in a priority choice distribution.

Here the minimal index in the list is return. In this case it returns 0.

Returns

minimal value return from distribution.

max()[source]

Calculate maximal value that can be sampled in a priority choice distribution.

Here the maximal index in the list is return. In this case it returns the length value of the list.

Returns

maximal value return from distribution.

mean()[source]

Calculate mean value (also called esperance) of the random variable.

Returns

mean value of the random variable.

var()[source]

Calculate variance value of the random variable.

Returns

variance value of the random variable.

_abc_impl = <_abc_data object>
class neuraxle.hyperparams.distributions.WrappedHyperparameterDistributions(hd: neuraxle.hyperparams.distributions.HyperparameterDistribution = None, hds: List[neuraxle.hyperparams.distributions.HyperparameterDistribution] = None, null_default_value=None)[source]
__init__(hd: neuraxle.hyperparams.distributions.HyperparameterDistribution = None, hds: List[neuraxle.hyperparams.distributions.HyperparameterDistribution] = None, null_default_value=None)[source]

Create a wrapper that will surround another HyperparameterDistribution. The wrapper might use one (hd) and/or many (hds) HyperparameterDistribution depending on the argument(s) used.

Parameters
_abc_impl = <_abc_data object>
class neuraxle.hyperparams.distributions.Quantized(hd: neuraxle.hyperparams.distributions.HyperparameterDistribution = None, hds: List[neuraxle.hyperparams.distributions.HyperparameterDistribution] = None, null_default_value=None)[source]

A quantized wrapper for another distribution: will round() the rvs number.

rvs() → int[source]

Will return an integer, rounded from the output of the previous distribution.

Returns

an integer.

pdf(x) → float[source]

Calculate the Quantized probability mass function value at position x of a continuous distribution. :rtype: float :param x: value where the probability mass function is evaluated. :return: value of the probability mass function.

cdf(x) → float[source]

Calculate the Quantized cumulative distribution function at position x of a continuous distribution. :rtype: float :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

min()[source]

Calculate minimum value that can be sampled in the quanitzed version of the distribution.

Returns

minimal value return from distribution.

max()[source]

Calculate maximal value that can be sampled in the quantized version of the distribution.

Returns

maximal value return from distribution.

mean() → float[source]

Calculate mean value (also called esperance) of the random variable.

Returns

mean value of the random variable.

var() → float[source]

Calculate variance value of the random variable.

Returns

variance value of the random variable.

_abc_impl = <_abc_data object>
class neuraxle.hyperparams.distributions.RandInt(min_included: int, max_included: int, null_default_value: int = None)[source]

Get a random integer within a range

__init__(min_included: int, max_included: int, null_default_value: int = None)[source]

Create a quantized random uniform distribution. A random integer between the two values inclusively will be returned.

Parameters
• min_included (int) – minimum integer, included.

• max_included (int) – maximum integer, included.

• null_default_value (int) – null default value for distribution. if None, take the min_included

probabilities()[source]
values()[source]
rvs() → int[source]

Will return an integer in the specified range as specified at creation.

Returns

an integer.

pdf(x) → float[source]

Calculate the random int mass function value at position x. :rtype: float :param x: value where the probability mass function is evaluated. :return: value of the probability mass function.

cdf(x) → float[source]

Calculate the random int cumulative distribution function value at position x. :rtype: float :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

min()[source]

Calculate minimum value that can be sampled in the randint distribution.

Returns

minimal value return from distribution.

max()[source]

Calculate maximal value that can be sampled in the randint distribution.

Returns

maximal value return from distribution.

mean()[source]

Calculate mean value (also called esperance) of the random variable.

Returns

mean value of the random variable.

var()[source]

Calculate variance value of the random variable.

Returns

variance value of the random variable.

_abc_impl = <_abc_data object>
class neuraxle.hyperparams.distributions.Uniform(min_included: float, max_included: float, null_default_value=None)[source]

Get a uniform distribution.

__init__(min_included: float, max_included: float, null_default_value=None)[source]

Create a random uniform distribution. A random float between the two values somehow inclusively will be returned.

Parameters
• min_included (float) – minimum integer, included.

• max_included (float) – maximum integer, might be included - for more info, see examples

• null_default_value (int) – null default value for distribution. if None, take the min_included

rvs() → float[source]

Will return a float value in the specified range as specified at creation.

Returns

a float.

pdf(x)[source]

Calculate the Uniform probability distribution value at position x.

Parameters

x – value where the probability distribution function is evaluated.

Returns

value of the probability distribution function.

cdf(x)[source]

Calculate the Uniform cumulative distribution value at position x. :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

min()[source]

Calculate minimum value that can be sampled in the uniform distribution.

Returns

minimal value return from distribution.

max()[source]

Calculate maximal value that can be sampled in the uniform distribution.

Returns

maximal value return from distribution.

mean()[source]

Calculate mean value (also called esperance) of the random variable.

Returns

mean value of the random variable.

var()[source]

Calculate variance value of the random variable.

Returns

variance value of the random variable.

_abc_impl = <_abc_data object>
class neuraxle.hyperparams.distributions.LogUniform(min_included: float, max_included: float, null_default_value=None)[source]

Get a LogUniform distribution.

For example, this is good for neural networks’ learning rates: that vary exponentially.

__init__(min_included: float, max_included: float, null_default_value=None)[source]

Create a quantized random log uniform distribution. A random float between the two values inclusively will be returned.

Parameters
• min_included (float) – minimum integer, should be somehow included.

• max_included (float) – maximum integer, should be somehow included.

• null_default_value (int) – null default value for distribution. if None, take the min_included

rvs() → float[source]

Will return a float value in the specified range as specified at creation.

Returns

a float.

pdf(x) → float[source]

Calculate the logUniform probability distribution value at position x. :rtype: float :param x: value where the probability distribution function is evaluated. :return: value of the probability distribution function.

cdf(x) → float[source]

Calculate the logUniform cumulative distribution value at position x. :rtype: float :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

min()[source]

Calculate minimum value that can be sampled in the LogUniform distribution.

Returns

minimal value return from distribution.

max()[source]

Calculate maximal value that can be sampled in the LogUniform distribution.

Returns

maximal value return from distribution.

mean()[source]

Calculate mean value (also called esperance) of the random variable.

Returns

mean value of the random variable.

var()[source]

Calculate variance value of the random variable.

Returns

variance value of the random variable.

_abc_impl = <_abc_data object>
class neuraxle.hyperparams.distributions.Normal(mean: float, std: float, hard_clip_min: float = None, hard_clip_max: float = None, null_default_value: float = None)[source]

Get a normal distribution.

__init__(mean: float, std: float, hard_clip_min: float = None, hard_clip_max: float = None, null_default_value: float = None)[source]

Create a normal distribution from mean and standard deviation.

Parameters
• mean (float) – the most common value to pop

• std (float) – the standard deviation (that is, the sqrt of the variance).

• hard_clip_min (float) – if not none, rvs will return max(result, hard_clip_min).

• hard_clip_max (float) – if not none, rvs will return min(result, hard_clip_min).

• null_default_value (float) – if none, null default value will be set to hard_clip_min.

rvs() → float[source]

Will return a float value in the specified range as specified at creation.

Returns

a float.

pdf(x) → float[source]

Calculate the Normal probability distribution value at position x. :rtype: float :param x: value where the probability distribution function is evaluated. :return: value of the probability distribution function.

cdf(x) → float[source]

Calculate the Normal cumulative distribution value at position x. :rtype: float :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

min()[source]

Calculate minimum value that can be sampled in the Normal distribution.

Returns

minimal value return from distribution.

max()[source]

Calculate minimum value that can be sampled in the Normal distribution.

Returns

minimal value return from distribution.

mean()[source]

Calculate mean value (also called esperance) of the random variable.

Returns

mean value of the random variable.

std()[source]

Calculate standard deviation value of the random variable.

Returns

standard deviation value of the random variable.

var()[source]

Calculate variance value of the random variable.

Returns

variance value of the random variable.

_abc_impl = <_abc_data object>
class neuraxle.hyperparams.distributions.LogNormal(log2_space_mean: float, log2_space_std: float, hard_clip_min: float = None, hard_clip_max: float = None, null_default_value=None)[source]

Get a LogNormal distribution.

__init__(log2_space_mean: float, log2_space_std: float, hard_clip_min: float = None, hard_clip_max: float = None, null_default_value=None)[source]

Create a LogNormal distribution.

Parameters
• log2_space_mean (float) – the most common value to pop, but before taking 2**value.

• log2_space_std (float) – the standard deviation of the most common value to pop, but before taking 2**value.

• hard_clip_min (float) – if not none, rvs will return max(result, hard_clip_min). This value is not checked in logspace (so it is checked after the exp).

• hard_clip_max (float) – if not none, rvs will return min(result, hard_clip_min). This value is not checked in logspace (so it is checked after the exp).

• null_default_value (int) – null default value for distribution. if None, take the hard_clip_min

rvs() → float[source]

Will return a float value in the specified range as specified at creation. Note: the range at creation was in log space. The return value is after taking an exponent.

Returns

a float.

pdf(x) → float[source]

Calculate the LogNormal probability distribution value at position x. :rtype: float :param x: value where the probability distribution function is evaluated. :return: value of the probability distribution function.

cdf(x) → float[source]

Calculate the LogNormal cumulative distribution value at position x. :rtype: float :param x: value where the cumulative distribution function is evaluated. :return: value of the cumulative distribution function.

min()[source]

Calculate minimum value that can be sampled in the LogNormal distribution.

Returns

minimal value return from distribution.

max()[source]

Calculate maximal value that can be sampled in the LogNormal distribution.

Returns

maximal value return from distribution.

mean()[source]

Calculate mean value (also called esperance) of the random variable.

Returns

mean value of the random variable.

var()[source]

Calculate variance value (also called esperance) of the random variable.

Returns

variance value of the random variable.

_abc_impl = <_abc_data object>
class neuraxle.hyperparams.distributions.DistributionMixture(distributions: Union[List[neuraxle.hyperparams.distributions.HyperparameterDistribution], Tuple[neuraxle.hyperparams.distributions.HyperparameterDistribution, ...]], distribution_amplitudes: Union[List[float], Tuple[float, ...]])[source]

Get a mixture of multiple distribution

__init__(distributions: Union[List[neuraxle.hyperparams.distributions.HyperparameterDistribution], Tuple[neuraxle.hyperparams.distributions.HyperparameterDistribution, ...]], distribution_amplitudes: Union[List[float], Tuple[float, ...]])[source]

Create a mixture of multiple distributions.

Distribution amplitude are normalized to make sure that the sum equals one. This normalization ensure to keep a random variable at the end (0 < probability < 1).

Parameters
• distributions – list of multiple instantiated distribution.

• distribution_amplitudes – list of float representing the amplitude in the probability distribution function for each distribution.

static build_gaussian_mixture(distribution_amplitudes: Union[numpy.ndarray, List[float], Tuple[float, ...]], means: Union[List[float], Tuple[float, ...]], stds: Union[List[float], Tuple[float, ...]], distributions_mins: Union[List[float], Tuple[float, ...]], distributions_max: Union[List[float], Tuple[float, ...]], use_logs: bool = False, use_quantized_distributions: bool = False)[source]

Create a gaussian mixture.

Will create a mixture distribution which consist of multiple gaussians of different amplitudes, means, standard deviations, mins and max.

Parameters
• distribution_amplitudes – list of different amplitudes to infer to the mixture. The amplitudes are normalized to sum to 1.

• means – list of means to infer mean to each gaussian.

• stds – list of standard deviations to infer standard deviation to each gaussian.

• distributions_mins – list of minimum value that will clip each gaussian. If it is -Inf or None, it will not be clipped.

• distributions_max – list of maximal value that will clip each gaussian. If it is +Inf or None, it will not be clipped.

• distributions_max – bool weither to use a quantized version or not.

• use_logs (bool) – use logs boolean

• use_quantized_distributions (bool) – use quantized distributions boolean

:return DistributionMixture instance

rvs() → float[source]

Will return a float value drawned from the distribution mixture.

Returns

a float.

pdf(x) → float[source]

Calculate the mixture probability distribution value at position x.

Return type

float

Parameters

x – value where the probability distribution function is evaluated.

Returns

value of the probability distribution function.

cdf(x) → float[source]

Calculate the mixture cumulative distribution value at position x.

Return type

float

Parameters

x – value where the cumulative distribution function is evaluated.

Returns

value of the cumulative distribution function.

mean() → float[source]

Calculate mean of the mixture.

Mean of the distribution mixture is calculated using the following equation:

$mean = \sum_{i=1}^n w_i * \mu_i,$

where $$w_i$$ and $$\mu_i$$ are respectively the amplitude and mean of each distribution.

Returns

mean value of the mixtture.

var() → float[source]

Calculate variance of the mixture.

Variance of a mixture is calculated using the following equation:

$variance = (\sum_{i=1}^n w_i * \sigma_i * \mu_i) - \mu^2,$

where $$w_i$$, $$\sigma_i$$ and $$\mu_i$$ are respectively the amplitude, standard deviation and mean of each distribution and $$\mu$$ is the mean of the whole mixture.

Returns

standard deviation value of the mixtture.

std() → float[source]

Calculate standard deviation of the mixture.

Returns

standard deviation value of the mixtture.

min() → float[source]

Calculate minimal domain value of the mixture.

Returns

minimal domain value of the mixtture.

max() → float[source]

Calculate minimal domain value of the mixture.

Returns

minimal domain value of the mixture.

_abc_impl = <_abc_data object>
neuraxle.hyperparams.distributions._calculate_sum(eval_func, limits: List[float], value_step: float = 1.0, tol: float = 1e-10, number_value_before_stop: int = 5)[source]
neuraxle.hyperparams.distributions._get_sum_starting_info(limits)[source]
neuraxle.hyperparams.distributions.get_index_in_list_with_bool(choice_list: List[Any], value: Any) → int[source]