# Optimizer using Natural Evolution Strategies¶

## NaturalEvolutionStrategiesOptimizer¶

class l2l.optimizers.naturalevolutionstrategies.optimizer.NaturalEvolutionStrategiesOptimizer(traj, optimizee_create_individual, optimizee_fitness_weights, parameters, optimizee_bounding_func=None)[source]

Class Implementing the separable natural evolution strategies optimizer in natural coordinates as in:

Wierstra, D., Schaul, T., Peters, J., & Schmidhuber, J. (2008). Natural evolution strategies. In Evolutionary Computation, 2008. CEC 2008.(IEEE World Congress on Computational Intelligence) (pp. 3381-3387).

Glasmachers, T., Schaul, T., Yi, S., Wierstra, D., & Schmidhuber, J. (2010). Exponential natural evolution strategies. In Proceedings of the 12th annual conference on Genetic and evolutionary computation (pp. 393-400).

Wierstra, D., Schaul, T., Glasmachers, T., Sun, Y., Peters, J., & Schmidhuber, J. (2014). Natural evolution strategies. In Journal of Machine Learning Research, 15(1) (pp. 949-980).

In the pseudo code the algorithm does:

For n iterations do:
• Sample individuals z from multinormal search distribution with parameters mu, sigma

s <- sample from N(0,1) z <- mu + sigma * s

• If mirrored sampling is enabled, also sample individuals with opposite perturbations s

z <- [mu + sigma * s, mu - sigma * s]

• evaluate individuals z and get fitnesses F_i(z)

• Update the parameters of the search distribution as

mu_{t+1} <- mu_{t+1} + eta_mu * sigma * sum(F_i * s_i) sigma_{t+1} <- sigma_t * exp(eta_sigma / 2 * sum(F_i * (s_i ** 2 - 1))

• If fitness shaping is enabled, F_i is replaced with the utility u_i in the previous step, which is calculated as:

u_i = max(0, log(n/2 + 1) - log(k)) / sum_{k=1}^{n}{max(0, log(n/2 + 1) - log(k))} - 1 / n

where k and i are the indices of the individuals in descending order of fitness F_i

Parameters: traj (Trajectory) – Use this trajectory to store the parameters of the specific runs. The parameters should be initialized based on the values in parameters optimizee_create_individual – Function that creates a new individual. All parameters of the Individual-Dict returned should be of numpy.float64 type optimizee_fitness_weights – Fitness weights. The fitness returned by the Optimizee is multiplied by these values (one for each element of the fitness vector) parameters – Instance of namedtuple() NaturalEvolutionStrategiesParameters containing the parameters needed by the Optimizer
post_process(traj, fitnesses_results)[source]
end(traj)[source]

## NaturalEvolutionStrategiesParameters¶

class l2l.optimizers.naturalevolutionstrategies.optimizer.NaturalEvolutionStrategiesParameters

Bases: tuple

Parameters: learning_rate_mu – Learning rate for mean of distribution learning_rate_sigma – Learning rate for standard deviation of distribution mu – Initial mean of search distribution sigma – Initial standard deviation of search distribution mirrored_sampling_enabled – Should we turn on mirrored sampling i.e. sampling both e and -e fitness_shaping_enabled – Should we turn on fitness shaping i.e. using only top fitness_shaping_ratio to update current individual? pop_size – Number of individuals per simulation. n_iteration – Number of iterations to perform stop_criterion – (Optional) Stop if this fitness is reached. seed – The random seed used for generating new individuals
fitness_shaping_enabled
learning_rate_mu
learning_rate_sigma
mirrored_sampling_enabled
mu
n_iteration
pop_size
seed
sigma
stop_criterion