Optimizer using Natural Evolution Strategies

NaturalEvolutionStrategiesOptimizer

class l2l.optimizers.naturalevolutionstrategies.optimizer.NaturalEvolutionStrategiesOptimizer(traj, optimizee_create_individual, optimizee_fitness_weights, parameters, optimizee_bounding_func=None)[source]

Bases: l2l.optimizers.optimizer.Optimizer

Class Implementing the separable natural evolution strategies optimizer in natural coordinates as in:

Wierstra, D., Schaul, T., Peters, J., & Schmidhuber, J. (2008). Natural evolution strategies. In Evolutionary Computation, 2008. CEC 2008.(IEEE World Congress on Computational Intelligence) (pp. 3381-3387).

Glasmachers, T., Schaul, T., Yi, S., Wierstra, D., & Schmidhuber, J. (2010). Exponential natural evolution strategies. In Proceedings of the 12th annual conference on Genetic and evolutionary computation (pp. 393-400).

Wierstra, D., Schaul, T., Glasmachers, T., Sun, Y., Peters, J., & Schmidhuber, J. (2014). Natural evolution strategies. In Journal of Machine Learning Research, 15(1) (pp. 949-980).

In the pseudo code the algorithm does:

For n iterations do:

  • Sample individuals z from multinormal search distribution with parameters mu, sigma

    s <- sample from N(0,1) z <- mu + sigma * s

  • If mirrored sampling is enabled, also sample individuals with opposite perturbations s

    z <- [mu + sigma * s, mu - sigma * s]

  • evaluate individuals z and get fitnesses F_i(z)

  • Update the parameters of the search distribution as

    mu_{t+1} <- mu_{t+1} + eta_mu * sigma * sum(F_i * s_i) sigma_{t+1} <- sigma_t * exp(eta_sigma / 2 * sum(F_i * (s_i ** 2 - 1))

  • If fitness shaping is enabled, F_i is replaced with the utility u_i in the previous step, which is calculated as:

    u_i = max(0, log(n/2 + 1) - log(k)) / sum_{k=1}^{n}{max(0, log(n/2 + 1) - log(k))} - 1 / n

    where k and i are the indices of the individuals in descending order of fitness F_i

Parameters:
  • traj (Trajectory) – Use this trajectory to store the parameters of the specific runs. The parameters should be initialized based on the values in parameters
  • optimizee_create_individual – Function that creates a new individual. All parameters of the Individual-Dict returned should be of numpy.float64 type
  • optimizee_fitness_weights – Fitness weights. The fitness returned by the Optimizee is multiplied by these values (one for each element of the fitness vector)
  • parameters – Instance of namedtuple() NaturalEvolutionStrategiesParameters containing the parameters needed by the Optimizer
post_process(traj, fitnesses_results)[source]

See post_process()

end(traj)[source]

See end()

NaturalEvolutionStrategiesParameters

class l2l.optimizers.naturalevolutionstrategies.optimizer.NaturalEvolutionStrategiesParameters

Bases: tuple

Parameters:
  • learning_rate_mu – Learning rate for mean of distribution
  • learning_rate_sigma – Learning rate for standard deviation of distribution
  • mu – Initial mean of search distribution
  • sigma – Initial standard deviation of search distribution
  • mirrored_sampling_enabled – Should we turn on mirrored sampling i.e. sampling both e and -e
  • fitness_shaping_enabled – Should we turn on fitness shaping i.e. using only top fitness_shaping_ratio to update current individual?
  • pop_size – Number of individuals per simulation.
  • n_iteration – Number of iterations to perform
  • stop_criterion – (Optional) Stop if this fitness is reached.
  • seed – The random seed used for generating new individuals
fitness_shaping_enabled
learning_rate_mu
learning_rate_sigma
mirrored_sampling_enabled
mu
n_iteration
pop_size
seed
sigma
stop_criterion