nifty8.minimization.stochastic_minimizer module

class ADVIOptimizer(controller, eta=1, alpha=0.1, tau=1, epsilon=1e-16, resample=True)

Bases: Minimizer

Provide an implementation of an adaptive step-size sequence optimizer, following https://arxiv.org/abs/1603.00788.

This stochastic optimizer keeps track of the evolution of the gradient over the last steps to adaptively determine the step-size of the next update. It is a variation of the Adam optimizer for Gaussian variational inference and it allows to optimizer stochastic loss functions.

Parameters:

• steps (int) – The number of concecutive steps during one call of the optimizer.

• eta (positive float) – The scale of the step-size sequence. It might have to be adapted to the application to increase performance. Default: 1.

• alpha (float between 0 and 1) – The fraction of how much the current gradient impacts the momentum. Lower values correspond to a longer memory.

• tau (positive float) – This quantity prevents division by zero.

• epsilon (positive float) – A small value guarantees Robbins and Monro conditions.

• resample (bool) – Whether the loss function is resampled for the next iteration. Stochastic losses require resampleing, deterministic ones not.

__call__(energy)

Performs the minimization of the provided Energy functional.

Parameters:

• energy (Energy) – Energy object at the starting point of the iteration

• preconditioner (LinearOperator, optional) – Preconditioner to accelerate the minimization

Returns:

• Energy (Latest energy of the minimization.)

• int (exit status of the minimization) – Can be controller.CONVERGED or controller.ERROR

__init__(controller, eta=1, alpha=0.1, tau=1, epsilon=1e-16, resample=True)

reset()