Reference

minimize(loss, train[, valid, params, …])

Minimize a loss function with respect to some symbolic parameters.

Base

This module defines a base class for optimization techniques.

build(algo, loss[, params, inputs, updates, …])	Construct an optimizer by name.
Optimizer(loss[, params, inputs, updates, …])	An optimizer computes gradient updates to iteratively optimize a loss.

First-Order Optimizers

This module defines first-order gradient descent optimizers.

SGD(loss[, params, inputs, updates, …])	Basic optimization using stochastic gradient descent.
NAG(loss[, params, inputs, updates, …])	Stochastic gradient optimization with Nesterov momentum.

Adaptive Optimizers

This module defines gradient descent optimizers with adaptive learning rates.

ADADELTA(loss[, params, inputs, updates, …])	ADADELTA optimizer.
ADAGRAD(loss[, params, inputs, updates, …])	ADAGRAD optimizer.
Adam(loss[, params, inputs, updates, …])	Adam optimizer using unbiased gradient moment estimates.
ESGD(*args, **kwargs)	Equilibrated SGD computes a diagonal Hessian preconditioner.
RMSProp(loss[, params, inputs, updates, …])	RMSProp optimizer.
RProp(loss[, params, inputs, updates, …])	Resilient backpropagation optimizer.

Datasets

This module contains a class for handling batched datasets.

In many optimization tasks, parameters must be updated by optimizing them with respect to estimates of a loss function. The loss function for many problems is estimated using a set of data that we have measured.

Dataset(inputs[, name, batch_size, …])

This class handles batching and shuffling a dataset.