Optimizers¶

class
tinynet.optims.
SGDOptimizer
(lr, momentum=None)¶ In this class, we implement the stochastic gradient descent algorithm, which is used to update the parameters in a neural network. The algorithm is simple:
\[w^{new} = w^{old}\lambda \nabla\]where \(\lambda\) is the preset learning rate, and \(\nabla\) is the gradient.