AdaHessian
ADAHESSIAN
ADAHESSIAN is a new stochastic optimization algorithm that directly incorporates approximate curvature information from the loss function, and it includes several novel performance-improving features, including a fast Hutchinson based method to approximate the curvature matrix with low computational overhead.
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
https://paperswithcode.com/paper/adahessian-an-adaptive-second-order-optimizer
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning