arxivst stuff from arxiv that you should probably bookmark

Riemannian stochastic quasi-Newton algorithm with variance reduction and its convergence analysis

Abstract · Mar 15, 2017 02:34 ·

cs-lg cs-na math-na math-oc stat-ml

Arxiv Abstract

  • Hiroyuki Kasai
  • Hiroyuki Sato
  • Bamdev Mishra

Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions. The present paper proposes a Riemannian stochastic quasi-Newton algorithm with variance reduction (R-SQN-VR). The key challenges of averaging, adding, and subtracting multiple gradients are addressed with notions of retraction and vector transport. We present a global convergence analysis and a local convergence rate analysis of R-SQN-VR under some natural assumptions. The proposed algorithm is applied to the Karcher mean computation on the symmetric positive-definite manifold and low-rank matrix completion on the Grassmann manifold. In all cases, the proposed algorithm outperforms the Riemannian stochastic gradient descent and the Riemannian stochastic variance reduction algorithms.

Read the paper (pdf) »