arxivst stuff from arxiv that you should probably bookmark

Bandit Structured Prediction for Neural Sequence-to-Sequence Learning

Abstract · Apr 21, 2017 11:56 ·

feedback gold sequence structured translations bandit simulated output translation stat-ml cs-cl cs-lg

Arxiv Abstract

  • Julia Kreutzer
  • Artem Sokolov
  • Stefan Riezler

Bandit structured prediction describes a stochastic optimization framework where learning is performed from partial feedback. This feedback is received in the form of a task loss evaluation to a predicted output structure, without having access to gold standard structures. We advance this framework by lifting linear bandit learning to neural sequence-to-sequence learning problems using attention-based recurrent neural networks. Furthermore, we show how to incorporate control variates into our learning algorithms for variance reduction and improved generalization. We present an evaluation on a neural machine translation task that shows improvements of up to 5.89 BLEU points for domain adaptation from simulated bandit feedback.

Read the paper (pdf) »