arxivst stuff from arxiv that you should probably bookmark

Build a Pong Simulator

Post · Apr 11, 2017 16:17 ·

environment simulator DQN cs-ai cs-lg stat-ml

Not just for pong, this paper simulates environments from 2d Atari games to 3d racing sims. It has DQN Scores on a bunch of games (and goes into high detail on how they got those scores), but they don’t explicitly compare those score with current state-of-the-art.

Highlights From the Paper

  • We also introduce a simulator that does not need to predict visual inputs after every action, reducing the computational burden in the use of the model.
  • Indeed we found that, the higher the number of consecutive prediction-dependent transitions, the more the model is encouraged to focus on learning the global dynamics of the environment, which results in higher long-term accuracy
  • Whilst the LSTM memory and our training scheme have proven to capture long-term dependencies, alternative memory structures are required in order, for example, to learn spatial coherence at a more global level than the one displayed by our model in the 3D mazes in oder to do navigation.

  • [Note:]

Arxiv Abstract

  • Silvia Chiappa
  • Sébastien Racaniere
  • Daan Wierstra
  • Shakir Mohamed

Models that can simulate how environments change in response to actions can be used by agents to plan and act efficiently. We improve on previous environment simulators from high-dimensional pixel observations by introducing recurrent neural networks that are able to make temporally and spatially coherent predictions for hundreds of time-steps into the future. We present an in-depth analysis of the factors affecting performance, providing the most extensive attempt to advance the understanding of the properties of these models. We address the issue of computationally inefficiency with a model that does not need to generate a high-dimensional image at each time-step. We show that our approach can be used to improve exploration and is adaptable to many diverse environments, namely 10 Atari games, a 3D car racing environment, and complex 3D mazes.

Read the paper (pdf) »