arxivst stuff from arxiv that you should probably bookmark

Construction of a Japanese Word Similarity Dataset

Abstract · Mar 17, 2017 07:53 ·

cs-cl

Arxiv Abstract

  • Yuya Sakaizawa
  • Mamoru Komachi

An evaluation of distributed word representation is generally conducted using a word similarity task and/or a word analogy task. There are many datasets readily available for these tasks in English. However, evaluating distributed representation in languages that do not have such resources (e.g., Japanese) is difficult. Therefore, as a first step toward evaluating distributed representations in Japanese, we constructed a Japanese word similarity dataset. To the best of our knowledge, our dataset is the first resource that can be used to evaluate distributed representations in Japanese. Moreover, our dataset contains various parts of speech and includes rare words in addition to common words.

Read the paper (pdf) »