Diachronic Sense Modeling with Deep Contextualized Word Embeddings:
An Ecological View
Abstract
Diachronic word embeddings have been
widely used in detecting temporal changes.
However, existing methods face the meaning
conflation deficiency by representing a word
as a single vector at each time period. To address this issue, this paper proposes a sense
representation and tracking framework based
on deep contextualized embeddings, aiming at
answering not only what and when, but also
how the word meaning changes. The experiments show that our framework is effective
in representing fine-grained word senses, and
it brings a significant improvement in word
change detection task. Furthermore, we model
the word change from an ecological viewpoint,
and sketch two interesting sense behaviors in
the process of language evolution, i.e. sense
competition and sense cooperation.