Language Modelling Makes Sense: Propagating Representations through
WordNet for Full-Coverage Word Sense Disambiguation
Abstract
Contextual embeddings represent a new generation of semantic representations learned from
Neural Language Modelling (NLM) that addresses the issue of meaning conflation hampering traditional word embeddings. In this
work, we show that contextual embeddings
can be used to achieve unprecedented gains
in Word Sense Disambiguation (WSD) tasks.
Our approach focuses on creating sense-level
embeddings with full-coverage of WordNet,
and without recourse to explicit knowledge of
sense distributions or task-specific modelling.
As a result, a simple Nearest Neighbors (kNN) method using our representations is able
to consistently surpass the performance of previous systems using powerful neural sequencing models. We also analyse the robustness
of our approach when ignoring part-of-speech
and lemma features, requiring disambiguation
against the full sense inventory, and revealing
shortcomings to be improved. Finally, we explore applications of our sense embeddings for
concept-level analyses of contextual embeddings and their respective NLMs.