资源论文Are Girls Neko or Shojo? Cross-Lingual Alignment of Non-Isomorphic ¯ Embeddings with Iterative Normalization

Are Girls Neko or Shojo? Cross-Lingual Alignment of Non-Isomorphic ¯ Embeddings with Iterative Normalization

2019-09-22 | |  133 |   70 |   0 0 0
Abstract Cross-lingual word embeddings (CLWE) underlie many multilingual natural language processing systems, often through orthogonal transformations of pre-trained monolingual embeddings. However, orthogonal mapping only works on language pairs whose embeddings are naturally isomorphic. For nonisomorphic pairs, our method (Iterative Normalization) transforms monolingual embeddings to make orthogonal alignment easier by simultaneously enforcing that (1) individual word vectors are unit length, and (2) each language’s average vector is zero. Iterative Normalization consistently improves word translation accuracy of three CLWE methods, with the largest improvement observed on EnglishJapanese (from 2% to 44% test accuracy)

上一篇:Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned

下一篇:Automatic and Human Evaluation of Local Topic Quality

用户评价
全部评价

热门资源

  • The Variational S...

    Unlike traditional images which do not offer in...

  • Learning to Predi...

    Much of model-based reinforcement learning invo...

  • Stratified Strate...

    In this paper we introduce Stratified Strategy ...

  • Learning to learn...

    The move from hand-designed features to learned...

  • A Mathematical Mo...

    Direct democracy, where each voter casts one vo...