Сontext-dependent Word Embeddings for Word Sense Induction in Russian Language
In the present work, contextualized word embeddings such as provided by ELMo or BERT are applied to the Word Sense Induction (WSI) task for the Russian language.
Since embeddings produced by these models depend on context, we presumed that they could be able to capture the particular word meaning used in a particular sentence. We have tested it on the three datasets available for Russian language WSI task. We created a WSI system for Russian language based on clustering context-dependent word embeddings constructed by pre-trained language models.