?
Rotations and Interpretability of Word Embeddings: The Case of the Russian Language
Ch. 11. P. 116-128.
Zobnin A.
Consider a continuous word embedding model. Usually, the cosines between word vectors are used as a measure of similarity of words. These cosines do not change under orthogonal transformations of the embedding space. We demonstrate that, using some canonical orthogonal transformations from SVD, it is possible both to increase the meaning of some components and to make the components more stable under re-learning. We study the interpretability of components for publicly available models for the Russian language (RusVectores, fastText, RDT).
In book
Vol. 10716. , Cham : Springer, 2018
Zobnin A., Elistratova E., , in : Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019). Issue W19-43.: Association for Computational Linguistics, 2019. P. 244-249.
Most word embedding algorithms such as word2vec or fastText construct two sort of vectors: for words and for contexts. Naive use of vectors of only one sort leads to poor results. We suggest using indefinite inner product in skip-gram negative sampling algorithm. This allows us to use only one sort of vectors without loss of ...
Added: November 9, 2019
Karyaeva M., Braslavski P., Sokolov V., Automatic Control and Computer Sciences 2019 Vol. 53 P. 638-643
The ability to identify semantic relations between words has made a word2vec model widely used in NLP tasks. The idea of word2vec is based on a simple rule that a higher similarity can be reached if two words have a similar context. Each word can be represented as a vector, so the closest coordinates of vectors can be interpreted ...
Added: April 10, 2020
Voronkov Ilia, Amajd M., Kaimuldenov Z., , in : Actual Problems of System and Software Engineering 2017. Proceedings of the 5th International Conference on Actual Problems of System and Software Engineering Supported by Russian Foundation for Basic Research. Project #17-07-20565 Moscow, Russia, November 14-16, 2017, 408 P. Vol. 1989.: Aachen : CEUR Workshop Proceedings, 2017. P. 362-370.
In this paper, we analyze the use of different neural networks for the
text classification task. The accuracy of the studied text classifiers can be
changed by a small number of previously classified texts. This is important due
to the fact that in many applications of text classification a large number of unlabeled texts are easily accessible, while ...
Added: August 16, 2018
Voynov A., Babenko A., , in : International Conference on Machine Learning (ICML 2020). Vol. 119.: PMLR, 2020. P. 9728-9738.
Added: January 14, 2021
Shirokov D., Journal of Computational and Applied Mathematics 2021 Vol. 391 Article 113450
We present a new formulation of the hyperbolic singular value decomposition (HSVD) for an arbitrary complex (or real) matrix without hyperexchange matrices and redundant invariant parameters. In our formulation, we use only the concept of pseudo-unitary (or pseudo-orthogonal) matrices. We show that computing the HSVD in the general case is reduced to calculation of eigenvalues, ...
Added: February 12, 2021
Doukhnitch E., Podbelskiy V. V., Parallel and Cloud Computing Research (PCCR) 2013 Vol. 1 No. 3 P. 41-49
In this study, new highly parallel algorithm of two-sided Jacobi 8-D transformation is suggested. It is oriented on VLSI-implementation of special processor array. This array is built using 8-D CORDIC algorithm for quaternion valued matrix singular value decomposition. Accuracy analysis and simulation results are added. Such array can be utilized to speed up the Jacobi ...
Added: November 7, 2013
Kutuzov A. B., Kuzmenko E., Marakasova A., , in : Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH). : Osaka : [б.и.], 2016. P. 26-34.
We present an approach to detect differences in lexical semantics across English language registers, using word embedding models from distributional semantics paradigm. Models trained on register-specific subcorpora of the BNC corpus are employed to compare lists of nearest associates for particular words and draw conclusions about their semantic shifts depending on register in which they ...
Added: November 12, 2016
Bobrov E., Kropotov D., Troshin S. et al., Optimization Methods and Software 2022 P. 1-16
The paper studies the multi-user precoding problem as a non-convex optimization problem for wireless multiple inputs and multiple outputs (MIMO) systems. In our work, we approximate the target Spectral Efficiency function with a novel computationally simpler function. Then, we reduce the precoding problem to an unconstrained optimization task using a special differential projection method and ...
Added: October 26, 2022
Shirokov D., , in : Advances in Computer Graphics: 40th Computer Graphics International Conference, CGI 2023, Shanghai, China, August 28 – September 1, 2023, Proceedings, Part IV. * 4. Vol. 14498.: Springer, 2024. P. 391-401.
This paper is a brief note on the natural implementation of singular value decomposition (SVD) and polar decomposition of an arbitrary multivector in nondegenerate real (Clifford) geometric algebras of arbitrary dimension and signature. We naturally define these and other related structures (operation of Hermitian conjugation, Euclidean space, and Lie groups) in geometric algebras. The results ...
Added: December 25, 2023
Glavas G., Franco-Salvador M., Ponzetto S. et al., Knowledge-Based Systems 2018 Vol. 143 P. 1-9
Recognizing semantically similar sentences or paragraphs across languages is beneficial for many tasks, ranging from cross-lingual information retrieval and plagiarism detection to machine translation. Recently proposed methods for predicting cross-lingual semantic similarity of short texts, however, make use of tools and resources (e.g., machine translation systems, syntactic parsers or named entity recognition) that for many ...
Added: October 29, 2020
Korogodina O., Karpik O., Klyshinsky E., , in : GraphiCon 2020 - Proceedings of the 30th International Conference on Computer Graphics and Machine Vision. : St. Petersburg : CEUR-WS, 2020.
Authors of Word2Vec claimed that their technology could solve the word analogy problem using the vector transformation in the introduced vector space. However, the practice demonstrates that it is not always true. In this paper, we investigate several Word2Vec and FastText model trained for the Russian language and find out reasons of such inconsistency. We ...
Added: October 21, 2020
Kutuzov A. B., Козлова О. С., , in : Компьютерная лингвистика и интеллектуальные технологии: По материалам ежегодной международной конференции «Диалог» (Москва,1–4 июля 2016 г.). Вып. 15.: М. : Изд-во РГГУ, 2016. P. 288-300.
In natural language processing, distributional semantic models are known as an efficient data driven approach to word and text representation, which allows computing meaning directly from large text corpora into word embeddings in a vector space. This paper addresses the role of linguistic preprocessing in enhancing performance of distributional models, and particularly studies pronominal anaphora ...
Added: November 12, 2016
Puzyrev D. A., Shelmanov A., Panchenko A. et al., , in : Analysis of Images, Social Networks and Texts. 8th International Conference AIST 2019. : Springer, 2019. P. 218-229.
In this paper, we present the first gold-standard corpus of Russian noun compounds annotated with compositionality information. We used Universal Dependency treebanks to collect noun compounds according to part of speech patterns, such as ADJ-NOUN or NOUN-NOUN and annotated them according to the following schema: a phrase can be either compositional, non-compositional, or ambiguous (i.e., ...
Added: October 30, 2019
Arefyev N V., Fedoseev M., Kabanov A. et al., , in : Компьютерная лингвистика и интеллектуальные технологии: по материалам ежегодной международной конференции «Диалог» (Москва, 17–20 июня 2020 г.). Issue 19(26): дополнительный том.: -, 2020. P. 13-32.
Expert-built lexical resources are known to provide information of good quality for the cost of low coverage. This property limits their applicability in modern NLP applications. Building descriptions of lexical-semantic relations manually in sufficient volume requires a huge amount of qualified human labour. However, given some initial version of a taxonomy is already built, automatic ...
Added: October 9, 2020
Ekaterina Artemova, Bakarov A., Artemov A. et al., Journal of Cognitive Science 2020 Vol. 1 No. 21 P. 15-52
In this paper, our focus is the connection and influence of language technologies on the research in neurolinguistics. We present a review of brain imaging-based neurolinguistics studies with a focus on the natural language representations, such as word embeddings and pre-trained language model. Mutual enrichment of neurolinguistics and language technologies leads to development of brain-aware natural ...
Added: January 17, 2020
Wohlgenannt G., Artemova E., Ilvovsky D., , in : Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH). : Osaka : [б.и.], 2016. Ch. 4. P. 18-26.
In this paper a social network is extracted from a literary text. The social network shows, how frequent the characters interact and how similar their social behavior is. Two types of similarity measures are used: the first applies co-occurrence statistics, while the second exploits cosine similarity on different types of word embedding vectors. The results ...
Added: March 6, 2017
Arefyev, N., Ermolaev P., Panchenko A., , in : Computational Linguistics and Intellectual Technologies. International Conference "Dialogue 2018" Proceedings. : M. : Conference Proceedings Editorial board, 2018. P. 68-84.
The paper describes our participation in the first shared task on word sense induction and disambiguation for the Russian language RUSSE'2018 [Panchenko et al., 2018]. For each of several dozens of ambiguous words, the participants were asked to group text fragments containing it according to the senses of this word, which were not provided beforehand, ...
Added: October 9, 2020
Ossadtchi A., Pronko P. K., Baillet S. et al., Frontiers in Neuroinformatics 2014 Vol. 7 No. 53 P. 1-11
Spatial component analysis is often used to explore multidimensional time series data whose sources cannot be measured directly. Several methods may be used to decompose the data into a set of spatial components with temporal loadings. Component selection is of crucial importance, and should be supported by objective criteria. In some applications, the use of ...
Added: January 19, 2014
Puzyrev D., Shelmanov A., Panchenko A. et al., , in : Proceedings of the 7th Workshop on Balto-Slavic Natural Language Processing, 2019, Florence, Italy, Association for Computational Linguistics. : Association for Computational Linguistics, 2019. P. 56-62.
aper presents the first gold-standard resource for Russian annotated with compositionality information of noun compounds. The compound phrases are collected from the Universal Dependency treebanks according to part of speech patterns, such as ADJ+NOUN or NOUN+NOUN, using the gold-standard annotations. Each compound phrase is annotated by two experts and a moderator according to the following ...
Added: October 30, 2019
Gharavi E., Veisi H., Россо П., Neural Computing and Applications 2020 Vol. 32 No. 14 P. 10593-10607
The efficiency and scalability of plagiarism detection systems have become a major challenge due to the vast amount of available textual data in several languages over the Internet. Plagiarism occurs in different levels of obfuscation, ranging from the exact copy of original materials to text summarization. Consequently, designed algorithms to detect plagiarism should be robust ...
Added: October 29, 2020
Kutuzov A. B., Velldal E., Øvrelid L., , in : Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning. : Berlin : Association for Computational Linguistics, 2016. P. 115-125.
This paper studies how word embeddings trained on the British National Corpus interact with part of speech boundaries. Our work targets the Universal PoS tag set, which is currently actively being used for annotation of a range of languages. We experiment with training classifiers for predicting PoS tags for words based on their embeddings. The ...
Added: November 12, 2016
Alexei Ossadtchi, Pronko P. K., Baillet S. et al., Frontiers in Neuroinformatics 2014 Vol. 7 No. January P. Article 53
Spatial component analysis is often used to explore multidimensional time series data whose sources cannot be measured directly. Several methods may be used to decompose the data into a set of spatial components with temporal loadings. Component selection is of crucial importance, and should be supported by objective criteria. In some applications, the use of ...
Added: January 29, 2014
Toldova S., Pisarevskaya D., Kobozeva M., , in : Artificial Intelligence and Natural Language, 7th International Conference, AINL 2018, St. Petersburg, Russia, October 17–19, 2018, Proceedings. Issue 930.: Switzerland : Springer, 2018. P. 79-87.
The identification of discourse connectives plays an important role in many discourse processing approaches. Among them there are functional words usually enumerated in grammars (iz-za ‘due to’, blagodarya ‘thanks to’,) and not grammaticalized expressions (X vedet k Y ‘X leads to Y’, prichina etogo ‘the cause is’). Both types of connectives signal certain relations between ...
Added: October 26, 2018
Yankovskaya A. E., Gorbunov I. V., Hodashinsky I. A., Pattern Recognition and Image Analysis 2021 Vol. 2 No. 27 P. 243-265
This paper starts a brief historical overview of occurrence and development of fuzzy systems and their applications. Integration methods are proposed to construct a fuzzy system using other AI methods, achieving synergy effect. Accuracy and interpretability are selected as main properties of rule-based fuzzy systems. The tradeoff between interpretability and accuracy is considered to be ...
Added: September 27, 2021