?
Large-scale transfer learning for natural language generation
P. 6053-6058.
Large-scale pretrained language models define state of the art in natural language processing, achieving outstanding performance on a variety of tasks. We study how these architectures can be applied and adapted for natural language generation, comparing a number of architectural and training schemes. We focus in particular on open-domain dialog as a typical high entropy generation task, presenting and comparing different architectures for adapting pretrained models with state of the art results.
In book
Association for Computational Linguistics, 2019
Polyakov E. V., Polyakov S. V., Abramov P., , in : Proceedings of 2019 XVI International Symposium "Problems of Redundancy in Information and Control Systems" (REDUNDANCY). : IEEE, 2019. P. 159-164.
Determining the tonality of the text is a difficult task, the solution of which essentially depends on the context, the field of study and the amount of text data. The analysis shows that the authors in their works do not jointly use the full range of possible transformations on the data and their combinations. The ...
Added: September 20, 2020
Tutubalina E., Алимова И. С., Мифтахутдинов З. et al., Bioinformatics 2021 Vol. 37 No. 2 P. 243-249
Drugs and diseases play a central role in many areas of biomedical research and healthcare. Aggregating knowledge about these entities across a broader range of domains and languages is critical for information extraction (IE) applications. To facilitate text mining methods for analysis and comparison of patient’s health conditions and adverse drug reactions reported on the ...
Added: January 13, 2021
Artemova E., , in : The Palgrave Handbook of Digital Russia Studies. : Palgrave Macmillan, 2021. Ch. 26. P. 465-481.
Deep learning is a term used to describe artificial intelligence (AI) technologies. AI deals with how computers can be used to solve complex problems in the same way that humans do. Such technologies as computer vision (CV) and natural language processing (NLP) are distinguished as the largest AI areas. To imitate human vision and the ...
Added: December 20, 2020
Association for Computational Linguistics, 2019
The 4th Workshop on Representation Learning for NLP (RepL4NLP) will be hosted by ACL 2019 and held on 2 August 2019. The workshop is being organised by Isabelle Augenstein, Spandana Gella, Sebastian Ruder, Katharina Kann, Burcu Can, Alexis Conneau, Johannes Welbl, Xian Ren and Marek Rei; and advised by Kyunghyun Cho, Edward Grefenstette, Karl Moritz ...
Added: November 1, 2019
Malykh V., Porplenko D., Tutubalina E., , in : Analysis of Images, Social Networks and Texts: 9th International Conference, AIST 2020, Skolkovo, Moscow, Russia, October 15–16, 2020, Revised Selected Papers. Vol. 12602.: Springer, 2021. P. 149-161.
We present a novel dataset of sports broadcasts with 8,781 games. The dataset contains 700 thousand comments and 93 thousand related news documents in Russian. We run an extensive series of experiments of modern extractive and abstractive approaches. The results demonstrate that BERT-based models show modest performance, reaching up to 0.26 ROUGE-1F-measure. In addition, human evaluation ...
Added: May 10, 2021
Cham : Springer, 2019
Intelligent Systems Conference (IntelliSys) 2018 is the fourth research conference in the series. This conference is a part of SAI conferences being held since 2013. The conference series has featured keynote talks, special sessions, poster presentation, tutorials, workshops, and contributed papers each year.
The conference focus on areas of intelligent systems and artificial intelligence (AI) and ...
Added: August 29, 2018
Gozuacik N., Sakar C. O., Ozcan S., Expert Systems with Applications 2021 Vol. 183 No. 30 November 2021 P. 1-13
Social media platforms are considered one of the most effective intermediaries for companies to interact with consumers. Social media-based decision support systems for the marketing domain are highly developed, but product development and innovation-oriented studies remain limited. This study offers a novel approach which utilises opinion retrieval theme along with sentiment analysis to support the ...
Added: December 12, 2021
Горшков И. А., Dolgaleva I., Труды Института системного программирования РАН 2018 Т. 29 № 4 С. 325-336
Nowadays, news portals are forced to seek new methods of engaging the audience due to the increasing competition in today’s mass media. The growth in the loyalty of news service consumers may further a rise of popularity and, as a result, additional advertising revenue. Therefore, we propose the tool that is intended for stylish presenting ...
Added: October 31, 2018
Polyakov E. V., Voskov L., Abramov P. et al., Informatsionno-upravliaiushchie sistemy [Information and Control Systems] 2020 No. 1 P. 2-14
Introduction: Sentiment analysis is a complex problem whose solution essentially depends on the context, field of study and amount of text data. Analysis of publications shows that the authors often do not use the full range of possible data transformations and their combinations. Only a part of the transformations is used, limiting the ways to ...
Added: February 20, 2020
Ilia Karpov, Nick Kartashev, , in : Analysis of Images, Social Networks and Texts. 10th International Conference, AIST 2021, Tbilisi, Georgia, December 16–18, 2021, Revised Selected Papers. : Cham : Springer, 2022. P. 1-10.
The ubiquity of the contemporary language understanding tasks gives relevance to the development of generalized, yet highly efficient models that utilize all knowledge, provided by the data source. In this work, we present SocialBERT - the first model that uses knowledge about the author’s position in the network during text analysis. We investigate possible models ...
Added: October 31, 2021
Golovanov S., Tselousov A., Rauf Kurbanov et al., , in : The NeurIPS '18 Competition: From Machine Learning to Intelligent Conversations. : Springer, 2020. P. 295-315.
Added: February 20, 2021
Cham : Springer, 2022
This book constitutes revised selected papers from the 9th International Conference on Analysis of Images, Social Networks and Texts, AIST 2020, held during December 16-18, 2021. The world of Data Science changes every year. At AIST, we exchange our understanding of the Science state-of-the-art, as well as how it applies to life and business. AIST ...
Added: January 4, 2022
Nikolaev K., Malafeev A., , in : Analysis of Images, Social Networks and Texts. 7th International Conference AIST 2018. : Springer, 2018. Ch. 12. P. 121-126.
This paper deals with automatic classification of questions in the Russian language. In contrast to previously used methods, we introduce a convolutional neural network for question classification. We took advantage of an existing corpus of 2008 questions, manually annotated in accordance with a pragmatic 14-class typology. We modified the data by reducing the typology to ...
Added: February 15, 2019
Switzerland : Springer, 2019
This book constitutes the refereed proceedings of the 11th International Conference on Intelligent Data Processing, IDP 2016, held in Barcelona, Spain, in October 2016.
The 11 revised full papers were carefully reviewed and selected from 52 submissions. The papers of this volume are organized in topical sections on machine learning theory with applications; intelligent data processing in life ...
Added: February 8, 2020
Alimova l., Tutubalina E., Journal of Biomedical Informatics 2020 Vol. 103 P. 1-9
Relation extraction aims to discover relational facts about entity mentions from plain texts. In this work, we focus on clinical relation extraction; namely, given a medical record with mentions of drugs and their attributes, we identify relations between these entities. We propose a machine learning model with a novel set of knowledge-based and BioSentVec embedding ...
Added: October 28, 2020
Berlin : Springer, 2014
This book constitutes the proceedings of the Third International Conference on Analysis of Images, Social Networks and Texts, AIST 2014, held in Yekaterinburg, Russia, in April 2014. The 11 full and 10 short papers were carefully reviewed and selected from 74 submissions. They are presented together with 3 short industrial papers, 4 invited papers and ...
Added: November 13, 2014
Atanov A., Ashukha A., Struminsky K. et al., , in : Proceedings of the 7th International Conference on Learning Representations (ICLR 2019). : ICLR, 2019. P. 1-17.
Bayesian inference is known to provide a general framework for incorporating prior knowledge or specific properties into machine learning models via carefully choosing a prior distribution. In this work, we propose a new type of prior distributions for convolutional neural networks, deep weight prior (DWP), that exploit generative models to encourage a specific structure of ...
Added: September 2, 2019
Koch S., Matveev A., Jiang Z. et al., , in : Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2019). : IEEE, 2019. P. 9601-9611.
We introduce ABC-Dataset, a collection of one million Computer-Aided Design (CAD) models for research of geometric deep learning methods and applications. Each model is a collection of explicitly parametrized curves and surfaces, providing ground truth for differential quantities, patch segmentation, geometric feature detection, and shape reconstruction. Sampling the parametric descriptions of surfaces and curves allows ...
Added: November 26, 2019
М. : Издательский центр «Российский государственный гуманитарный университет», 2019
The book includes 64 papers submitted to the International conference in computer linguistics and intellectual technologies Dialogue 2019 and presents a broad spectrum of theoretical and applied research of natural language description, language simulation, and creation of applied computer technologies. ...
Added: October 16, 2019
Lobacheva E., Chirkova N., Vetrov D., / International Conference on Machine Learning. Series 1 "Workshop on Learning to Generate Natural Language". 2017.
Recurrent neural networks show state-of-the-art results in many text analysis tasks but often require a lot of memory to store their weights. Recently proposed Sparse Variational Dropout (Molchanov et al., 2017) eliminates the majority of the weights in a feed-forward neural network without significant loss of quality. We apply this technique to sparsify recurrent neural ...
Added: October 19, 2017
Morozov N., Rakitin D., Oleg Desheulin et al., , in : Neural Fields across Fields: Methods and Applications of Implicit Neural Representations. ICLR 2023 Workshop. : [б.и.], 2023. Ch. 8.
In view synthesis, a neural radiance field approximates underlying density and radiance fields based on a sparse set of scene pictures. To generate a pixel of a novel view, it marches a ray through the pixel and computes a weighted sum of radiance emitted from a dense set of ray points. This rendering algorithm is ...
Added: July 18, 2023
Durandin O., Malafeev A., Zolotykh N., , in : Analysis of Images, Social Networks and Texts. 6th International Conference, 2017, Revised Selected Papers. Vol. 10716.: Cham : Springer, 2018. Ch. 4. P. 34-46.
The paper deals with Google’s universal parser SyntaxNet. The system was used to analyze the Universal Dependencies linguistic corpora. We conducted an error analysis of the output of the parser to reveal to what extent the error types are connected with or preconditioned by the language types. In particular, we carried out several experiments, clustering ...
Added: December 1, 2017
Фирсанова В. И., International Journal of Open Information Technologies 2021 Vol. 9 No. 12 P. 53-59
The paper presents a study on question answering systems evaluation. The purpose of the study is to determine if human evaluation is indeed necessary to qualitatively measure the performance of a sociomedical dialogue system. The study is based on the data from several natural language processing experiments conducted with a question answering dataset for inclusion of people with autism spectrum disorder and state-of-the-art ...
Added: September 25, 2023
Davletov A., Gordeev D., Nikolay Arefyev et al., , in : Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021). : Association for Computational Linguistics, 2021. P. 1249-1254.
This work describes our approach for subtasks of SemEval-2021 Task 8: MeasEval: Counts and Measurements which took the official first place in the competition. To solve all subtasks we use multi-task learning in a question-answering-like manner. We also use learnable scalar weights to weight subtasks’ contribution to the final loss in multi-task training. We fine-tune ...
Added: September 23, 2021