Structured Sparsification of Gated Recurrent Neural Networks

This paper examines echo state networks (ESNs), one of the most prevalent approaches to implementing reservoir computing. An ESN consists of a recurrent neural network with fixed (untrained) weights and a readout layer that is typically linear and trainable. This approach enables the creation of energyefficient and computationally efficient neural networks capable of real-time learning. However, since ...

Added: May 26, 2026

Ансамбль современных моделей компьютерного зрения для задачи обнаружения дипфейков

Pikul A. S., Безопасность информационных технологий 2024 Т. 31 № 4 С. 116–127

This article explores the potential use of modern computer vision architectures for the task of deepfake detection. The following architectures are considered: EfficientNet, Vision Transformer (ViT), VisionLSTM (ViL), Vision KAN, and Mamba Vision. The novelty of the approach lies in the application and comparison of these architectures, as well as their combination into paired ensembles ...

Added: December 12, 2025

Application of Large Language Models to Solving Differential Equations: Constructing Baseline Models with LSTM and GRU

Surkov A., Zakharov V., Sergei Koltcov et al., , in: Smart Technologies, Systems and Applications: 4th International Conference, SmartTech-IC 2024, Quito, Ecuador, December 2–4, 2024, Revised Selected Papers, Part IIVol. 2: Revised Selected Papers, Part II.: Springer, 2025. P. 239–252.

Currently, large language models are actively developing and beginning to be used to solve some mathematical problems. With the emergence of xLSTM model, which demonstrates the results comparable with transformer-based models, there has been a surge of interest in recurrent neural networks. This paper considers the application of baseline recurrent models such as LSTM and ...

Added: September 11, 2025

Prediction of Industrial Cyber Attacks Using Normalizing Flows

V.P. Stepashkina, M.I. Hushchyn, Doklady Mathematics 2024 Vol. 110 No. 1 P. S95–S102

This paper presents the development and evaluation of methods for detecting cyberattacks on industrial systems using neural network approaches. The focus is on the task of detecting anomalies in multivariate time series, where the diversity and complexity of potential attack scenarios require the use of advanced models. To address these challenges, a transformer-based autoencoder architecture ...

Added: March 25, 2025

The Appliance of Deep Neural Networks in the Process of Managing Chemical Enterprises

Kulyasova E. V., Kulyasov N.S., Puchkov A. Y., , in: Journal of Physics: Conference Series Volume 1260, 2019 Mechanical Science and Technology Update 23–24 April 2019, Omsk, Russian Federation.: IOP Publishing, 2019. Ch. 3 P. 032024–032024.

This article is introduced into the perspective tendencies of the digital transformation of chemical enterprises which allow to improve the process of managing enterprises of the branch. Presented the algorithms of managing and technological information processing based on deep neural network apparatus. New approaches to data processing known as video analytics are applied; it allows ...

Added: September 27, 2024

Using a Recurrent Neural Network To Inform the Use of Prostate- specific Antigen (PSA) and PSA Density for Dynamic Monitoring of the Risk of Prostate Cancer Progression on Active Surveillance

Sushentsev N., Abrego L., Colarieti A. et al., EUROPEAN UROLOGY OPEN SCIENCE 2023 Vol. 52 P. 36–39

The global uptake of prostate cancer (PCa) active surveillance (AS) is steadily increasing. While prostate-specific antigen density (PSAD) is an important baseline predictor of PCa progression on AS, there is a scarcity of recommendations on its use in follow-up. In particular, the best way of measuring PSAD is unclear. One approach would be to use ...

Added: February 28, 2024

Оптимизация физико-информированных нейронных сетей для решения нелинейного уравнения Шредингера

Чупров И. А., Гао Ц., Efremenko D. et al., Доклады Российской академии наук. Математика, информатика, процессы управления (ранее - Доклады Академии Наук. Математика) 2023 Т. 514 № 2 С. 28–38

Физико-информированные нейронные сети (Physics Informed Neural Networks – PINN) являются перспективным методом решения уравнений в частных производных с помощью машинного обучения. В работе рассмотрено применение PINN к нелинейному уравнению Шредингера для описания ...

Added: December 19, 2023

Научная конференция ЛОМОНОСОВСКИЕ ЧТЕНИЯ. Тезисы докладов. 15-25 апреля 2019 г.сборник

Захарова Т. В., Yuzhakov T., ООО «Макс Пресс», 2019.

В настоящий сборник вошли тезисы докладов секции Вычислительной математики и кибернетики конференции «Ломоносовские чтения‑2019», проводимой Московским государственным университетом имени М. В. Ломоносова в 2019 году. ...

Added: December 13, 2023

Loss function dynamics and landscape for deep neural networks trained with quadratic loss

Nakhodnov M., Kodryan M., Lobacheva E. et al., , in: Doklady MathematicsVol. 106. Issue 1: Supplement.: Pleiades Publishing, Ltd. (Плеадес Паблишинг, Лтд), 2023. P. 43–62.

Knowledge of the loss landscape geometry makes it possible to successfully explain the behavior of neural networks, the dynamics of their training, and the relationship between resulting solutions and hyperparameters, such as the regularization method, neural network architecture, or learning rate schedule. In this paper, the dynamics of learning and the surface of the standard ...

Added: June 9, 2023

Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes

Kodryan M., Lobacheva E., Nakhodnov M. et al., , in: Thirty-Sixth Conference on Neural Information Processing Systems : NeurIPS 2022.: Curran Associates, Inc., 2022. P. 14058–14070.

A fundamental property of deep learning normalization techniques, such as batch normalization, is making the pre-normalization parameters scale invariant. The intrinsic domain of such parameters is the unit sphere, and therefore their gradient optimization dynamics can be represented via spherical optimization with varying effective learning rate (ELR), which was studied previously. However, the varying ELR ...

Added: December 20, 2022

Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations

Belomestny D., Naumov A., Puchkin N. et al., Neural Networks 2023 Vol. 161 P. 242–253

This paper investigates the approximation properties of deep neural networks with piecewise-polynomial activation functions. We derive the required depth, width, and sparsity of a deep neural network to approximate any Hölder smooth function up to a given approximation error in Hölder norms in such a way that all weights of this neural network are bounded ...

Added: July 13, 2022

Прогнозирование энергопотребления на основе автоматического машинного обучения

Danilov K., Автоматизация. Современные технологии 2020 Т. 74 № август 2020 С. 402–407

Рассмотрена задача прогнозирования энергопотребления на основе автоматического машинного обучения. Приведена схема процесса автоматического создания и применения модели прогнозирова ния. Предлагаемый подход апробирован на основе данных о потреблении электроэнергии в регионах России. Проведённый вычислительный эксперимент показал высокую эффективность разработан ной модели. Точность прогнозирования составила 97...99 %. ...

Added: June 13, 2022

Self-supervised recurrent depth estimation with attention mechanisms

Makarov I., Bakhanova M., Nikolenko S. et al., PeerJ Computer Science 2022 Vol. 8 Article e865

Depth estimation has been an essential task for many computer vision applications, especially in autonomous driving, where safety is paramount. Depth can be estimated not only with traditional supervised learning but also via a self-supervised approach that relies on camera motion and does not require ground truth depth maps. Recently, major improvements have been introduced ...

Added: February 1, 2022

On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay

Lobacheva E., Kodryan M., Chirkova N. et al., , in: Advances in Neural Information Processing Systems 34 (NeurIPS 2021).: Curran Associates, Inc., 2021. P. 21545–21556.

Added: December 29, 2021

Gender domain adaptation for automatic speech recognition

Sokolov A., Savchenko A., , in: 2021 IEEE 19th World Symposium on Applied Machine Intelligence and Informatics (SAMI).: IEEE, 2021. P. 413–418.

This paper is focused on the finetuning of acoustic models for speaker adaptation goals on a given gender. We pretrained the Transformer baseline model on Librispeech-960 and conducted experiments with finetuning on the gender-specific test subsets. The obtained word error rate (WER) relatively to the baseline is up to 5% and 3% lower on male ...

Added: September 26, 2021

On the Embeddings of Variables in Recurrent Neural Networks for Source Code

Chirkova N., , in: 2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2021).: Association for Computational Linguistics, 2021. P. 2679–2689.

Source code processing heavily relies on the methods widely used in natural language processing (NLP), but involves specifics that need to be taken into account to achieve higher quality. An example of this specificity is that the semantics of a variable is defined not only by its name but also by the contexts in which ...

Added: August 31, 2021

Black-Box Optimization with Local Generative Surrogates

Belavin V., Ustyuzhanin A., Sergey Shirobokov et al., , in: Advances in Neural Information Processing Systems 33 (NeurIPS 2020).: Curran Associates, Inc., 2020. P. 14650–14662.

Added: February 14, 2021

Deep learning approach for predicting functional Z-DNA regions using omics data

Beknazarov N., Jin S., Poptsova M., Scientific Reports 2020 Vol. 10 P. 19134

Computational methods to predict Z-DNA regions are in high demand to understand the functional role of Z-DNA. The previous state-of-the-art method Z-Hunt is based on statistical mechanical and energy considerations about B- to Z-DNA transition using sequence information. Z-DNA CHiP-seq experiment results showed little overlap with Z-Hunt predictions implying that sequence information only is not ...

Added: December 11, 2020