Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization

Demidovich Y.; Petr Ostroukhov; Malinovsky G.; Horvath S.; Takáč M.; Richtárik P.; E. Gorbunov

doi:10.48550/arXiv.2412.02781

Публикации

?

Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization

Demidovich Y., Petr Ostroukhov, Malinovsky G., Horvath S., Takáč M., Richtárik P., Горбунов Э. А.

Язык: английский

DOI

Текст на другом сайте

Ключевые слова: distributed optimization Federated learning Optimizations Random Reshuffling

В книге

The Thirteenth International Conference on Learning Representations: ICLR 2025

ICLR, 2025.

Efficient Conformal Prediction under Data Heterogeneity

Plassier V., Kotelevskii N., Rubashevskii A. и др., , in: Proceedings of The 27th International Conference on Artificial Intelligence and Statistics (AISTATS 2024), 2-4 May 2024, Palau de Congressos, Valencia, Spain. PMLR: Volume 238Vol. 238.: Valencia: PMLR, 2024. P. 4879–4887.

Добавлено: 30 мая 2024 г.

Queuing dynamics of asynchronous Federated Learning

Leconte L., Jonckheere M., Самсонов С. В. и др., , in: Proceedings of The 27th International Conference on Artificial Intelligence and Statistics (AISTATS 2024), 2-4 May 2024, Palau de Congressos, Valencia, Spain. PMLR: Volume 238Vol. 238.: Valencia: PMLR, 2024. P. 1711–1719.

Добавлено: 26 мая 2024 г.

Federated Learning Strategies Over Wireless Channels

Али А., , in: 22nd International Conference, NEW2AN 2022, Tashkent, Uzbekistan, December 15–16, 2022, Proceedings. Internet of Things, Smart Spaces, and Next Generation Networks and Systems. LNCS, volume 13772Issue 13772.: Springer, 2023. P. 525–533.

Добавлено: 18 мая 2023 г.

Adaptation Diffusion Strategy Over Wireless Fading Channels

Али А., , in: 2022 International Conference on Smart Applications, Communications and Networking (SmartNets).: IEEE, 2022. P. 1–4.

Добавлено: 16 мая 2023 г.

АДАПТАЦИЯ СТРАТЕГИЯ ДИФФУЗИИ ПО БЕСПРОВОДНЫМ КАНАЛАМ С ЗАМИРАНИЕМ

Али А., Кучерявый Е. А., Эбрахим А., В кн.: Инновационные, информационные и коммуникационные технологии. Сборник трудов XIX Международной научно-практической конференции.: М.: Ассоциация выпускников и сотрудников ВВИА им. проф. Жуковского, 2022. С. 38–42.

Добавлено: 22 февраля 2023 г.

Decentralized personalized federated learning: Lower bounds and optimal algorithm for all personalization modes

Sadiev A., Borodich E., Безносиков А. Н. и др., EURO Journal on Computational Optimization 2022 Vol. 10 Article 100041

Добавлено: 28 октября 2022 г.

Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices

Рябинин М. К., Горбунов Э. А., Plokhotnyuk V. и др., , in: Advances in Neural Information Processing Systems 34 (NeurIPS 2021).: Curran Associates, Inc., 2021. P. 18195–18211.

Добавлено: 1 февраля 2022 г.

MARINA: Faster Non-Convex Distributed Learning with Compression

Горбунов Э. А., Burlachenko K., Li Z. и др., , in: Proceedings of the 38th International Conference on Machine Learning (ICML 2021)Vol. 139.: PMLR, 2021. Ch. 139 P. 3788–3798.

Добавлено: 25 октября 2021 г.

Local SGD: Unified Theory and New Efficient Methods

Горбунов Э. А., Hanzely F., Richtarik P., , in: International Conference on Artificial Intelligence and Statistics, 13-15 April 2021, VirtualVol. 130.: PMLR, 2021. Ch. 130 P. 3556–3564.

Добавлено: 25 октября 2021 г.

Derivative-Free Method For Composite Optimization With Applications To Decentralized Distributed Optimization

Безносиков А. Н., Горбунов Э. А., Гасников А. В., IFAC-PapersOnLine 2021 Vol. 53 No. 2 P. 4038–4043

Добавлено: 14 октября 2021 г.

Federated Learning in Named Entity Recognition

Efim Luboshnikov, Макаров И. А., , in: Recent Trends in Analysis of Images, Social Networks and Texts. 9th International Conference, AIST 2020, Skolkovo, Moscow, Russia, October 15–16, 2020 Revised Supplementary ProceedingsVol. 12602.: Springer, 2021. Ch. 8 P. 90–101.

This article is devoted to the implementation of the federated approach to named entity recognition. The novel federated approach is designed to solve data privacy issues. The classic BiLSTM-CNNs-CRF and its modifications trained on a single machine are taken as baseline. Federated training is conducted for them. Influence of use of pretrained embedding, use of ...

Добавлено: 24 марта 2021 г.

Linearly Converging Error Compensated SGD

Eduard Gorbunov, Kovalev D., Makarenko D. и др., , in: Advances in Neural Information Processing Systems 33 (NeurIPS 2020).: Curran Associates, Inc., 2020. P. 20889–20900.

Добавлено: 7 декабря 2020 г.

Towards accelerated rates for distributed optimization over time-varying networks

Рогозин А. В., Lukoshkin V., Гасников А. В. и др., / Series arXiv "math". 2020.

Добавлено: 7 октября 2020 г.

Optimal distributed convex optimization on slowly time-varying graphs

Рогозин А. В., Uribe C., Гасников А. В. и др., IEEE Transactions on Control of Network Systems 2020 Vol. 7 No. 2 P. 829–841

Добавлено: 7 октября 2020 г.

On the Complexity of Approximating Wasserstein Barycenters

Крошнин А. В., Tupitsa Nazarii, Двинских Д. М. и др., , in: Proceedings of Machine Learning ResearchVol. 97: International Conference on Machine Learning, 9-15 June 2019, Long Beach, California, USA.: PMLR, 2019. P. 3530–3540.

We study the complexity of approximating the Wasserstein barycenter of m discrete measures, or histograms of size n, by contrasting two alternative approaches that use entropic regularization. The first approach is based on the Iterative Bregman Projections (IBP) algorithm for which our novel analysis gives a complexity bound proportional to $m n^2 / \epsilon^2$ to approximate the original non-regularized barycenter. ...

Добавлено: 11 июня 2019 г.