?
Supporting Efficient Execution of Many-Task Applications with Everest
P. 266-270.
Distributed computing systems are widely used for execution of loosely coupled many-task applications. There are two important classes of such applications. Bag-of-tasks applications, eg, parameter sweeps or Monte Carlo simulations, represent a set of independent tasks. Workflows, which are used for automation of complex computational and data processing pipelines, consist of multiple tasks with control or data dependencies. The paper discusses the common problems related to the efficient execution of such applications on distributed computing resources and the relevant solutions implemented within the Everest platform.
Language:
English
Sukhoroslov O. V., Voloshinov V., Smirnov S., , in : Supercomputing. RuSCDays 2020. Communications in Computer and Information Science. Vol. 1331: 6th Russian Supercomputing Days, RuSCDays 2020, Moscow, Russia, September 21–22, 2020, Revised Selected Papers.: Switzerland : Springer, 2020. P. 634-646.
Added: October 29, 2021
Smirnov S., Sukhoroslov O. V., Voloshinov V., , in : Supercomputing. RuSCDays 2018. Communications in Computer and Information Science, vol 965. Springer, Cham. : Springer, 2019. P. 687-698.
High-performance computing plays an increasingly important role in modern science and technology. However, the lack of convenient interfaces and automation tools greatly complicates the widespread use of HPC resources among scientists. The paper presents an approach to solving these problems relying on Everest, a web-based distributed computing platform. The platform enables convenient access to HPC ...
Added: October 19, 2019
Sukhoroslov O. V., Nazarenko A., Aleksandrov R., The Journal of Supercomputing 2019 Vol. 75 No. 12 P. 7857-7871
The paper studies the performance of algorithms for scheduling of many-task applications in distributed computing systems. Two important classes of such applications are considered: bags-of-tasks and workflows. The comparison of algorithms is performed on the basis of discrete-event simulation for various application cases and system configurations. The developed simulation framework based on SimGrid toolkit provides ...
Added: August 27, 2018
Toporkov V. V., Toporkova A. S., Tselishev A. undefined. et al., , in : Proceedings of the 2010 ACS/IEEE International Conference on Computer Systems and Applications, Hammamet, Tunisia, May 16-19th, 2010. : Los Alamitos : IEEE CS Pre, 2010. P. 18-25.
In this paper, we present an approach to scalable co-scheduling in distributed computing for complex sets of interrelated tasks(jobs). The scalability means that schedules are formed for job models with various levels of task granularity, data replication policies, and processor resource and memory can be upgraded. The necessary of guaranteed job execution at the required ...
Added: April 20, 2012
Toporkov V., Yemelyanov D., Potekhin P. et al., Computing and Informatics 2015 Vol. 34 No. 1 P. 45-76
In this paper, we address problems of efficient computing in distributed systems with non-dedicated resources including utility grid. There are global job flows from external users along with resource owner's local tasks upon the resource non-dedication condition. Competition for resource reservation between independent users, local and global job flows substantially complicates scheduling and the requirement ...
Added: July 10, 2015
Sukhoroslov O. V., Journal of Parallel and Distributed Computing 2018 Vol. 118 No. 1 P. 177-188
The paper presents an approach to the design and implementation of web-based environments for practical exercises in parallel and distributed computing (PDC). The presented approach introduces minimal development and operational costs by relying on Everest, a general-purpose platform for building computational web services. The flexibility of proposed service-oriented architecture enables the development of different types ...
Added: August 27, 2018
Toporkov V., Yemelyanov D., Anna Toporkova et al., , in : ICIT 2013.The 6th International Conference on Information Technology. Cloud Computing. May 8th-10th, 2013, Amman, Jordan, IEEE Jordan Chapter, ICIT 2013. : Jordan : IEEE Computer Society, 2013. Ch. 697. P. 1-11.
In this work, we introduce slot selection and co-allocation algorithms for parallel jobs in distributed computing with non-dedicated resources. A single slot is a time span that can be assigned to a task, which is a part of a job. The job launch requires a co-allocation of a specified number of slots starting synchronously. The ...
Added: November 29, 2013
Association for Computing Machinery (ACM), 2021
Welcome to the 40th ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing (PODC 2021), held virtually (due to the COVID-19 pandemic) on July 26-30, 2021. PODC is the premier forum for presentation of research on all aspects of distributed computing, including the theory, design, implementation, and applications of distributed algorithms, systems, and networks.
This volume contains ...
Added: October 14, 2021
Toporkov V., Anna Toporkova, Tselishchev A. et al., Procedia Computer Science 2015 Vol. 51 No. 1 P. 845-854
Distributed environments with the decoupling of users from resource providers are generally termed as utility Grids. The paper focuses on the problems of efficient job flow distribution and scheduling in virtual organizations (VOs) of utility Grids while ensuring the VO stakeholders preferences and providing dependable strategies for resources utilization. An approach based on the combination ...
Added: June 1, 2015
Топорков В. В., Емельянов Д. М., Toporkova A. S., Информационные технологии в науке, образовании и управлении 2017 № 3 С. 72-80
This paper deals with problems of scheduling in distributed environments with non-dedicated resources
in Grid. ...
Added: January 26, 2018
Cham : Springer, 2016
This book constitutes thoroughly revised selected papers of the 6th International Conference on Numerical Analysis and Its Applications, NAA 2016, held in Lozenetz, Bulgaria, in June 2016. The 90 revised papers presented were carefully reviewed and selected from 98 submissions. The conference offers a wide range of the following topics: Numerical Modeling; Numerical Stochastics; Numerical Approx-imation ...
Added: August 30, 2016
Toporkov V., Toporkova A. S., Tselishchev A. et al., , in : Intelligent Distributed Computing VII. Proceedings of the 7th International Symposium on Intelligent Distributed Computing - IDC 2013, Prague, Czech Republic, September 2013. Vol. 511: Studies in Computational Intelligence.: Dordrecht, L., Cham, Heidelberg, NY : Springer, 2014. P. 109-118.
In this work, we introduce heuristic slot selection and co-allocation strategies for parallel jobs in distributed computing with non-dedicated and heterogeneous resources (clusters, CPU nodes equipped with multicore processors, networks etc.). A single slot is a time span that can be assigned to a task, which is a part of a job. The job launch ...
Added: March 13, 2015
Kazeev N., Ustyuzhanin A., Artemov A. et al., Journal of Physics: Conference Series 2015 Vol. 664 No. 3 P. 032019-032019
During LHC Run 1, the LHCb experiment recorded around 1011 collision events. This paper describes Event Index — an event search system. Its primary function is to quickly select subsets of events from a combination of conditions, such as the estimated decay channel or number of hits in a subdetector. Event Index is essentially Apache ...
Added: October 24, 2016
Smirnov S., Voloshinov V., O.V. Sukhoroslov, , in : Proceedings of the 9th International Conference "Distributed Computing and Grid Technologies in Science and Education" (GRID'2021), Dubna, Russia, July 5-9, 2021. : CEUR Workshop Proceedings, 2021. P. 413-417.
ParaSCIP is rather advanced open-source solver for discrete and global optimization problems. This solver is distinguished by that it can run on distributed memory systems and use up to 80,000 cores, solving open problems from the MIPLIB test libraries. Earlier, using this solver, we confirmed the conjecture on optimal packing of nine congruent circles on ...
Added: October 30, 2022
Springer, 2021
10th International Conference, IACC 2020, Panaji, Goa, India, December 5–6, 2020, Revised Selected Papers, Part II
series: Communications in Computer and Information Science (2021) volume 1368 ...
Added: July 7, 2021
Dagstuhl Publishing, 2021
Welcome to the DISC 2021, the 35th International Symposium on Distributed Computing, held on October 4–18, 2021. DISC is an international forum on the theory, design, analysis, and implementation of distributed systems and networks, focusing on distributed computing in all its forms. DISC is organized in cooperation with the European Association for Theoretical Computer Science ...
Added: October 14, 2021
Dordrecht, L., Cham, Heidelberg, NY : Springer, 2014
This book represents the combined peer-reviewed proceedings of the Seventh International Symposium on Intelligent Distributed Computing - IDC-2013, of the Second Workshop on Agents for Clouds - A4C-2013, of the Fifth International Workshop on Multi-Agent Systems Technology and Semantics - MASTS-2013, and of the International Workshop on Intelligent Robots - iR-2013. All the events were ...
Added: March 13, 2015
NY : ACM, 2016
Added: August 30, 2018
Vostokin S., Popov S., O. Sukhoroslov, , in : Proceedings of the 9th International Conference "Distributed Computing and Grid Technologies in Science and Education" (GRID'2021), Dubna, Russia, July 5-9, 2021. : CEUR Workshop Proceedings, 2021. P. 558-561.
The paper describes the experience of building distributed web applications based on the interactive computing technologies of the Jupyter project. The new architecture of such applications is proposed, considering the possibility of deploying a Jupyter notebook server separately from computing resources, and the possibility to interact with several computing resources simultaneously. These features are implemented ...
Added: October 30, 2022
Toporkov V. V., Toporkova A. S., Bobchenkov A. V. et al., , in : Advances in Information Technology from AI to Virtual Reality. : Amman : UbiCC Publisher, 2011. P. 2-20.
In this work, we present slot selection algorithms for job batch scheduling in distributed computing with non-dedicated resources. Jobs are parallel applications and these applications are independent. Existing approaches towards resource co-allocation and parallel job scheduling in economic models of distributed computing are based on search of time-slots in resource occupancy schedules. The sought time-slots ...
Added: April 12, 2012
Toporkov V., Anna Toporkova, Tselishchev A. et al., , in : Proceedings of the Ninth International Conference on Dependability and Complex Systems DepCoS-RELCOMEX. June 30 – July 4, 2014, Brunów, Poland. Vol. 286: Advances in Intelligent Systems and Computing.: Dordrecht, L., Cham, Heidelberg, NY : Springer, 2014. P. 455-465.
The paper is devoted to comparing the results of an independent job batch scheduling in terms of a virtual organization policy and available resources usage efficiency in large distributed environments like utility Grid. A hybrid approach is proposed on the basis of a cyclic scheduling scheme and backfilling combination. Additionally the paper offers a heuristic ...
Added: July 8, 2014
Abdullin A., Stepanov D., Akhin M., , in : Communications in Computer and Information Science. Vol. 779: Tools and Methods of Program Analysis.: Springer, 2018. P. 1-10.
Software analysis is becoming increasingly important as a way of software quality assurance. Most works in this area focus their attention on a single machine scenario, when the analysis is run and implemented on a single processing node, as it seems to be a good fit for the current software development methodologies. We argue that ...
Added: November 12, 2018
Ponomarenko A., Yury M., Andrey L. et al., Business Informatics 2014
The ability to scale is desirable in computer system as well as business settings. The distributed systems clearly demonstrate this ability and powerfulness to process a very big amount of data. Many system that have distributed architecture like Hadoop file system or distributed torrent tracker are based on the distribute hash table (DHT) which manages ...
Added: October 20, 2014
M. : ISC RAS, 2020
The book presents proceedings of the XXIII International Scientific Conference "Distributed computer and communication networks: control, computation, communications (DCCN-2020)" ...
Added: October 31, 2020