Book chapter
YT – эволюция системы распределенных вычислений
In book
GridMD is a C++ class library intended for constructing simulation applications and running them in
distributed environments. The library abstracts away from details of distributed environments, so that almost no knowledge of distributed computing is required from a physicist working with the library. She or he just uses GridMD function calls inside the application C++ code to perform parameter sweeps or other tasks that can be distributed at run-time. In this paper we briefly review the GridMD architecture. We also describe the job manager component which submits jobs to a remote system. The C++ source code of our PBS job manager may be used as a standalone tool and it is freely available as well as the full library source code. As illustrative examples we use simple expression evaluation codes and the real application of Coulomb cluster explosion simulation by Molecular Dynamics.
In this work, we introduce slot selection and co-allocation algorithms for parallel jobs in distributed computing with non-dedicated and heterogeneous resources. A single slot is a time span that can be assigned to a task, which is a part of a job. The job launch requires a co-allocation of a specified number of slots starting synchronously. The challenge is that slots associated with different resources of distributed computational environments may have arbitrary start and finish points that do not match. Some existing algorithms assign a job to the first set of slots matching the resource request without any optimization (the first fit type), while other algorithms are based on an exhaustive search. In this paper, algorithms for effective slot selection of linear complexity on an available slots number are studied and compared with known approaches. The novelty of the proposed approach consists of allocating alternative sets of slots. It provides possibilities to optimize job scheduling.
This chapter describes an economic model for independent job flow management in distributed computing environments with non-dedicated resources. The model is based on the concept of fair resource distribution between users and owners of computational nodes by means of economic mechanisms in a virtual organization. Scheduling is performed in cycles in accordance with dynamically updated schedules on local processor nodes. Schedule optimization is performed using dynamic programming methods using the set of criteria in accordance with the economic policy of the virtual organization.
This book constitutes the proceedings of the 12th International Conference on Parallel Computing Technologies, PaCT 2013, held in St. Petersburg, Russia, during September 30-October 4, 2013. The 41 full papers presented together with 2 invited papers were carefully reviewed and selected from 83 submissions. The papers are organized in topical sections on all technological aspects of the applications of parallel computer systems High level parallel programming languages and systems, methods and tools for parallel solution of large-scale problems, languages, environments and software tools supporting parallel processing, operating systems, scheduling, mapping, load balancing, general architectural concepts, cellular automata, performance measurement and analysis tools, teaching parallel processing, software for grid and cloud computing, scalable computing, fragmentation and aggregation of algorithms and programs as well as programs assembling and reuse.
This chapter describes an economic model for independent job flow management in distributed computing environments with non-dedicated resources. The model is based on the concept of fair resource distribution between users and owners of computational nodes by means of economic mechanisms in a virtual organization. Scheduling is performed in cycles in accordance with dynamically updated schedules on local processor nodes. Schedule optimization is performed using dynamic programming methods using the set of criteria in accordance with the economic policy of the virtual organization.
This work presents slot selection algorithms in economic models for independent job batch scheduling in distributed computing with non-dedicated resources. Existing approaches towards resource co-allocation and multiprocessor job scheduling in economic models of distributed computing are based on search of time-slots in resource occupancy schedules. The sought time-slots must match requirements of necessary span, computational resource properties, and cost. Usually such scheduling methods consider only suited variant of time-slot set. This work discloses a scheduling scheme that features multi-variant search. Two algorithms of linear complexity for search of alternative variants are proposed and compared. Having several optional resource configurations for each job makes an opportunity to perform an optimization of execution of the whole batch of jobs and to increase overall efficiency of scheduling.
The structure of the RISA GRID segment, that is implemented in the Russian Academy of Sciences and based on the computational clusters of the Joint Supercomputer Center, has been described. The structure and the main points of the components interaction have been discussed. The functioning of the computing systems being a part of GRID has been described. The basic principles of the network environment for distributed computation software design and operation have been presented.
In this work, we present slot selection algorithms for job batch scheduling in distributed computing with non-dedicated resources. Jobs are parallel applications and these applications are independent. Existing approaches towards resource co-allocation and parallel job scheduling in economic models of distributed computing are based on search of time-slots in resource occupancy schedules. The sought time-slots must match requirements of necessary span, computational resource properties, and cost. Usually such scheduling methods consider only one suited variant of time-slot set. This work discloses a scheduling scheme that features multi-variant search. Two algorithms of linear complexity for search of alternative variants are proposed. Having several optional resource configurations for each job makes an opportunity to perform an optimization of execution of the whole batch of jobs and to increase overall efficiency of scheduling.