Design and evaluation of scalable approaches to schedule a stream of batch jobs in large-scale grids

I think the grid computing stimulates the cooperation among people, that agree to share resources and knowledge, to address problems that are not reachable individually. The central purpose of grid computing is to enable a community of users to perform work on a pool of shared resources. Since the number of jobs to be done in most cases outnumbers the number of available resources, somebody must decide how to allocate resources to jobs. Historically, this has been known as the Scheduling Problem. A large amount of research in scheduling was motivated by the proliferation of massively parallel machines in the early 1990s, and the desire to use these very expensive resources as efficiently as possible. Nowadays, grid is the emerging computing platform. Grid computing makes the scheduling problem even more difficult since, due to its heterogeneous and highly distributed nature, it cannot be served by a single centralized scheduling framework. In this Thesis we propose the study conducted to design and evaluate a hierarchical framework to dynamically schedule a continuous stream of independent, batch jobs, on large-scale grids. Our scheduler aims to schedule arriving jobs respecting their computational requirements (hardware constraints, software resources needed to be executed, deadlines), and optimizing the utilization of hardware and software resources. We designed a lightweight Meta-Scheduler able to classify incoming jobs according to their relevance, and to schedule jobs among underlying resources balancing the workload among them. Furthermore, we designed different algorithms as Local-Schedulers, which are able to carry out the job machine associations using dynamic information about the environment. Eventually, the Thesis presents two important related results obtained by scientific collaborations. The former concerns to the study of the self-optimizing system behavior. The latter concerns to the comparison of one of our Local-Scheduler solution with a Schedule-Based algorithm proposed by Dalibor Klus´aˇcek and Hana Rudov´a of the University of Brno.