Two-Stage Genetic-Based Optimization for Resource Provisioning and Scheduling of Multiple Workflows on the Cloud Under Resource Constraints

Abstract

Resource provisioning and scheduling are essential challenges in handling multiple workflow requests within cloud environments, particularly given the constraints imposed by limited resource availability. Although workflow scheduling has been extensively studied, few methods effectively integrate resource provisioning with scheduling, especially under cloud resource limitations and the complexities of multiple workflows. To address this challenge, we propose an innovative two-stage genetic-based optimization approach. In the first stage, candidate cloud resources are selected for the resource pool under the given resource constraints. In the second stage, these resources are provisioned and task scheduling is optimized on the selected resources. A key advantage of our approach is that it reduces the search space in the first stage through a novel encoding scheme that enables a caching strategy, in which intermediate results are stored and reused to enhance optimization efficiency in the second stage. The proposed solution is evaluated through extensive simulation experiments, assessing both resource selection and task scheduling across a diverse range of workflows. The results demonstrate that the proposed approach outperforms existing algorithms, particularly for highly parallel workflows, highlighting its effectiveness in managing complex workflow scheduling under resource-constrained cloud environments.