Resource-based job execution management
How to accelerate workload automation with smart resource targeting
2 min
Dec 5, 2019 from Activeeon
A job scheduler executes workloads following a certain scheduling policy.
An advanced job scheduling solution, like ProActive Workflows & Scheduling, supports several scheduling policies that determine how jobs and tasks will be scheduled. These include First-In-First-Out (FIFO), Earliest deadline first (EDF), and License-based policies. These policies are applied while taking into consideration jobs’ and tasks’ priorities. By default, ProActive Workflows & Scheduling uses the FIFO policy, but administrators can configure the proper policy that fits with their scheduling constraints.
When dealing with resource-consuming tasks (i.e., tasks that require certain resources such as CPU, memory or storage resources), ProActive resource manager puts forward the concept of Node Selection Scripts to target resource-enabled compute nodes. That is, a node selection script allows the scheduler to execute tasks on particular ProActive nodes. For instance, a selection script can specify that a task must be executed on a system with a certain memory threshold available. A selection script is always executed before the task itself on any candidate node, and the first node that satisfies the script constraints is elected.
ProActive Workflows & Scheduling further allows to limit the resources consumed for tasks’ executions. This can be achieved at two levels. First, at the level of global resources consumed by the scheduler on a compute infrastructure. Indeed, users can set the maximal number of ProActive nodes to be deployed on a specific infrastructure. Limiting the number of ProActive nodes implies limiting the number of tasks to be executed concurrently (on a given infrastructure), hence reducing resource consumption. Second, resource consumption can be controlled at a fine-grained level by limiting the resources used by ProActive nodes (CPU, memory, etc.). Accordingly, this limits the resources consumed by the tasks executed by the considered nodes.