Resource-based job execution management

How to accelerate workload automation with smart resource targeting

2 min

Dec 5, 2019 from Veranika Tsiareshchanka

controller cpu by clyde he

A job scheduler executes workloads following a certain scheduling policy. An advanced job scheduling solution, like ProActive Workflows & Scheduling, supports several scheduling policies that determine how jobs and tasks will be scheduled. These include First-In-First-Out (FIFO), Earliest deadline first (EDF), and License-based policies. These policies are applied while taking into consideration jobs’ and tasks’ priorities. By default, ProActive Workflows & Scheduling uses the FIFO policy, but administrators can configure the proper policy that fits with their scheduling constraints.

When dealing with resource-consuming tasks (i.e., tasks that require certain resources such as CPU, memory or storage resources), ProActive resource manager puts forward the concept of Node Selection Scripts to target resource-enabled compute nodes. That is, a node selection script allows the scheduler to execute tasks on particular ProActive nodes. For instance, a selection script can specify that a task must be executed on a system with a certain memory threshold available. A selection script is always executed before the task itself on any candidate node, and the first node that satisfies the script constraints is elected.

ProActive Workflows & Scheduling further allows to limit the resources consumed for tasks’ executions. This can be achieved at two levels. First, at the level of global resources consumed by the scheduler on a compute infrastructure. Indeed, users can set the maximal number of ProActive nodes to be deployed on a specific infrastructure. Limiting the number of ProActive nodes implies limiting the number of tasks to be executed concurrently (on a given infrastructure), hence reducing resource consumption. Second, resource consumption can be controlled at a fine-grained level by limiting the resources used by ProActive nodes (CPU, memory, etc.). Accordingly, this limits the resources consumed by the tasks executed by the considered nodes.

More articles

All our articles