Features - ProActive Workflows & Scheduling

Job scheduling software

At a glance

ProActive Workflows & Scheduling is an open source job scheduling software for IT automation that distributes workloads and applications to accelerate their execution. The solution includes a workflow studio, a scheduler, a resource manager, a job planner, analytics dashboards and many other advanced features for enterprise use

The scalability of ProActive Workflows & Scheduling solution has been tested on over 20000 CPU cores, and demonstrated resource elasticity, automatically scaling up and down depending on the workload.

Features are available through GUI, REST API or CLI.

Workflows & Scheduling product sheet

Workflow Studio

ProActive Workflow Studio

Easily design efficient workflows

Workflow Studio is a workflow orchestrator allowing to set dependencies between multi-language tasks. It includes a catalog with pre-built workflows for various business needs (big data analysis, machine learning, finance, data connectors, etc.).

The Workflow Studio provides an easy to use graphical interface for creating workflows as representation of your distributed computations. Workflows can be composed of one or more tasks and their dependencies. You can set advanced flow controls such as replication, if/else branching and loops, between these tasks.

Workflow Studio exists in two versions: a multi-platform native client (compatible with Windows, Linux and MacOS X), and a web interface directly accessible from your web browser.

Tasks can be written in shell, linux bash, windows cmd, docker, kubernetes, java, scala, javascript, groovy, ruby, python, perl, php, powershell, R, etc.

The Scheduler

Everything from your web browser

The Scheduler and it’s web portal are dedicated to users running workflows. The scheduler executes submitted workflows as jobs. The scheduler web portal allows to monitor execution and retrieve job results. The progress of workflow execution is graphically visible in the portal.

The scheduler portal allows to see job status, as well as to set and edit job priorities. You can filter queue visualization by job priority, workflow name, job number, user name, state and aven more advanced parameters. The scheduler portal also allows to visualize the details of each task the job is composed of.

The log panelboard eases job output visualization and allows to fetch all logs with possibility to filter by displaying only errors or only output, or both. The logs can also be downloaded from the portal.

ProActive Workflow Studio

Resource Manager

ProActive Workflow Studio

Aggregate resources to reduce infrastructure costs

The Resource Manager web portal allows administrators to set up infrastructure policies, and users to monitor their resources. It is the place where you can connect compute resources, and even control dynamic policy-based provisioning of resources.

Workflows & Sheduling is cross-plaftorm and cloud-oriented, and allows to manage heterogeneous computing resources: on-premises, clusters, clouds. With its advanced resource manager, you can aggregate resources into hybrid infrastructures and easily manage them using this resource-agnostic solution. Resources coming from multiple origins are unified as ProActive Nodes (usually tied to the number of cores available on a compute host) and can be accessed transparently: desktop machines (Windows, Linux, MacOS X), all kinds of standalone server machines, cluster nodes managed by common batch schedulers (Slurm, LSF, SGE) and private or public cloud (OpenStack, CloudStack, VMWare, Windows Azure, Amazon EC2, etc.).

ProActive Workflows & Scheduling also allows to control resource acquisition behavior according to various resource management policies such as load-based or time-based. For cloud or hybrid infrastructures, cloud computing power is unlocked according to your needs. With elastic scalability of the solution, virtual machines are only deployed when needed which helps save money on licence costs and VMs. With smart and fully configurable policies, unused virtual machines can be shut down whenever it’s possible.

Automatic Scaling and Cloud Resource Elasticity

The right computing power for your actual workload

ProActive Cloud Automation allows you to automatically adjust your computing capacity up or down according to your need in cloud resources. With vertical and horizontal scaling, you can ensure that the number of resources you’re using increases automatically during demand spikes to maintain performance, and decreases automatically during demand falls to reduce costs.

Configurable load factor allows you to minimize cloud spendings by deploying virtual machines only when needed. Make sure you never exceed your budget with min/max virtual machines threshold. Smart and fully configurable elastic policy allows to shutdown unused virtual machines as soon as it is possible. Moreover, it helps prevent time-consuming re-deployments with the possibility to adjust idle nodes’ release in order to avoid scale up and scale down cycles.

Truly enable the monitoring of real performance indicators of the various components of your services. Based on relevant conditions, you seamlessly trigger the activation of new resources (CPU, memory or VMs) upon demand spikes, configure them dynamically, ensuring continuous quality of service. When demand decreases, you automatically release resources for the sake of cost management, taking into account application semantics for non-disturbance of on-going services.

cloud resources elasticity and scalability

Workload Automation

ProActive Job Planner

Flexibility is your key to increase productivity

ProActive Workflows & Scheduling handles all kind of workload automation from scientific simulations to IT processes and data processing. With Workflows & Scheduling, you can automate a large panel of tasks: Native processes, Java applications, Shell Scripts, Javascripts, and any JSR223-compatible script engine. Distribute them easily and focus on your business!

Workflows & Scheduling can achieve productivity improvement for many different use cases, from CFD simulations with MPI based apps, to IT automation reusing a mix of native executables and custom made tasks.

Your recurring tasks can be easily planned. Be notified by email once the job is done. Learn more about workload automation.

Time-based scheduling

Automate execution of recurring jobs with Job planner

Job planner allows to set up custom and recurrent execution of selected workflows based on calandar rules. You can schedule recurring jobs (i.e. every hour, every 1st day of the month, every week day, etc.) or even integrate with any other calendars (office holidays, maintenance days, etc.) to completely automate batch job execution. Once the rules are defined, the job planner offers a complete calendar visualization of all workflows scheduled per year, month, week or day. Learn more about the Job Planner.

ProActive Workflow Studio

Desktop Agents

ProActive Workflow Studio

Keep the control of your machines

Desktop machines are often under-used IT resources. Desktop Agent smart software offers this computational power in a unified platform, while keeping usage under control for the sake of desktop end-users.

It runs as a background service under a restricted user and does not interfere with the day-to-day usage of the desktop machine. Typical configuration consists in the definition of a weekly working plan (like night or weekend) or limitation of memory and CPU usage. Desktop Agents are currently available for Windows and Linux.

ProActive desktop agent installed on a compute host starts and stops ProActive nodes according to a schedule, restarts ProActive nodes in case of failure and enforces resource limits for the tasks.

File Sharing

All your data on all your resources

Data integration is a key point of every business. You need to have a perfect architecture in order to integrate your data with efficiency and logic. With its Dataspaces component, Workflows & Scheduling reduces the complexity and the efforts usually needed for your data integration.

As an example, you are be able to seamlessly grab the input data from an HTTP server, process it on multiple hosts even if there is no shared file system between them, and upload the output data to an FTP server.

ProActive Workflow Studio

Multitenancy

ProActive Workflow Studio

A unified platform for all your users and all your apps

Share all your resources with all your users with Enterprise-class user management functionalities:

  • Integration with your Active Directory or LDAP;
  • Impersonation, so users can run tasks under their system account to enforce existing system policies;
  • Authorization so resources can be tied to a group of users and advanced authorizations can be set to allow or restrict access;
  • Monitoring and Accounting, allowing administrators to monitor usage of the platform for each user and application.

Monitoring

Unified supervision for your resources and applications

Workflows & Scheduling helps you save your time and money by providing a very wide range of monitoring mechanisms accessible through our web portals and APIs.

At any time, you can supervise your jobs and its tasks, and get fine details on duration, results, execution node, and failures, among others. Get the state of your execution nodes themselves: disk space left, CPU usage, memory usage, network usage, disk IO usage, list of processes. Do not forget to supervise your virtual resources: as your infrastructure grows with our cloud connectors, your supervision and control on the new virtual resources will grow too, allowing you to get vendor-specific information from the IaaS service provider monitoring API.

ProActive Workflow Studio

Fault tolerance & Error Management

ProActive Workflow Studio

Be sure to get things done

Job failures might occur caused by a brief anomaly in the execution node, so marking a job as failed in such conditions could represent a waste of resources. We let you configure your workflow and specify policies of re-scheduling that better adjust to your needs. You can define the strategy to adopt in case of errors in each workflow.

Activeeon’s solution also handles network failures and outages, and offers mechanisms to recover computations executed in a lost node once it is back, saving time and money. In a case where a node that was executing a task is no longer contactable, the scheduler tries to find another node automatically. Workflows & Scheduling preserves the state of the scheduler, all your job results and its logs.

Energy Saving

Prepare your IT for future regulations

Energy saving is not only an option: it is also a way to be prepared to regulation regarding the energy consumption in IT departments.

Green IT in ProActive is achieved with resource policies. They give you control over resources acquisitions and promote elasticity thanks to On-Demand Computing, Desktop Machines Recycling and Cloud Bursting.

ProActive Workflow Studio

Virtualization

ProActive Workflow Studio

Optimize your IT with virtualization

Do not get limited by your site physical infrastructure and come with us to explore the new Pay As You Go Virtual world. Add virtual machines to your pool of resources and use them transparently to execute your workflows.

You will not experience vendor lock in as we provide support for several cloud service providers such as Azure and Numergy, and also for OpenStack, VMWare and CloudStack based providers.

Keep controls with policies that trigger cloud bursting mechanisms for the expansion of your platform towards the cloud, so you only worry about your business.

Security

Integration with enterprise-class security

Workflows & Scheduling is a multi-user system built with security in mind.

Pluggable user authentication backends are supported. File-based and LDAP-based authentications backends are built-in. Group-based access restrictions are enforced. Finer-grained permission control is possible thanks to Java Authentication and Authorization Service (JAAS).

Activeeon’s solution also makes it possible to cope with all enterprise firewall and filtering policies.

ProActive Workflow Studio