Distributed and parallel data processing architecture
Workflows & Scheduling solution from Activeeon is used to orchestrate and schedule analytics workflows at scale over the cloud. Activeeon provided the customer with a unique workflow solution to handle job submission based on dynamic inputs. Data analytics orchestrated by Workflows & Scheduling can be executed and paused on distinct machines or groups of machines, triggered on events and/or on schedule.
The solution provided is currently used in dev and prod environments. It handles node failures and recovery, as well as fault tolerance for every application and service managed by ProActive Workflows & Scheduling. The solution includes a powerful interface and features data search by status, groups or distinct machines, or by execution time. A REST interface is provided for third-party tools integration.
With ProActive Workflows & Scheduling, the customer benefits from distributed and parallelized data processing, allowing to retrieve reliable results to make business decisions. Activeeon also helped the customer migrate from AWS to Azure cloud infrastructure and benefit from its scalability capacities.
As shown above, the data is ingested and stored in different storage types. ProActive from Activeeon is consuming this data based on parameters collected from the master data storage to perform analytics. The results are then published in multiple places to be consumed by operators.