Predictive analytics on IoT sensors data for mining machines

Control analytics at scale on large amount of IoT data generated every day

Key Information

IoT

Every mining machine carries more than 10 sensors

1,200

Tasks executed every hour

Big Data

Integration with various Big Data tools

Predictive analytics on the state of mining machines

A Japanese multinational corporation manufacturing construction, mining, industrial and military equipment works with Activeeon to retrieve data from sensors situated on machines in use, in order to perform predictive analytics on their state. These analytics help ensure the health and performance of mining machines and prevent issues, by applying real-time control and optimizations. To do so, the company needed to perform streaming over hot and cold storage and batch analytics, which represents 1200 tasks per hour.

Some of the requirements were:

  • Composable and flexible workflows to orchestrate data analytics
  • REST interface for full integration
  • Powerful searchable interface by status, groups, machines and execution times

Distributed and parallel data processing architecture

Workflows & Scheduling solution from Activeeon is used to orchestrate and schedule analytics workflows at scale over the cloud. Activeeon provided the customer with a unique workflow solution to handle job submission based on dynamic inputs. Data analytics orchestrated by Workflows & Scheduling can be executed and paused on distinct machines or groups of machines, triggered on events and/or on schedule.

Big Data and IT architecture for data analytics

The solution provided is currently used in dev and prod environments. It handles node failures and recovery, as well as fault tolerance for every application and service managed by ProActive Workflows & Scheduling. The solution includes a powerful interface and features data search by status, groups or distinct machines, or by execution time. A REST interface is provided for third-party tools integration.

With ProActive Workflows & Scheduling, the customer benefits from distributed and parallelized data processing, allowing to retrieve reliable results to make business decisions. Activeeon also helped the customer migrate from AWS to Azure cloud infrastructure and benefit from its scalability capacities.

As shown above, the data is ingested and stored in different storage types. ProActive from Activeeon is consuming this data based on parameters collected from the master data storage to perform analytics. The results are then published in multiple places to be consumed by operators.

Integration of the scheduler with big data solutions

Activeeon has achieved this integration thanks to its open architecture. ProActive scheduler has been integrated with multiple big data solutions such as:

  • Spark
  • Hadoop, Cloudera
  • HBase
  • MEMSQL
Discover all ProActive connectors

Moreover, the various languages supported by the solution enabled the data analytics team to select their favorite language such as R, Python or Matlab.

Contact Activeeon team