Scheduling over HPC infrastructure and elastic scalability
CNES works with Activeeon on Sentinel satellite imagery analysis as part of PEPS (Plateforme d’exploitation des produits Sentinel) project. PEPS platform is part of PEPS aims to provide external users, in SaaS mode, the access to satellite imagery data, for research, education, agronomy or commercial use. Today PEPS platform counts more than 2800 users and stores more than 10TB of data per day. In order to make all this massive data available to users, PEPS includes online data processing services, which are handled by ProActive Workflows & Scheduling from Activeeon.
At CNES, ProActive Workflows & Scheduling solution ensures processing (orthorectification) of images acquired from Sentinel satellites and elastic scalability at HPC infrastructure level acting as a meta-scheduler. ProActive Workflows & Scheduling provides an open API in order to transform incoming processing queries from PEPS platform into job submissions.The workload in then distributed between the on-premise datacenter and the cloud by the resource manager included as part of ProActive Workflows & Scheduling solution, allowing to manage hybrid and multi-cloud infrastructures.
The usage of ProActive Workflows & Scheduling solution at CNES allowed to increase computing power and capacity for satellite imagery processing. At the cutting-edge computing center, CNES has 8000 processors, but sometimes the need exceeds on-premise capacity due to a high amount of data to process. Storage and processing capacities are distributed, in an optimal way, between on-premises and Azure cloud, allowing to run the same processing chains seamlessly on both infrastructures. CNES has also been convinced by the elasticity and scalability of ProActive solution provided by Activeeon.