[CNES] Satellite imagery processing at scale with cloud bursting elasticity

Customer use case of CNES (Centre national d’études spatiales)

CNES logo

Key Information

7 Petabytes of collected images

Resource agnostic: cloud & on-prem

> 500 CPU cores leveraged

Scheduling over HPC infrastructure and elastic scalability

CNES works with Activeeon on Sentinel satellite imagery analysis as part of PEPS (Plateforme d’exploitation des produits Sentinel) project. PEPS platform is part of PEPS aims to provide external users, in SaaS mode, the access to satellite imagery data, for research, education, agronomy or commercial use. Today PEPS platform counts more than 2800 users and stores more than 10TB of data per day. In order to make all this massive data available to users, PEPS includes online data processing services, which are handled by ProActive Workflows & Scheduling from Activeeon.

At CNES, ProActive Workflows & Scheduling solution ensures processing (orthorectification) of images acquired from Sentinel satellites and elastic scalability at HPC infrastructure level acting as a meta-scheduler. ProActive Workflows & Scheduling provides an open API in order to transform incoming processing queries from PEPS platform into job submissions.The workload in then distributed between the on-premise datacenter and the cloud by the resource manager included as part of ProActive Workflows & Scheduling solution, allowing to manage hybrid and multi-cloud infrastructures.

The usage of ProActive Workflows & Scheduling solution at CNES allowed to increase computing power and capacity for satellite imagery processing. At the cutting-edge computing center, CNES has 8000 processors, but sometimes the need exceeds on-premise capacity due to a high amount of data to process. Storage and processing capacities are distributed, in an optimal way, between on-premises and Azure cloud, allowing to run the same processing chains seamlessly on both infrastructures. CNES has also been convinced by the elasticity and scalability of ProActive solution provided by Activeeon.

CNES architecture

Why Proactive ?
workflow icon
Multi-Cloud Ecosystem Platform
scalable icon
Remove complexity for Data Scientists
control icon
Provide Cloud performance

About CNES

CNES is the French government Space Agency responsible for shaping and implementing french space policy in Europe. Its task is to invent the space systems of the future, bringing space technologies to maturity and guarantee France’s independent access to space. PEPS is the French platform that offers access to the products of the Sentinel satellites (S1A and S1B, S2A and S2B, S3A and S3B) of the European Union Program for Earth observation and monitoring Copernicus. Proactive Solution provided by Activeeon (Open Source editor) allows enhancing PEPS data and making them available to API providers.

CNES has benefited from the early program Azure PoC in the box and the datacenter preview. It has enabled them to understand better the constraints and potential of the cloud. Moreover, the variety of VMs provided by Azure helped them identify performance improvement gains in their algorithm which benefited them in their on-prem HPC.

For a more complete description, do not hesitate read our blog article “CNES: open access to satellite data”.

“Now, thanks to ProActive we can do everything we ever dreamed of!”