Runtime environment for the validation of the Copernicus Ground Motion Service
The Copernicus Ground Motion Service (EGMS) is a European Union (EU) initiative under the Copernicus program, which aims to provide near-real-time information about ground deformation caused by natural or man-made hazards. The service uses a variety of data sources, including satellite radar imagery, to monitor and analyze ground motion in areas prone to landslides, sinkholes, earthquakes, and other hazards. Given the sensitive nature of the service, EGMS product validation is a key activity in assuring the user community (especially the decision makers) of the quality of the ground motion and deformation information provided.
The main goals of the EGMS validation system are as follows: to provide a reproducible environment on top of modern cloud infrastructures (with a particular focus on the European geo clouds), to enable the development of scientific tools that validate EGMS characteristics, to facilitate the reproducibility of the validation tasks, and to account for key performance indicators (which will allow shareholders to monitor the quality of the primary EGMS product).
To achieve the first goal of providing a reproducible environment, we have focused on providing Terraform modules that facilitate the deployment of our software stack on any supported cloud platform. The software stack is built on top of the Kubernetes container orchestration system, which runs on top of a managed cloud environment. Kubernetes provides uniform services regardless of the underlying cloud platform.
For the goals of developing the validation tools and the execution of those tools we decided on using an unified approach based on the JupyterHub solution. JupyterHub is used for providing an unified development environment based on R and Python EO software tools (based on modified Pangeo Docker images). Also Jupyter is used for executing the validation tools outside of JupyterHub by leveraging an internal python service that uses papermill to execute the notebook and then “nbconvert” to generate a html webpage containing the required visualizations and documentation in human readable form.
The validation system is complemented by an bespoke web dashboard aimed for providing reports and information related to the status of the various key performance indicators.
Overall the whole validation system was developed by solely using FOSS4G components: GeoPandas, RasterIO, GeoNode, GeoServer and JupyterHub.