10 years of open-source software in emergency management: the case of the European Flood Awareness Service
The European Flood Awareness System and the Global Flood Awareness System (EFAS and GloFAS), are the two Early Warning Service for floods part of the Copernicus Emergency Management Service (CEMS), operated by the EU Joint Research Centre (JRC). EFAS and GloFAS aims to complement national and regional service by providing medium-range flood forecasts and hydrological outlooks for large, transboundary rivers. Data and products are accessible to eligible users through the Climate Data Store and dedicated web interfaces. ECMWF, having the role of the computational centre within CEMS, is responsible for running the forecasts and the post-processing, on top of co-developing and hosting the EFAS and GloFAS information systems.
These two information systems consist on back-end/front-end web services based on OGC standards and open-source software. As it is often the case, a web-based mapviewer allows to display different layers, produced by a WMS back-end. These layers are the graphical representation of the output of the hydrological models and meteorological observations, like flood probability, soil moisture, return period, observed precipitation etc. For most layers a new forecast is produced every 12 hours for EFAS and every 24 hours for GloFAS.
Unlike many similar services, however, the aim of EFAS and GloFAS is not only to offer the latest forecasts or the latest observations but also to browse through data from previous days, so that older forecasts can be compared with actual observed events. This inherently means supporting the time dimension within the WMS standard, and managing large quantity of data that accumulates every day. In the case of EFAS, for example, an additional 1.5 Gb of data is produced twice a day.
It also means handling the inevitable changes in data formats and structures that arise as the service grows and new features are added, without breaking backward compatibility. New layers are added, old layers are removed, changes in the geographical domain or the projection for a certain layer must be supported from a certain date onward, etc. Not to mention increasing the number of forecast cycles from one per day to two or more.
To make matters worse, data access must be restricted on both front-end and back-end based on a matrix of user privileges, requested product and requested date. For example some layers are offered to all users with no time restrictions, while others are restricted to some users for the latest 30 days, and freely accessible to all users for dates older than 30 days ago.
In this talk we describe the challenges of developing and operating an authentication-aware web service heavily based on large geospatial datasets with a strong diachronic component.