Enhancing Researchers' Data FAIR Experience for producing Policy-Relevant Insights through STAC Open Source Software and Specifications
The Joint Research Centre (JRC) of the European Commission is committed to providing independent, evidence-based science and knowledge that supports EU policies. To facilitate this, the JRC has developed the Big Data Analytics Platform (BDAP), a data platform that allows data scientists to easily access, analyze, view, and reuse scientific data to generate and communicate evidence-based insights and foresight.
BDAP hosts spatiotemporal data at petabyte scale from various domains, including elevation, meteorological, administrative, and satellite Earth Observation data. Its architecture leverages almost entirely on Free and Open Source software and tools. The platform offers a cluster environment with both CPU and GPU machines, allowing for large-scale data processing. Additionally, users can visualize and interactively analyze their data through Jupyter Notebooks and Voilà dashboards.
Recently, BDAP implemented the Spatio Temporal Asset Catalog (STAC) specification to describe its data. The catalog hosts different types of data, which share the basic STAC fields. Thanks to the STAC modularity each data type can be described with its own STAC extensions.
BDAP reuses and benefits from various STAC Free and Open Source software and tools. In particular, from the STAC ecosystem it implements the STAC Browser for displaying and searching data, it provides STAC compliant APIs through STAC FAST-API backed by an elasticsearch instance, and uses PySTAC as a Python library for working with STAC metadata. This implementation helps BDAP in its FAIRification process improving users' search, access, and reuse of data.
In this presentation, the design and implementation of the STAC compliant set of software tools will be described. Some real use cases will be presented, with an example on the creation of analysis ready data cubes from Sentinel-2 Earth Observation satellite imagery.