FOSS4G 2024 Academic Track

Advances in spatial and spectral resolution in private sector satellite imagery, together with geography aware algorithms, have created new venues for the use of Artificial Intelligence (AI) in geospatial applications, sometimes referred to as AI4Geo. However, these advancements are accompanied by significant costs in the procurement of data, computing resources, communication infrastructure and human expertise. We describe a case study in central Bali in which we developed multiple AI4Geo approaches to assist the WISNU foundation, a Non-Governmental Organization in Bali, Indonesia, in their ongoing efforts to manage community resources and to perform land mapping across small villages in Bali.

Concepts
The concept we explore here is multipath AI4Geo that seeks to find the “best” approach to AI4Geo for resource constrained environments. The assumption that larger models are always better does not hold where AI4Geo, trained on data from dominant western institutions, is applied in the majority world. Some of the most ambitious AI4Geo models are trained for land cover categories that are mostly of interest to the Northern Hemisphere. Given this imbalance, we ask how participants from low-resourced environments can best make use of AI4Geo.

Methodology
Based on field data from a study site in Bali, Indonesia, we have developed multiple open source AI4Geo land cover approaches to find the best way to represent agroforestry, a key indicator of sustainable and robust food production. We compare the image segmentation results from small models such as Random Forests (RF) and Support Vector Machines (SVM) with large models such as U-Net and ResNet152 not only along established model performance metrics such as f-score, but also in terms of their suitability for use in low-resource conditions. This generally includes limited ability to collect large data sets, limited computational infrastructure, limited AI expertise and limited internet connectivity. We then describe a mixed-method multi-pathway approach to produce good AI4Geo results while building capacity for the NGO to continue the integration of AI4Geo into its operations while planning for an even more challenging AI4Geo future dominated by large homogenizing AI models.

Here are links to code experiments and instructions on generating the required input data for the U-Net model from geospatial shapefiles.

Small models (RF, SVM based on the Orfeo library)
https://github.com/realtechsupport/cocktail/tree/main/code

Large models (Custom designed U-Net and SATLAS based ResNet models)
https://github.com/realtechsupport/cocktail/tree/main/satlas_test
https://github.com/realtechsupport/cocktail/blob/main/sandbox/working_model/working_model_inference.ipynb

Results
While RF, SVM and U-Net approaches were all able to detect agroforestry in 8-band, 3-meter spatial resolution datasets provided by Planet Labs, we found that the SVM algorithm was most responsive to the limitations of our dataset while producing useful results that we could verify in the field. SVM was furthermore painless to update with additional field data. Figure 1 summarizes the results from the image segmentation after model training.

While U-Net’s f1 accuracy for agroforestry exceeds that of RF and SVM, it is likely an overestimate of the actual extent of agroforestry. We believe this to be the case because the U-Net architecture ingested patches of 16 x 16 pixels, and these dimensions exceed the size of the smaller agroforestry plots detected in the field. The choice of the input patches was in turn a function of the dimensions of the U-Net architecture selected for its ability to minimize loss during training across all land cover categories.

As opposed to the three other models listed above, the large ResNet152 model was not trained on data Planet Lab satellite imagery but on Sentinel-2 imagery. Because Sentinel-2 only has a maximum spatial resolution of 10m/pixel it is not able to distinguish small scale landscape features, agroforestry that typically utilizes small plots in random arrangements. While the ResNet algorithm was trained on the largest dataset, with over 300 distinct labels across 137 classes represented across 64 million images, the class labels are not tuned to the spectral signatures of agroforestry and deliver only crude results in our selected study area, as Figure 2 shows. Moreover, The ResNet152 model that supports multi spectral Sentinel-2 input has over 80 million trainable parameters, exceeding our bespoke U-Net model by more than an order of magnitude, thus making its use more costly.

While we have not fine-tuned the Resnet152 model with our own highest resolution Planet Lab data due to spatial resolution mismatches, it seems clear that the effort would exceed the capacities of our partner organization WISNU. Our dilemma is that the most promising large models are unwieldy and not adapted to our land cover conditions while the smaller models we have end to end control over can be tuned with smaller dataset but run the risk of becoming obsolete in the AI arms race over time, where larger and more powerful models become standard-bearers. While the agroforestry specific results we observe are characteristic of our study area and the constraints our project operates under, the homogenizing forces of large models pose a condition all AI4Geo operations are faced with. For that reason, the territory of this project is significant beyond the immediate results we produce.

Our solution to this dilemma is two-fold. We deploy multi-pathway AI4Geo across various technical complexity levels while retaining agency for local stakeholders. The Wisnu foundation does preliminary studies of Sentinel-2 satellite imagery through the QGIS environment to survey sites and build simple datasets. They then use QGIS integrated small model approaches such as Random Forests to build baseline segmentation maps of a given area. The research team will then collect Planet Labs based higher resolution data and use the cocktail suite of models, including U-Net, to deepen the study results. Parallel to this approach, we together use the SATLAS ResNet models to find synergies in those results. Across the approaches, we build land cover analysis results that optimize limited resources while producing solid analytical results.