BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//pretalx//talks.osgeo.org//foss4g-2026//speaker//HCSYMV
BEGIN:VTIMEZONE
TZID:JST
BEGIN:STANDARD
DTSTART:20000101T000000
RRULE:FREQ=YEARLY;BYMONTH=1
TZNAME:JST
TZOFFSETFROM:+0900
TZOFFSETTO:+0900
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
UID:pretalx-foss4g-2026-TBW3GK@talks.osgeo.org
DTSTART;TZID=JST:20260903T110000
DTEND;TZID=JST:20260903T113000
DESCRIPTION:Machine learning is widely used to derive spatial data for envi
 ronmental decision-making\, including in digital soil mapping (DSM)\, wher
 e mapped products support carbon accounting\, land management\, crop model
 ling\, food security\, and policy reporting. In DSM\, soil organic carbon 
 (SOC) has become a key target variable because many climate- and land-rela
 ted decisions now depend on spatially explicit estimates of SOC stocks and
  change. Recent advances in machine learning-based spatial modelling have 
 accelerated SOC mapping\, while growing standardisation efforts have impro
 ved the consistency of data collection and analysis workflows.\n\nPredicti
 ons from machine learning-based environmental models are inherently uncert
 ain\, both due to incomplete knowledge of the processes controlling the ta
 rget variable (epistemic uncertainty) and irreducible variability in the s
 ystem itself (aleatoric uncertainty). Predictions should therefore always 
 be accompanied by their uncertainty estimates to support decision-making a
 nd avoid overconfidence in model outputs. Previous studies show that inter
 pretation and decision-making depend not only on the reported uncertainty 
 values\, but also on the type of uncertainty reported and how it is commun
 icated. This makes uncertainty quantification especially valuable in open-
 source applications\, as it improves the reliability and explainability of
  shared models and datasets.\n\nUncertainty quantification in DSM has rece
 ived increasing attention in recent years\, although most studies either r
 eport a single uncertainty measure or compare uncertainty performance acro
 ss different modelling algorithms. In SOC modelling\, uncertainty is most 
 commonly expressed through prediction intervals\, often derived from Quant
 ile Regression Forest (QRF) or related ensemble-based approaches\, with 90
 % prediction intervals widely adopted following the GlobalSoilMap framewor
 k (Arrouays et al.\, 2014). \n\nHowever\, much less attention has been pai
 d to comparing different uncertainty quantification methods within a singl
 e model family\, such as Random Forest. In applied workflows\, the modelli
 ng algorithm is often chosen before uncertainty becomes a methodological c
 onsideration\, usually on the basis of predictive performance. Therefore\,
  uncertainty quantification often needs to be added to an existing workflo
 w rather than drive model choice. This creates a need for systematic compa
 rison of uncertainty quantification methods within a shared model framewor
 k\, allowing different uncertainty representations to be evaluated under o
 therwise identical modelling conditions\, with attention not only to imple
 mentation but also to the sources of uncertainty represented and the quali
 ty of the resulting estimates.\n\nWe address this need by systematically c
 omparing uncertainty quantification approaches applicable to Random Forest
 -type models\, which remain among the most widely used methods in DSM due 
 to their robustness and compatibility with open-source geospatial workflow
 s. We examine how different approaches influence the spatial pattern of es
 timated uncertainty\, identify the main drivers of these differences\, and
  assess the quality and interpretability of the resulting uncertainty esti
 mates. More broadly\, we aim to provide a structured comparison of uncerta
 inty quantification methods that can be integrated into Random Forest-base
 d soil modelling workflows\, highlighting their practical differences and 
 implications for use.\n\nWe apply the comparison to a baseline Random Fore
 st (RF) model for national-scale SOC prediction in Estonia\, largely devel
 oped by Kmoch et al. (2021) and Choi et al. (2025)\, and organise the test
 ed methods into two broad groups reflecting primarily model-related versus
  data-related sources of uncertainty.\n\nThe first group includes RF-based
  interval methods. Under Quantile Regression Forest (QRF)\, we consider (a
 ) conventional 90% prediction intervals expressed through interval width\,
  (b) quantile formulations emphasising the tails of the predictive distrib
 ution\, and (c) formulations emphasising the central region containing mos
 t predictions. We also incorporate conformal prediction into the RF pipeli
 ne to generate prediction intervals with user-specified confidence levels\
 , including (d) split conformal prediction with a simple scoring function 
 (Singh et al.\, 2024)\, (e) conformalized quantile regression\, which prod
 uces wider intervals for more difficult predictions (Romano et al.\, 2019)
 \, and (f) class-conditional conformal prediction based on land-use or rel
 ated covariate classes. In addition\, we assess model sensitivity through 
 (g) hyperparameter resampling. \n\nThese RF-based interval methods describ
 e uncertainty given the available model and training data\, but not whethe
 r the available data themselves are representative of the prediction domai
 n or free from substantial error. This limitation is especially relevant i
 n DSM\, where sample coverage is often sparse or uneven\, and where both p
 redictors and observations contain uncertainty. We therefore distinguish a
  second group of approaches targeting data uncertainty\, particularly unce
 rtainty related to representativeness and extrapolation. This group includ
 es the Area of Applicability (AoA) framework of Meyer and Pebesma (2021)\,
  from which we consider both (h) the binary AoA mask and the (i) continuou
 s dissimilarity index (DI) as possible uncertainty expressions. We additio
 nally explore related out-of-distribution detection approaches as compleme
 ntary indicators of covariate-space dissimilarity.\n\nWe compare the resul
 ting uncertainty representations through a joint evaluation framework comb
 ining interval quality diagnostics and spatial pattern analysis. For RF-ba
 sed interval methods\, this includes empirical coverage of nominal predict
 ion intervals\, average interval width\, and conditional coverage\, allowi
 ng under- and over-confidence to be distinguished. For dissimilarity-based
  approaches\, we assess whether areas flagged as dissimilar\, outside the 
 AoA\, or otherwise weakly supported by the training data are associated wi
 th larger residuals and poorer interval calibration. In addition\, we exam
 ine spatial autocorrelation and clustering of high-uncertainty regions for
  each method\, and compare these with known features of the sampling desig
 n and environmental covariates.\n\nPreliminary results show that the choic
 e of uncertainty quantification method can lead to substantially different
  uncertainty patterns even when applied to the same RF model. Although dis
 similarity-based approaches sometimes highlight the same areas as interval
 -based methods\, and high dissimilarity index values often coincide with w
 ider intervals and poorer interval calibration\, this relationship is not 
 consistent across all soil types and land cover classes. These findings in
 dicate that method choice influences the uncertainty that is ultimately co
 mmunicated and interpreted\, and that combining multiple quantification me
 thods provides a stronger basis for interpreting model outputs and support
 ing decision-making. Furthermore\, disagreement between methods may itself
  serve as an additional uncertainty diagnostic.\n\nTo support reproducibil
 ity and reuse\, the full Python workflow and code for the uncertainty quan
 tification and comparison framework will be made openly available after th
 e publication of related work.\n\n\nCitations:\n\nArrouays\, D.\, McBratne
 y\, A.B.\, Minasny\, B.\, Hempel\, J.W.\, Heuvelink\, G.B.M.\, MacMillan\,
  R.A.\, Hartemink\, A.E.\, Lagacherie\, P. & McKenzie\, N.J. (2014). The G
 lobalSoilMap project specifications. GlobalSoilMap: Basis of the global sp
 atial soil information system.\nChoi\, J.\, Kmoch\, A.\, & Uuemaa\, E. (20
 25). Optimisation of sampling design for multivariate soil mapping with ma
 chine learning. In Proceedings of the 2025 conference on Big Data from Spa
 ce (BiDS’25). Publications Office of the European Union. https://doi.org
 /10.2760/2119408\nKmoch\, A.\, Kanal\, A.\, Astover\, A.\, Kull\, A.\, Vir
 ro\, H.\, Helm\, A.\, Pärtel\, M.\, Ostonen\, I. & Uuemaa\, E. (2021). Es
 tSoil-EH: a high-resolution eco-hydrological modelling parameters dataset 
 for Estonia. Earth System Science Data\, 13(1)\, 83-97.\nMeyer\, H.\, & Pe
 besma\, E. (2021). Predicting into unknown space? Estimating the area of a
 pplicability of spatial prediction models. Methods in Ecology and Evolutio
 n\, 12(9)\, 1620-1633.\nRomano\, Y.\, Patterson\, E.\, & Candes\, E. (2019
 ). Conformalized quantile regression. Advances in Neural Information Proce
 ssing Systems\, 32.\nSingh\, G.\, Moncrieff\, G.\, Venter\, Z.\, Cawse-Nic
 holson\, K.\, Slingsby\, J.\, & Robinson\, T. B. (2024). Uncertainty quant
 ification for probabilistic machine learning in earth observation using co
 nformal prediction. Scientific Reports\, 14(1)\, 16166.
DTSTAMP:20260717T225741Z
LOCATION:Cosmos1
SUMMARY:Comparing uncertainty quantification methods for Random Forest-base
 d digital soil mapping - Liina Hints
URL:https://talks.osgeo.org/foss4g-2026/talk/TBW3GK/
END:VEVENT
END:VCALENDAR