How I began adding open source GIS tools in an otherwise proprietary toolchain to unlock previously impossible capabilities, and the pitfalls encountered along the way.

In my organization, GIS users are divided between teams which use proprietary ESRI and the minority which use open OGC-based tooling.

Migrating a browser based GIS tool built on the ESRI stack from an on-prem deployment to the cloud provided us with an opportunity to connect tools from these two groups. We could now connect the webapp to services in the cloud to run the heavier compute required for modelling and simulation, and these backend tools were overwhelmingly built upon open source packages. This surfaced the friction of interoperability between these ecosystems, data moving between them needed to be transformed.

We began integration by hand-rolling geojson conversions, but quickly moved to using more robust inline data converters such as terraformer and serde_esri when subtle bugs began to appear. One example was that the winding order differences between polygons in the OGC and ESRI standard would cause errors in polygon area calculations.

The cloud migration also enabled us to host new data sources and format in cloud storage, including cloud native formats such as geoparquet. However, GIS data from different sources and in different formats, some from ESRI systems and others from open standards, were not immediately compatible. Bridging these data sources and making them useful to consumers which might want either ESRI or OGC formatted data (with the additional complexity of access control) is a challenge we are still working through.

As integration needs grew, I created esri_json, a rust crate heavily inspired by serde_esri, with deeper integration to my display formatting needs. Tools built with this translation layer and compiled to wasm could be dropped into the existing webapp. The tools can ingest data and create geometries and symbology which can be displayed in the web browser without needing the web app developers to transform data (because the tool handles the conversion). Having data and custom tools on the same platform, with interoperability handled by the tools, unlocked workflows which were previously impossible.

The technical work of interoperability was also a process of building trust. Each problem solved and new capability unlocked strengthened the case for open source GIS tooling within the organization. Contributing performance improvements back to the geo rust crate which were discovered while working on these tools was a natural extension of using it heavily and participating in the open source geospatial community.

This talk is an anecdotal guide for people navigating similar divides between open and proprietary tooling, different data standards and changing how people think about possibilities with the tools they use.

<Email response with clarification below>
Hi!

I intend the core for this talk to be angled toward (hopefully practical) lessons based on my experiences with integrating open source geospatial tools into the stack, such as the technical and organisational challenges which we faced and how we overcame or are working through them. The primary means of this advocacy would be incrementally demonstrating novel value while not disrupting the established system too much. Additionally, on how interoperability was the key factor in bringing open source into the conversation (the prior impression my managers had was that these were two incompatible GIS paradigms ).

Specific open source tools would be included as part of case studies and examples, such as but not limited to :

1. shapely/geopandas for python backend tools
   a greenfield simulation tool which had good organisation support, but we hit a hurdle with polygon conversion which resulted in area statistics being unreliable (technical challenge), which led it to be perceived as "wrong" until we had reliable format conversion

2. geo, serde_esri and compiling to wasm
   a parallel implementation of an existing analysis workflow which faced resistance because of minor differences in results, and how we reframed it as a complementary tool instead of a competition or replacement of the existing workflow (organisational challenge)

These examples would be coming from the perspective of an IC pushing for ground up change.

Best,
Daniel Tan