Large-scale geospatial processing tasks such as spatial analysis, spatial indexing, and vector processing often require significant computing resources. In many production environments, however, computing resources remain underutilized during periods of low activity.
This talk introduces an Idle-Aware Geo-Processing Scheduler, a worker-based processing system designed to run geospatial workloads opportunistically by utilizing idle compute resources.
The system operates as a distributed worker processing engine built around a task queue architecture. When new geospatial jobs enter the queue, the scheduler automatically scales workers based on workload demand. For example, if multiple jobs are submitted simultaneously, multiple workers can be spawned to process them in parallel.
Conversely, when the workload decreases or the queue becomes empty, workers automatically scale down, allowing the system to return to minimal resource usage. This mechanism enables geospatial processing tasks to run as background pipelines without interfering with primary system workloads.
The architecture follows cloud-native principles, allowing the system to be deployed on cloud environments where compute resources can scale elastically. By combining auto-scaling workers, task queues, and open-source geospatial tools, spatial processing pipelines can dynamically expand during high workloads and shrink during idle periods.
This talk will cover:
-The design of an idle-aware scheduler for geospatial workloads
-Architecture of a worker-based geospatial processing engine
-Applying auto-scaling and cloud-native deployment to spatial processing pipelines
-Task queue orchestration for distributed geospatial jobs
-Integrating open-source geospatial tools into scalable processing workflows

This approach enables geospatial teams to build flexible spatial processing pipelines that adapt to workload demands while efficiently utilizing available computing resources.