09-11, 15:30–16:00 (America/Chicago), Grand F
See a web map but can’t download the data? This talk will focus on how to find the data you want from a web mapping using the tools available in the developer console of your web browser
Often maps displayed on the web lack data that are directly downloadable. However, almost all web maps are “scrapable”, creating scraped data that may be processed using open-source tools. This presentation will cover how to leverage a web browser’s developer console to (1) locate and copy markup (HTML/SVG) used to render the map from the Elements tab, (2) discover and utilize undocumented APIs in the Network tab, and (3) provide insight for basic, high level, “best” practices for these techniques.
For the first method, the session will cover how to use regular expressions to extract information from maps rendered as SVG or HTML. For the second method, the session will cover how to send requests to undocumented APIs, so participants can retrieve their desired data. The talk will include methods for parsing and reformatting both point and polygon data, with a detailed, step-by-step walkthrough using R and a “no Code” Google Sheets option. Since applicable for many platforms, parsing methods in Python will also be discussed in brief. Finally, the importance of adequately and responsibly attributing data sourced from the web will be emphasized, supplanted by examples of how data scraped from web maps has been used on projects at NORC.
Attendees will learn how to enhance their skills at geospatial data extraction, and participants who create maps on the web will be taught how the underlying data they display may be exposed to web scrapers.
