Source-linked AI summary

GIS Copilot: Towards an Autonomous GIS Agent for Spatial Analysis

Temitope Akinboyewa, Zhenlong Li, Huan Ning, M. Naser Lessani

arXiv:2411.03205v4cs.AIcs.ETcs.HCcs.SE

TL;DR

LLM capabilities remain incompletely integrated into established GIS platforms, despite their promise for geospatial analysis. This paper implements a QGIS Copilot that translates natural-language requests into tool-based workflows and code. It automates basic and intermediate tasks effectively, while full autonomy remains challenging for advanced multistep processes without explicit guidance.

  • Problem

    Generative AI has shown promise in GIS, but LLM capabilities have not been fully embedded in established platforms such as QGIS, where tools, parameters, and customized workflows complicate integration.

    Source: §1 · p. 2
  • Method

    The authors implement a QGIS Copilot that uses LLM reasoning to analyze natural-language tasks, select GIS tools, and generate and execute Python code.

    Source: §1 · p. 2, §5 · p. 21
  • Results

    The GIS Copilot automates basic and intermediate geospatial tasks effectively, while full autonomy remains challenging for advanced multistep processes without explicit instructions.

    Source: §7 · p. 25
  • Takeaways & Limitations

    The Copilot is a step toward autonomous GIS and may simplify geospatial analysis and broaden access to it.

    Source: §7 · p. 25

    During advanced testing, the agent frequently omitted data reprojection unless instructed, a limitation that may leave non-experts unaware of potential misalignment or scale problems.

    Source: §6.2 · p. 23

Abstract

from arXiv · show

Recent advancements in Generative AI offer promising capabilities for spatial analysis. Despite their potential, the integration of generative AI with established GIS platforms remains underexplored. In this study, we propose a framework for integrating LLMs directly into existing GIS platforms, using QGIS as an example. Our approach leverages the reasoning and programming capabilities of LLMs to autonomously generate spatial analysis workflows and code through an informed agent that has comprehensive documentation of key GIS tools and parameters. The implementation of this framework resulted in the development of a "GIS Copilot" that allows GIS users to interact with QGIS using natural language commands for spatial analysis. The GIS Copilot was evaluated with over 100 spatial analysis tasks with three complexity levels: basic tasks that require one GIS tool and typically involve one data layer to perform simple operations; intermediate tasks involving multi-step processes with multiple tools, guided by user instructions; and advanced tasks which involve multi-step processes that require multiple tools but not guided by user instructions, necessitating the agent to independently decide on and executes the necessary steps. The evaluation reveals that the GIS Copilot demonstrates strong potential in automating foundational GIS operations, with a high success rate in tool selection and code generation for basic and intermediate tasks, while challenges remain in achieving full autonomy for more complex tasks. This study contributes to the emerging vision of Autonomous GIS, providing a pathway for non-experts to engage with geospatial analysis with minimal prior expertise. While full autonomy is yet to be achieved, the GIS Copilot demonstrates significant potential for simplifying GIS workflows and enhancing decision-making processes.

1 Introduction

Although generative AI has shown promise in geospatial tasks, integrating LLM capabilities into established GIS platforms remains incomplete and poses challenges in coordinating tools, commands, and customized workflows. The paper proposes and implements a QGIS Copilot that translates natural-language requests into spatial-analysis workflows and code.

1 Introduction

  • Established GIS platforms have not fully embedded LLM capabilities, while their diverse tools, frameworks, data standards, and workflows complicate reliable, adaptable integration.Natural-language requests must be translated into commands aligned with native tools and parameters, and workflows often depend on user-specific requirements.Source: §1 · p. 2
  • The study implements a QGIS plugin that analyzes natural-language tasks, breaks them into steps, selects GIS tools, and generates and executes Python code.The framework aims to leverage existing GIS capabilities alongside LLM reasoning.Source: §1 · p. 2
  • The framework is presented as extensible across established GIS platforms, with natural-language interaction intended to broaden access for non-expert users.The authors describe it as the first deeply integrated GIS Copilot to support diverse geospatial tasks, transparency, real-time reasoning, and external-tool expansion, while noting that full autonomy is not achieved.Source: §1 · p. 3
  • The paper reviews related work, details its framework and prototype, evaluates cases at different complexity levels, and then discusses findings, limitations, and conclusions.Source: §1 · p. 3

2 Related Work

Prior work has applied LLMs to autonomous GIS and specific geospatial tasks, but the reviewed approaches have not fully integrated generative AI capabilities within established GIS platforms. The paper situates its work among platform-integrated tools and related industry efforts.

2.1 Autonomous GIS and the integration of generative AI into geospatial tasks

  • Autonomous GIS work spans data retrieval, information extraction, spatial analysis, mapping, and automated shapefile processing, including agents that plan and execute tools from predefined pools.LLM-Geo demonstrates geospatial operations such as aggregation, chart generation, and map presentation; GeoGPT plans and executes GIS tools through LangChain (Li and Ning, 2023).Source: §2.1 · p. 3
  • Other studies examine LLMs for cartography, mapping, geospatial tasks, and spatial-task benchmarking, but the reviewed approaches have not fully integrated generative AI into established GIS platforms.Examples include ChatGPT-generated visualization code and MapGPT’s natural-language interface to mapping tools.Source: §2.1 · pp. 3–4

2.2 Integrating generative AI into existing GIS platforms

  • QGIS efforts include a data-retrieval agent, a plugin offering guidance rather than task automation, and a workflow tool that generates code but leaves parameter configuration to users.Ning et al. (2024) focus on automated data downloading, while IntelliGeo (2024) generates PyQGIS scripts or graphical models.Source: §2.2 · p. 4
  • ChatGeoAI generates code for spatial tasks, while this study describes deeper QGIS integration, broader data and tool support, visible reasoning and code, and a self-correcting mechanism.The framework also permits external libraries such as seaborn; the comparison is presented by the authors relative to ChatGeoAI.Source: §2.2 · p. 4
  • ESRI’s described tools recommend operations or accept natural-language task inputs, while Earth Copilot targets natural-language access to NASA Earth science data.The passage characterizes ESRI’s AI Assistant as early-stage at the time of writing and its tool recommendations as guidance rather than task automation (Crawford et al., 2024; ESRI, 2024a, 2024b; Microsoft, 2024).Source: §2.2 · pp. 4–5

3 Methodology

The GIS Copilot combines spatial-data understanding, documented GIS tools, workflow planning, code generation, and execution within QGIS. Its interface supports user requests, workflow inspection, code review, and viewing analysis outputs.

3.1 Framework of the GIS Copilot

  • The framework uses data insights and geoprocessing-tool documentation to select tools, manage workflows, generate code, execute analyses, and automatically correct execution errors.Its four core modules are data understanding, GIS interface interaction, code review/debugging, and tool documentation.Source: §3.1 · p. 5

3.1.1 Spatial analysis toolbox

  • The toolbox combines QGIS and external-provider algorithms with customized tools for statistical analysis, charts, and thematic maps.QGIS provides algorithms alongside providers including GDAL/OGR, GRASS GIS, SAGA GIS, and Orfeo Toolbox; customized tools can use GeoPandas, PySAL, and Matplotlib.Source: §3.1.1 · pp. 5–6

3.1.2 Toolbox documentation

  • Tool documentation supplies descriptions, parameters, and code examples to guide code generation, and the framework supports adding customized tools.The implementation compiles 390 native QGIS and GDAL tools and allows plug-and-play additions such as kernel-density tools built with external Python libraries.Source: §3.1.2 · p. 6

3.1.3 Data understanding module

  • Before analysis, the data-understanding module collects file format, attribute fields, coordinate reference system, extent, and raster resolution to inform tool and attribute selection.This allows users to omit data details such as field names from requests.Source: §3.1.3 · p. 6

3.1.4 Code review/debugging module

  • The code review/debugging module monitors generated code during execution and iteratively attempts to correct errors and regenerate code.Source: §3.1.4 · p. 6

3.1.5 GIS interface interaction

  • The interface-interaction module accesses user-loaded layers and reloads Copilot results into QGIS for visualization and further user processing.The passage also describes the module as enabling direct interaction with the host GIS interface.Source: §3.1.5 · pp. 6–7

3.2 Workflow of the GIS Copilot

  • For each natural-language request, the agent identifies the spatial problem, breaks it into steps, and selects tools based on data type, projection, structure, and user preferences.It then generates Python code guided by tool documentation, parameter requirements, and examples, adjusting parameters to the request.Source: §3.2 · p. 7, §3.2.1 · pp. 7–8, §3.2.2 · p. 8

3.2.3 Execution and visualization of results; 3.3 User interface design

  • Code runs within QGIS; geoprocessing outputs become map layers, while summaries and charts appear as reports, and the six-tab interface exposes requests, code, workflows, reports, settings, and help.The execution module reviews, corrects, and reruns code when errors occur; users can edit generated code and select QGIS layers for analysis.Source: §3.2.3 · p. 8, §3.3 · p. 8

4 Case Studies

The case studies span single-tool operations, guided multistep workflows, and advanced tasks where the agent must choose the steps independently. Across 110 evaluated tasks, success rates declined from basic to intermediate to advanced complexity, and reported failures included parameter, data-field, and projection issues.

4.1 Basic level tasks

  • The agent generated 2,000-foot health-facility buffers and converted Puerto Rico’s DEM into contour lines at a 50-meter interval.The buffer task used health-facility data; the contour task used gdal:contour to produce vector output.Source: §4.1.1 · p. 9, §4.1.2 · p. 10, Figure 4 · p. 10
  • For basic filtering and clipping, the agent selected counties above 50,000 population and clipped U.S. land-cover raster data to Pennsylvania.The cases used extract-by-attribute and GDAL Clip Raster by Mask Layer, respectively.Source: §4.1.3 · pp. 11–12, §4.1.4 · p. 12, Figure 6 · p. 12

4.2.1 Zonal statistics of average elevation for counties in South Carolina

  • In the South Carolina zonal-statistics task, the agent clipped a DEM, generated a pixel-value histogram, calculated county average elevations, and made a choropleth.Darker map shades represent higher average elevations; the workflow diagram shows the analysis steps.Source: §4.2.1 · pp. 13–14, Figure 8 · p. 14

4.2 Intermediate level tasks

  • The agent merged four DEM tiles for Richland County to calculate five terrain attributes and joined county-level data to create a contiguous-U.S. obesity-risk index map.The terrain outputs were slope, aspect, hillshade, TRI, and TPI; the obesity-risk task used county boundaries and place-visit rates.Source: §4.2.2 · p. 15, §4.2.3 · pp. 16–17, Figure 9 · p. 16

4.3.1 Fast-food accessibility and obesity correlation analysis for Pennsylvania counties

  • For Pennsylvania, the agent calculated per-capita fast-food accessibility, mapped the scores, and analyzed their correlation with county obesity rates.It selected the specific fields without explicit user guidance.Source: §4.3.1 · pp. 17–18, Figure 11 · p. 18

4.3.2 Spatial distribution of COVID-19 cases across US counties

  • For U.S. counties, the agent joined COVID-19 case data to county boundaries using GEOID and FIPS, then mapped cases with darker blue for higher counts.The task used COVID-19 data through December 2020.Source: §4.3.2 · pp. 18–19, Figure 12 · p. 19

4.3.3 Generating the Normalized Difference Vegetation Index (NDVI) from satellite imageries

  • For Akure, the agent identified Landsat 8 NIR and red bands and calculated NDVI, while the raster’s symbology was manually adjusted for clearer viewing.Higher NDVI values indicate denser vegetation in the resulting visualization.Source: §4.3.3 · pp. 19–20, Figure 13 · p. 20

4.4 Performance evaluation

  • 95% of 60 basic tasks succeeded, compared with 80% of 30 intermediate tasks and 75% of 20 advanced tasks.Tasks still unsuccessful after three attempts counted as failures.Source: §4.4 · p. 20, Table 1 · p. 21

5 Discussion

The discussion presents GIS Copilot as a QGIS integration of LLMs with existing geoprocessing tools, supporting natural-language workflows alongside manual operations. Evaluation shows stronger performance on basic and guided intermediate tasks than on unguided advanced tasks, while extensibility and inspectable code are presented as design strengths.

5 Discussion

  • The QGIS framework covers vector and raster geoprocessing tools and supports natural-language interaction through the implemented Copilot.The framework is described as using existing GIS tools to generate executable spatial analysis algorithms.Source: §5 · p. 21
  • Users can add custom tools through a predefined template, keeping the framework adaptable to user needs and new GIS tools and analyses.Source: §5 · p. 22
  • Following Li and Ning (2023), QGIS integration lets users combine agent-generated analysis with manual operations, dataset selection, and visualization inside the GIS environment.The authors describe this as a hybrid approach in which users can continue analysis with either the Copilot or manual methods.Source: §5 · p. 22
  • Over 100 tasks, the Copilot showed high accuracy on basic tasks and strong workflow performance on guided intermediate tasks, but lower success on unguided advanced tasks.Advanced tasks required the agent to determine the workflow without explicit guidance.Source: §5 · p. 22
  • Because opaque outputs may discourage adoption, the Copilot exposes tool selection and generated code for inspection, editing, and tracing how results are produced.Users are encouraged to review the generated code and results rather than rely on them blindly.Source: §5 · pp. 22–23

6 Limitations and Future Work

The limitations section identifies reliability gaps in data and tool matching and GIS-specific knowledge, then outlines additions intended to address these issues and advance the Copilot toward autonomy.

6.1 The need for a data validation module to reduce tool and parameter mismatch

  • Parameter mismatches can cause code failure, such as assigning vector input to gdal:proximity, which requires raster data; the authors propose data-type validation and automatic conversion.Source: §6.1 · p. 23

6.2 Expanding LLMs knowledge in GIScience fundamentals

  • The agent frequently omitted reprojection unless instructed and struggled with some GDAL tools; the authors suggest retrieval-augmented generation to handle documentation-length constraints.Source: §6.2 · p. 23

6.3 Building an experience-based knowledge system

  • Internet-derived knowledge lacks practical GIS experience; an experience database could accumulate errors and solutions to help the agent recognize recurring issues and apply corrections.Source: §6.3 · pp. 23–24

6.4 Integration of a prompt-generation module

  • Task descriptions may omit technical details, so an intermediate module could extract key requirements and turn them into structured prompts for geospatial tasks.Source: §6.4 · p. 24

6.5 Integration with a geospatial data retrieval agent

  • A geospatial data-retrieval agent could provide relevant datasets directly, reducing user input and improving efficiency, particularly for non-experts.Source: §6.5 · p. 24

6.6 Exploring the integration of open-source LLM models

  • The study uses OpenAI GPT; integrating open-source models could offer greater customization and accessibility, including GIS-specific fine-tuning.Source: §6.6 · pp. 24–25

7 Conclusions

The conclusion presents the QGIS Copilot as effective for basic and intermediate geospatial tasks, while noting that complex workflows still challenge full autonomy. It frames the system as a step toward autonomous GIS and identifies further development directions.

  • The QGIS Copilot automates basic and intermediate geospatial tasks, but full autonomy remains challenging for advanced multistep processes without explicit guidance.Source: §7 · p. 25
  • The authors identify improving complex-workflow performance and error correction, and expanding adaptability to other GIS platforms, as future research directions.Source: §7 · p. 25
  • The source code, QGIS plugin, and testing datasets and cases are available through GitHub, the QGIS plugin page, and a dataset link.Source: §7 · p. 25

Appendix 1: Sample of QGIS native tool documentation (structured in TOML format)

The appendix illustrates QGIS tool documentation and code for a vector-layer clipping workflow, including its inputs, output options, and result loading.

Appendix 2: Sample of customized tool documentation (structured in TOML format)

The customized tool documentation describes a Gaussian KDE workflow that turns an input point layer into a density raster, with larger values for locations containing more clustered points.

Appendix 2: Sample of customized tool documentation (structured in TOML format)

Appendix 3: Basic level test cases

The basic-level test-case appendix lists single-operation examples, including county selection, a U.S. obesity bar chart, and clipping a DEM to Pennsylvania boundaries; the datasets are for testing only.

Appendix 3: Basic level test cases

  • The appendix states that all included datasets are for testing purposes only.The county population task names SC_Counties and a population-increase dataset.Source: Appendix 3: Basic level test cases · p. 31
  • The listed cases include selecting counties with less than 10% population growth from 2005 to 2008 and clipping a DEM to Pennsylvania boundaries.The clipping case is paired with raster-analysis clipping and a GDAL clip-raster-by-mask-layer tool entry.Source: Appendix 3: Basic level test cases · p. 31
  • A separate case requests a clearly labeled bar chart of obesity prevalence across the United States.The table associates the plotting case with seaborn and county obesity data.Source: Appendix 3: Basic level test cases · p. 31

1 PA_DEM PA_DEM_Clipping

The listed cases span raster analysis, vector queries and transformations, plotting, raster reporting, and image processing, with requested outputs named for several tasks.

1 PA_DEM PA_DEM_Clipping

  • One raster-analysis case requests Puerto Rico DEM contour lines at 50-meter intervals.The table associates the case with GDAL contour generation.Source: §1 · p. 31
  • Vector cases include adding latitude and longitude fields to fast-food restaurants and selecting Pennsylvania counties below $50,000 median household income.The county-selection task is categorized as an attribute query.Source: §1 · p. 32
  • Other spatial-query cases ask how many damaged houses fall within the San Andreas Special Studies Zones and where railways intersect roads.The railway-road intersection output is named Railway_Road_Intersection.Source: §1 · p. 32
  • An affine-transformation case requests scaling a building layer by a factor of 2 and rotating it 30 degrees.The listed input is Penn_State_Building.Source: §1 · pp. 32–33
  • A statistical-plotting case requests a scatter plot of county obesity rates and Covid-19 death rates.The listed output is Covid_Obesity_ScatterPlot.Source: §1 · p. 33
  • Raster tasks include reporting SC_DEM information in HTML and generating a histogram of its pixel values.The table lists gdalinfo for the report and seaborn for the histogram.Source: §1 · p. 33
  • Image-processing requests include a Fourier conversion and an RGB composite of three Landsat 5 bands using the most commonly used composition.The table associates the composite task with band rearrangement.Source: §1 · pp. 33–34

2 LandSat/TM5_L2/L T05_L2SP_029030_ 20110605_20200822 _02_T1_SR_B3.TIF

This section documents GIS tasks across raster and vector processing, spatial analysis, mapping, reporting, and decision-support workflows. It includes successful algorithm outputs as well as cases explicitly marked failed.

2 LandSat/TM5_L2/L T05_L2SP_029030_ 20110605_20200822 _02_T1_SR_B3.TIF

  • The part also records a successful Landsat band composition and outputs for reprojection, distance-raster generation, and terrain roughness analysis.The roughness task asks for an index based on DEM data; the distance-raster task concerns fast-food restaurants.Source: §2 · pp. 34, 35, 42
  • The flood-to-river distance histogram and nearest-river distance tasks are marked failed, as is the request to draw shortest lines between flood locations and rivers.The passages name Flood2015may.shp and River.shp for the failures; a separate shortest-line passage lists native:shortestline as the algorithm.Source: §2 · pp. 34–35
  • The outputs include data conversion, filtering, geometry measurement, buffering, and clipping operations, using point-layer creation, expression or attribute extraction, geometry-column export, buffer, and PDAL clip algorithms.The CSV-to-shapefile task also lists savefeatures and loadlayer; a separate request specifies a 0.1 m buffer.Source: §2 · pp. 35–36
  • Overlay and related tasks list outputs for extracting intersecting points, symmetrical difference, union, and eliminating selected polygons.The intersection task includes reprojection and extract-by-location algorithms; the symmetrical-difference and union tasks each list their corresponding native algorithms.Source: §2 · pp. 37–38
  • Zonal statistics for summing pixel values within census tracts is marked failed, and the fast-food count-by-Pennsylvania-county task is also marked failed.The latter task lists PennsylvaniaCounties and PA_Fastfoods as its inputs.Source: §2 · pp. 39–40
  • The request to identify five nearest neighbors to a center point is marked failed for Points.shp and Center.shp.The associated method passage lists native:joinbynearest.Source: §2 · p. 41
  • Tasks 52–54 reproject and rasterize buildings, create a binary mask for elevations above 1000 m, and set zero-valued pixels to no data.The listed operations include GDAL reprojection and rasterization, followed by raster-calculator steps.Source: §2 · p. 43
  • Tasks 55–57 convert the mask to vector, save data as a shapefile, and create Thiessen polygons for point data.The listed tools include polygonization and Voronoi polygon generation.Source: §2 · pp. 44–45
  • Tasks 58–59 request an HTML report describing two datasets and potential applications, followed by an HTML distance matrix.The listed approaches include dataset information and hillshade generation, and use of pandas and geopandas for matrix analysis.Source: §2 · p. 45

Appendix 4: Intermediate level test cases

  • Intermediate tests include DEM clipping, histograms and zonal statistics, building-area reports, road-length calculations, ruggedness summaries, and terrain characterization.Other prompts cover coordinate and elevation extraction, raster merging, county counts, buffers, and distance analysis.Source: Appendix 4: Intermediate level test… · pp. 47–49
  • Further intermediate prompts cover poverty-ratio selection, centroid export, hospital and store proximity, building terrain measures, elevation histograms, and street-view year mapping.The prompts specify, among other workflows, a 50-bin elevation histogram and finding coffee shops with the most nearby bus stops or metro entrances.Source: Appendix 4: Intermediate level test… · pp. 50–52
  • The hospital-accessibility task was marked failed after requesting nearest hospitals for census tracts, centroid-to-hospital distances, and a distance raster.A separate task requested a 100 m raster of distances from San Francisco census tracts to the nearest hospitals.Source: Appendix 4: Intermediate level test… · pp. 50–51
  • The tree-location error task and building-area population allocation task are marked failed.The error task sought each ground-truth trunk’s nearest estimated location within 3 meters and the mean matched distance; allocation apportioned block-group population by building area.Source: Appendix 4: Intermediate level test… · pp. 53, 55
  • Intermediate tests include census joins and population allocation, county raster clipping, EV-weight calculation, watershed modeling, and filtered IDW raster generation.The watershed workflow specifies DEM sink filling, flow direction, watershed and flow-accumulation analysis, then streamlines using a threshold above 200.Source: Appendix 4: Intermediate level test… · pp. 54–58

Appendix 5: Advanced level test cases

  • Advanced cases cover street-tree counts within street buffers, developed-land area, protected-region road lengths, county queries, and county-boundary measurements.The road-length task asks for lengths inside each protected region, identified by “fid,” and is marked failed.Source: Appendix 5: Advanced level test cases · pp. 59–60

2 TL_2019_42_PRISE CROADS

  • The requests cover county weather-station names, Pennsylvania tree-planting suitability, hazard-waste block-group population, and solar-farm suitability based on slope and power-line proximity.Tree-planting counties qualify above 2.5 inches of annual rain; solar areas require slope below 5 degrees and distance within 500 meters of the nearest power line.Source: §2 · pp. 60–61
  • Tasks 11–13 requested sidewalk-length calculations near schools, county HIV-rate joins with thematic mapping, and fast-food accessibility mapping with obesity-correlation analysis.The fast-food task specified a scatter plot with a regression line.Source: §2 · p. 62
  • Other analyses requested viewsheds from observation points and DEM extraction at street-view locations for comparison with image metadata.The viewshed prompts specified a 5-kilometer maximum radius and cellular-tower visibility parameters; the extraction prompt requested descriptive statistics of elevation differences.Source: §2 · pp. 63–64
  • Three tasks reported failures: answering cell-tower visibility questions, proposing a Walmart site near roads, and identifying camping-suitable areas by aspect and road distance.The reported outputs were a failed viewshed file, a failed Walmart site file, and a failed suitability result.Source: §2 · pp. 63–64
  • Task 18 requested an Akure NDVI from satellite imagery, while task 19 requested DEM value extraction at checkpoints followed by RMSE calculation.The supplied passages describe the requested analyses but give no corresponding results.Source: §2 · p. 64
Loading 2411.03205v4…