Source-linked AI summary

Circuitscape in Julia: High Performance Connectivity Modelling to Support Conservation Decisions

Ranjan Anantharaman, Kimberly Hall, Viral Shah, Alan Edelman

arXiv:1906.03542v1q-bio.QM

TL;DR

Growing connectivity workloads exceed the capacity of existing tools, limiting analysis of increasingly large landscapes. This paper upgrades Circuitscape in Julia with improved parallel processing and solvers; it reports speed improvements of up to 1800% and support for landscapes with billions of pixels.

  • Problem

    The Python implementation handled up to tens of millions of nodes, while increasingly fine-resolution connectivity datasets can involve hundreds of millions of nodes and dozens of focal pairs.

    Source: §2 · p. 2
  • Method

    The authors reimplemented Circuitscape in Julia, adding parallel computation and a large-problem solver based on preconditioned conjugate gradients with algebraic multigrid.

    Source: §4 · p. 5, §5.3 · p. 6, §8 · p. 7
  • Results

    Up to 1800% improvement in computation time, with the Julia upgrade able to solve landscapes with billions of pixels.

    Source: §8 · p. 7
  • Takeaways & Limitations

    The faster upgrade supports sensitivity analysis and rapid prototyping of resistance maps, and can shorten the turnaround for conservation decisions through stronger stakeholder engagement.

    Source: §7 · p. 6

    Sparse direct solvers can run out of memory on larger matrices because factorization produces less-sparse factors.

    Source: §5.1 · p. 5

Abstract

from arXiv · show

Connectivity across landscapes influences a wide range of conservation-relevant ecological processes, including species movements, gene flow, and the spread of wildfire, pests, and diseases. Recent improvements in remote sensing data suggest great potential to advance connectivity models, but computational constraints hinder these advances. To address this challenge, we upgraded the widely-used Circuitscape connectivity package to the high performance Julia programming language. Circuitscape.jl allows users to solve problems faster via improved parallel processing and solvers, and supports applications to larger problems (e.g., datasets with hundreds of millions of cells). We document speed improvements of up to 1800\%. We also demonstrate scaling of problem sizes up to 437 million grid cells. These improvements allow modelers to work with higher resolution data, larger landscapes and perform sensitivity analysis effortlessly. These improvements accelerate the pace of innovation, helping modelers address pressing challenges like species range shifts under climate change. Our collaboration between ecologists and computer scientists has led to the use of connectivity models to inform conservation decisions. Further, these next generation connectivity models will produce results faster, facilitating stronger engagement with decision-makers.

1 Introduction

Connectivity models help assess movement and flow across heterogeneous landscapes, supporting conservation decisions about populations, risk, and land protection. Circuitscape uses electrical circuit theory to model movement across multiple pathways, complementing least-cost approaches that identify a single route.

Conservation applications

  • Connectivity maps inform resource evaluations and risk assessments involving population dynamics, genetic diversity, disease, wildfire, and land protection under climate change.The cited applications include decisions about where to invest in land protection or restoration to support species range shifts.Source: §1 · p. 2

Representing landscapes

  • Resistance grids assign each landscape cell a relative resistance to movement and can be abstracted as graphs for quantifying ecological distance.Resistance weights may be developed iteratively using expert opinion and information on species’ life histories and movement behaviors [Spear et al., 2010, Zeller et al., 2012].Source: §1 · p. 2

Connectivity approaches

  • Least-cost distance models identify a single best route, whereas isolation by resistance models potential movement across multiple pathways as random walks.The single-route approach depends strongly on start and end points and is less suited to comparing restoration or protection options across multiple habitat patches and pathways.Source: §1 · p. 2

Circuitscape

  • Circuitscape calculates effective resistance and current flow across heterogeneous landscapes, with flexible inputs for structural and species-specific functional connectivity.Its applications span gene flow, animal movement, fire, water, and disease spread, and its outputs have been used to inform conservation decisions.Source: §1 · p. 2

2 Reimplementing Circuitscape in Julia

The Python implementation could not meet growing workloads involving hundreds of millions of nodes and many focal pairs. The authors reimplemented Circuitscape in Julia to improve computational capacity while retaining accessibility and maintainability.

Computational demands

  • The Python package handled problems up to tens of millions of nodes, below increasingly common workloads of hundreds of millions of nodes and dozens of focal pairs.The authors identify these newer community demands as the reason Circuitscape needed an upgrade.Source: §2 · p. 2

Existing alternatives

  • GFLOW delivered state-of-the-art performance on large clusters, but its Windows incompatibility and complicated binary builds constrained accessibility and integration.The authors also identify difficulty integrating GFLOW into decision-support tools and workflows as a barrier for practitioners.Source: §2 · p. 2

Julia implementation

  • The authors chose Julia for an open-source Circuitscape implementation intended to be maintainable, high performance, accessible, and cross-platform.They describe Julia as combining scripting-language readability with the performance of statically compiled languages.Source: §2 · p. 3
  • Julia’s sparse-matrix and factorization support enabled multiple solvers, while its JIT compiler specializes generic code for precision, index types, and platforms.The authors state that these capabilities supported new user-facing features with relatively little effort in a straight reimplementation.Source: §2 · p. 3

Goals and expected use

  • The rewrite aims to support relatively fine-resolution modeling of gene flow and species movement at continental or global scales while remaining open source and accessible.The authors expect faster processing of large datasets to encourage high-resolution data use, sensitivity testing, and dynamic simulations.Source: §2 · p. 3

Paper scope

  • The paper proceeds from algorithm and numerical-methods overviews to new features, benchmarks, and discussion of collaboration with conservation managers and stakeholders.The stated benchmark coverage includes synthetic problems and real user data.Source: §2 · p. 3

3 Overview of the Algorithm

Circuitscape converts a resistance raster and focal locations into a graph-based circuit problem. Solving that problem yields voltage and current quantities that are written as resistance and current maps for analysis.

Inputs

  • Circuitscape takes a raster landscape as input, with each cell’s value representing habitat, dispersal-route, or movement-barrier qualities.Resistance values are problem-specific and may be derived empirically from movement or genetic data or from expert opinion.Source: §3 · p. 3

Solving

  • For each focal-node pair, Circuitscape forms a current-source and sink setup and solves the graph-Laplacian system to obtain nodal voltages.The algorithm then computes effective resistance from the solution.Source: §3 · p. 4

Graph construction

  • The model represents grid cells as graph nodes and assigns edge weights proportional to movement probabilities or migrant exchange between neighboring cells.Focal points identify landscape locations of interest, such as habitat patches, protected areas, or populations to connect.Source: §3 · p. 4

Outputs

  • Branch currents represent movement probabilities along graph edges, and their accumulation produces nodal-current maps written as raster ASCII grids for GIS.The algorithm also writes nodal voltages to disk.Source: §3 · p. 4

4 Overview of Numerical Methods

Circuitscape solves a sparse linear system derived from the landscape graph, offering direct and iterative solver options. For large problems, it uses preconditioned conjugate gradients with algebraic multigrid.

4 Overview of Numerical Methods

  • Circuitscape’s main computation applies Ohm’s and Kirchhoff’s laws to large resistive grids by solving a sparse linear system.The system uses the graph Laplacian as the conductance matrix, with current sources and nodal voltages as its other quantities.Source: §4 · p. 5
  • Cholesky factorization suits smaller study areas with many focal nodes because one factorization can serve multiple pairs through back substitution.It scales poorly in memory for larger matrices because factorization can produce factors less sparse than the original matrix.Source: §4 · p. 5

4.1 Preconditioner Implementation

  • For large problems, the authors use preconditioned conjugate gradients with an algebraic multigrid preconditioner.Their Smoothed Aggregation AMG variant is suited to matrices generated by elliptic partial differential equations, including planar-graph Laplacians.Source: §4 · p. 5, §4.1 · p. 5

5 New Features

The upgrade adds a Cholesky solver and uses Julia’s generic programming and parallel-computation support to extend Circuitscape’s capabilities. The new solver also supports localized tile and sliding-window analyses.

5.1 New Solver Mode

  • The new Cholesky-based solver supports landscape tiling and sliding-window analyses that yield fine-scale, localized results.It uses the CHOLMOD library; sparse direct solvers can run out of memory on larger matrices because factorization factors are often less sparse than the original matrix.Source: §5.1 · p. 5

5.2 Generic Programming

  • A 437-million-pixel Mojave desert tortoise dataset crashed the old version but ran smoothly to completion after the upgrade.The previous Python package’s hardcoded 32-bit matrix indexing caused crashes on simulations with hundreds of millions of nodes; Circuitscape.jl offers a runtime option for 64-bit indexing.Source: §5.2 · pp. 5–6

5.3 Improved Parallel Computations

  • Circuitscape.jl extends parallelism to more problem types and can run multiple Circuitscape instances in parallel across landscape tiles.These parallel features work on Linux, macOS, and Windows.Source: §5.3 · p. 6

6 Benchmarks

The benchmarks compare Circuitscape versions using standard synthetic problems and a real Sonoran Desert dataset. On the real-data all-to-one run, the Julia version substantially reduced execution time.

6.1 Experimental Setup

  • The experiments used an Intel Xeon Silver 4114 CPU at 2.20 GHz with 384 GB RAM, Julia v1.1.0, and Circuitscape v5.5.0, compared with Circuitscape v4.0.5.Source: §6.1 · p. 6

6.2 Standard Benchmarks

  • The standard benchmark tests used synthetic problems available from the Circuitscape/BigTests repository.Source: §6.2 · p. 6

6.3 Benchmarks on Real Data

  • Nearly 18x speedup reduced the Sonoran Desert all-to-one computation from over 2 days to less than 3 hours.The prior version did not support parallelism for this mode; the Julia version supported it with little effort.Source: §6.3 · p. 6

7 Discussion

The discussion presents faster computation as a way to avoid coarsening analyses and to support sensitivity testing and stakeholder engagement. It also notes Julia’s scripting and compiler capabilities.

  • The Julia upgrade means users no longer have to coarsen analyses because of tooling limitations.Source: §7 · p. 6
  • Faster runs enable sensitivity analysis of individual resistance-map layers and rapid prototyping of resistance maps.The paper says previous Circuitscape versions made practical sensitivity testing prohibitively slow.Source: §7 · p. 6
  • Cutting execution time from weeks to days or days to hours enables stronger stakeholder engagement and faster conservation-decision turnaround.The authors describe this as improving the feedback loop with policy makers.Source: §7 · p. 6
  • Julia’s ease of use and mathematical syntax support fast scripting, while its modern compiler lets prototypes scale to large datasets.Source: §7 · p. 6

8 Conclusion

The conclusion presents the Julia upgrade as enabling larger, finer-resolution ecological analyses, with faster computation and support for very large landscapes. The package is open source and available under the MIT license.

  • Up to 1800% improvement in computation time and the ability to solve landscapes with billions of pixels are reported for the Julia upgrade.Source: §8 · p. 7
  • The upgrade is intended to let researchers analyze ecological processes over large landscapes at fine resolutions.Source: §8 · p. 7
  • Circuitscape.jl is open source under the MIT license, with binaries available for download and installation through the Julia package manager.Source: §8 · p. 7
Loading 1906.03542v1…