Source-linked AI summary
Parcels v0.9: prototyping a Lagrangian Ocean Analysis framework for the petascale age
Michael Lange, Erik van Sebille
TL;DR
The paper addresses the difficulty of scaling and customizing Lagrangian ocean analysis for increasingly large OGCM outputs. It introduces Parcels as a flexible framework with an API demonstrated through an application and accuracy tests, while identifying unstructured-grid support, efficiency, and OGCM coupling as future work.
Problem
Existing Lagrangian analysis codes may not scale easily to petascale OGCM data and make runtime coupling, flexible execution, and customized particle behavior difficult.
Method
Parcels defines a prototype API and composable abstractions for customizable, flexible Lagrangian particle simulations, with a conceptual blueprint for future versions.
Results
The paper demonstrates Parcels through a full scientific experiment and assesses its accuracy against idealized analytical tests as a benchmark for future versions.
Takeaways & Limitations
Parcels v0.9 provides a proof of concept for high-level Python construction of Lagrangian ocean experiments focused on customizability, flexibility, and ease of use.
Takeaways & Limitations
Unstructured-grid support is deferred to Parcels v1.0, while future development also targets greater efficiency and tools for runtime OGCM integration.
Abstract
from arXiv · showhide
As Ocean General Circulation Models (OGCMs) move into the petascale age, where the output from global high-resolution model runs can be of the order of hundreds of terabytes in size, tools to analyse the output of these models will need to scale up too. Lagrangian Ocean Analysis, where virtual particles are tracked through hydrodynamic fields, is an increasingly popular way to analyse OGCM output, by mapping pathways and connectivity of biotic and abiotic particulates. However, the current software stack of Lagrangian Ocean Analysis codes is not dynamic enough to cope with the increasing complexity, scale and need for customisation of use-cases. Furthermore, most community codes are developed for stand-alone use, making it a nontrivial task to integrate virtual particles at runtime of the OGCM. Here, we introduce the new Parcels code, which was designed from the ground up to be sufficiently scalable to cope with petascale computing. We highlight its API design that combines flexibility and customisation with the ability to optimise for HPC workflows, following the paradigm of domain-specific languages. Parcels is primarily written in Python, utilising the wide range of tools available in the scientific Python ecosystem, while generating low-level C-code and using Just-In-Time compilation for performance-critical computation. We show a worked-out example of its API, and validate the accuracy of the code against seven idealised test cases. This version~0.9 of Parcels is focussed on laying out the API, with future work concentrating on optimisation, efficiency and at-runtime coupling with OGCMs.
1 Introduction
Parcels is introduced as a flexible, scalable framework for Lagrangian ocean analysis aimed at petascale OGCM data and customizable particle behavior. Version 0.9 establishes the API and demonstrates accuracy evaluation, while performance and coupling remain future priorities.
- Motivation: Petascale OGCM outputs may require trajectory codes to handle petabytes of hydrodynamic data and gigabytes of particle trajectories.The authors identify scalability, online OGCM execution, and selective regional data filtering as key challenges.
- Motivation: Existing community codes primarily track water parcels or passive particulates, making behaviors such as sinking, fragmentation, and swimming difficult to incorporate.These capabilities would support studies of fish, icebergs, and marine debris.
- Contribution: Parcels v0.9 is a new Lagrangian particle-trajectory framework designed around scalability, performance, flexibility, and customizable particle-dispersion schemes.The framework is intended to remain adaptable to OGCM data and execution formats while allowing new functionality such as active behaviors.
- Evaluation: The paper presents a worked-out Parcels application and evaluates code accuracy against analytical solutions in idealized tests.The evaluation is positioned as a benchmark for future versions.
2 Prototype design and philosophy
Parcels v0.9 defines flexible interfaces and composable abstractions for dynamic Lagrangian particle tracking. Its Python API separates model specification from implementation while supporting adaptable data layouts, custom kernels, OGCM integration, and JIT performance optimization.
- Design philosophy: Parcels defines interfaces and composable abstractions intended to capture general domain-specific challenges in particle tracking.The framework is designed as a conceptual blueprint with three software layers.
- Design philosophy: The domain-specific-language approach separates scientific problem definitions from hardware-specific implementations through runtime code generation.This separation allows computational experts to optimize performance for changing HPC architectures.
- User-facing API: The user-facing Python API lets scientists define complete particle-tracking models concisely while customizing particle memory layouts and kernel behavior.Users can combine predefined and custom kernels to build complex particle behavior incrementally.
- Execution layer: Parcels accommodates dynamic particle sets, variable hydrodynamic sampling, and flexible particle-update loops for large-scale applications.Its execution layer establishes an interface for integration with external OGCMs and leaves room for later performance specialization.
- Data layout: High-level abstractions for particle and field data allow memory layouts to change without committing applications to a particular implementation.This modular structure supports different OGCM data formats and future optimization under the hood.
3 A worked-out example: tracking virtual foraminifera in the Agulhas region
Parcels is demonstrated through a backward-time experiment tracking virtual foraminifera in the Agulhas region, combining custom kernels, field sampling, and scalable data handling. The example follows particles from a Cape Town seafloor site through sinking and lifespan phases while recording trajectory properties.
- Scientific experiment: Particles first sink at 200 m per day while being advected by deep circulation, then are tracked backward at 50 m depth with temperature recorded daily.The experiment therefore combines sinking, horizontal advection, particle ageing, and along-trajectory temperature sampling.
- Input data: The OFES input provides three-dimensional velocities and temperature on 1/10° horizontal resolution, 54 vertical levels, and three-day averages for 2006.Snapshots 3165–3289 cover a subdomain around the core site off Cape Town.
- Data handling: Parcels reads hydrodynamic fields during integration, storing only three consecutive timeslices instead of loading the full 6GB example into memory.The same strategy is intended for larger regional domains or longer time series.
- Parcels API: The API defines a custom particle class with temperature and age variables, then combines sinking, temperature-sampling, ageing, advection, and boundary-deletion kernels.The kernels are concatenated with the built-in AdvectionRK4_3D kernel and executed through a ParticleSet.
- Execution and output: Particle integration runs in three-day loops that can release new particles every three days, advance the field window, and write trajectories and properties to indexed NetCDF output.Indexed output supports continually added and deleted particles and can be converted to array format for analysis.
4 Model evaluation
Parcels v0.9 treats accuracy as the primary validation focus rather than performance or speed. Performance is deferred to the v1.0 release.
- Evaluation focus: Parcels v0.9 concentrates on accuracy, while performance and speed are reserved for the v1.0 release.The authors identify optimisation and efficiency as future work.
4.1 Unit tests and continuous integration
Parcels uses unit testing and continuous integration to validate code integrity across its execution modes. The test suite covers a broad range of functionality, although its completeness cannot be guaranteed.
- Continuous integration: Every GitHub code change automatically triggers validation of the entire codebase through continuous integration.The validation uses the Travis CI web service.
- Unit tests: The tests include component checks such as verifying that ParticleSet particles receive their assigned longitudes and latitudes.These small checks are intended to help prevent bugs.
- Unit tests: Parcels v0.9 has over 150 unit tests checking the integrity and consistency of the codebase.Where relevant, tests run in both SciPy and JIT modes.
- Scope: The authors attempted broad functionality coverage, but acknowledge that unit-test completeness can never be guaranteed.This limits what the test suite alone can establish about all possible code changes.
4.2 Idealised and analytic test cases
Parcels v0.9 is validated against seven idealised and analytical test cases using RK4 integration in JIT mode. The reported trajectories closely match analytical solutions across rotating, spherical, time-varying, gyre, inertial, and diffusive flows.
- Test design: Seven idealised and analytical test cases are used to validate Parcels v0.9 accuracy with RK4 integration in JIT mode.Velocities are generated in Python and converted directly into FieldSets.
- Solid-body rotation: A solid-body rotation test yields a maximum distance error below 3 mm after 24 hours over path lengths exceeding 5 km.All four particles complete the expected full circle.
- Spherical coordinates: In spherical zonal flow, the particles travel 4.9·10^6 m in 57 days, corresponding to almost 45° longitude at the equator.Particles at higher latitudes travel farther in degrees because zonal distance depends on cosine latitude, while orthographic projection shows equal travel distances.
- Time-varying flow: In the time-varying flow test, the largest positional errors after four days are 6 cm zonally and 4 mm meridionally.All particles follow the analytical pathways very closely.
- Peninsula flow: For steady flow around a peninsula, the largest streamfunction error is 0.008 m^2/s, corresponding to a positional error of 10^-5 degrees, or 1 m.The trajectories closely follow the analytical streamlines.
- Stommel gyre: In the Stommel gyre test, particles remain close to their streamlines over 50 days, with a largest streamfunction error of 0.05 m^2/s and positional error below 5 km.Streamline deviation is used as the accuracy measure.
5 Future outlook
Parcels v0.9 establishes an API and design philosophy intended to support flexible particle behavior, composable kernels, and future efficiency and coupling improvements. The outlook identifies data movement and parallel execution as central priorities.
- The API and design philosophy are the paper’s core contributions, providing a conceptual basis for future framework improvements.
- Dynamic code generation enables runtime kernel specialization while preserving a clear initial kernel interface.
- Hydrodynamic field-data interaction can dominate computation when large datasets must be read from files.
- Direct OGCM coupling can avoid disk-bandwidth bottlenecks for local, high-particle-density simulations, but requires additional computation.
- The v0.9 prototype is not parallel because its NetCDF workflow lacks domain decomposition and parallel reading, and particle exchange is unsupported.
- Custom particle behaviors, including sinking and aging, can be implemented in a few lines of Python and applied across diverse oceanographic models.
- The execution engine is designed for at-runtime OGCM compatibility, including extensible field interfaces, host-controlled timesteps, generated C kernels, and error recovery.
6 Conclusions
The conclusions present Parcels v0.9 as a prototype demonstrating a customizable, flexible, and easy-to-use API through scientific experiments. Accuracy is assessed as a benchmark, while efficiency and OGCM runtime integration remain future goals.
- Parcels v0.9 provides a proof-of-concept API for writing full-fledged scientific experiments in high-level Python.
- The current implementation’s accuracy is assessed to provide a benchmark for future versions.
- Future development will target greater framework efficiency and tools for porting generated C code into OGCM runtime integration.
7 Code availability
The Parcels v0.9 code is openly available under the MIT license through GitHub, with an archived version on Zenodo and further project information online.
- The Parcels code is licensed under the MIT license and available through the OceanParcels GitHub repository.
- Version 0.9 is archived on Zenodo, with additional information provided on the project webpage.