Source-linked AI summary
Mantid - Data Analysis and Visualization Package for Neutron Scattering and $μSR$ Experiments
O. Arnold, J. C. Bilheux, J. M. Borreguero, A. Buts, S. I. Campbell, L. Chapon, M. Doucet, N. Draper, R. Ferraz Leal, M. A. Gigg, V. E. Lynch, A. Markvardsen, D. J. Mikkelson, R. L. Mikkelson, R. Miller, K. Palmen, P. Parker, G. Passos, T. G. Perring, P. F. Peterson, S. Ren, M. A. Reuter, A. T. Savici, J. W. Taylor, R. J. Taylor, R. Tolchenov, W. Zhou, J. Zikovsky
TL;DR
Neutron and muon facilities face complex, high-volume data-reduction and analysis demands that require detailed instrument knowledge. Mantid addresses this through an extensible framework combining algorithms, instrument geometry, fitting, visualization, and near-real-time integration. It is the main reduction software at SNS and ISIS and is used or considered at several other neutron facilities.
Problem
Growing data volumes and complex reduction chains make it difficult for individual researchers to transform instrument data into physically meaningful results.
Method
Mantid uses an extensible framework combining technique-specific algorithms, instrument definitions, fitting, visualization, and Python-based user extension.
Results
Mantid is the main reduction software in use at SNS and ISIS and is partially in use or under consideration at several other neutron facilities.
Takeaways & Limitations
Near-real-time integration at SNS and ISIS processes live data into scientifically useful results, enabling more efficient use of experiment time.
Takeaways & Limitations
Data-reduction chains must accommodate legacy requirements and be tightly controlled and validated against existing known-good results.
Abstract
from arXiv · showhide
The Mantid framework is a software solution developed for the analysis and visualization of neutron scattering and muon spin measurements. The framework is jointly developed by software engineers and scientists at the ISIS Neutron and Muon Facility and the Oak Ridge National Laboratory. The objectives, functionality and novel design aspects of Mantid are described.
1. Introduction
Mantid addresses the growing data-analysis burden at large-scale neutron and muon facilities by providing a common, extensible framework spanning experiments and later analysis. Its goals include broad scientific functionality, quality development, platform support, free distribution, and user extension.
- Motivation: Large facilities generate increasing data volumes, while data reduction and analysis remain complex and time-consuming for individual researchers.Data reduction transforms instrument datasets into physical units and requires detailed instrument knowledge.
- Project objective: Mantid bridges experiment and publication with a common interface for seamless data reduction and analysis at facilities and users’ home institutes.The framework is intended to support both experiment-time use and final analysis and fitting.
- Project goals: The framework provides technique-independent neutron- and muon-specific tools for data reduction, visualization, and scientific analysis.This is stated as one of the project’s main goals.
- Project goals: Mantid emphasizes professional software development practices to ensure quality.Quality is identified as a core project goal rather than an incidental implementation detail.
- Project goals: The project supports Linux, OS X, and Windows, and intends its software, source, and documentation to be freely distributable.These goals address platform availability and distribution.
- Project goals: Mantid is designed to be extensible by instrument scientists and users, with comprehensive, well-maintained documentation identified among its contributions.Extensibility and documentation are both explicit project objectives or contributions.
2. General Description of the Mantid Project
Mantid consolidates instrument-specific neutron and muon analysis into a professionally maintained, collaborative framework. Its abstraction and modular architecture support multiple facilities, data types, platforms, and user-built extensions.
- Project scope: Mantid is an international STFC–DOE collaboration developing a high-performance framework for neutron diffraction, scattering, reflectometry, SANS, and µSR analysis.The project began at ISIS in 2007, with SNS and HFIR joining in 2010 and later contributions from additional facilities.
- Motivation: Bespoke instrument software creates vast, unmanageable libraries that are vulnerable to single-point failures and loss of developer knowledge.Staff departures can force facilities to refactor existing mission-critical code.
- Project objective: Mantid provides professionally developed and maintained bespoke software under a science-led strategic development and maintenance plan.The stated objective is to benefit both facilities and their user communities.
- Design principles: The project’s abstraction ethos aims for code to operate across all data types from participating facilities, making the framework easier to use and maintain in principle.The claim is explicitly framed as an intended design consequence.
- Architecture: Mantid uses a modular C++/Python architecture with user-built plug-ins, ParaView visualization access, GPLv3 licensing, and support for commonly used operating systems.The modular design is intended to extend the framework to almost any application.
3. Neutron Scattering
Modern time-of-flight neutron facilities produce broad-coverage, high-volume datasets that complicate reduction. Event-mode acquisition adds filtering and time-resolved capabilities while increasing data-reduction complexity.
- Technique context: Neutron scattering determines material structure and dynamics, and experiments must relate measured counts to the physically meaningful S(q, ω).This relationship is central to interpreting neutron measurements.
- Technique context: Pulsed spallation sources use time-of-flight instruments that collect data over a wide range in S(q, ω) from a single pulse.The wide coverage is presented as an advantage of TOF instruments.
- Data scale: Modern instruments can generate gigabyte-scale files, combine several files into n-dimensional datasets up to 1 TB, and record event-mode neutron lists with timestamps and metadata.The passage describes both individual file scale and combined dataset scale.
- Event-mode acquisition: Event mode supports filtering by time or metadata, efficient sparse-data storage, and time-resolved experiments.These advantages arise because each detected neutron is stored with collection time and other metadata.
- Reduction challenge: Large data volumes, n-dimensional data, and event-mode formats add complexity to neutron data-reduction chains.The passage links these properties to additional reduction challenges.
4. Muon Spin Relaxation/Rotation/Resonance (µSR)
µSR provides local, time-dependent measurements across a broad range of materials and complements neutron scattering. Mantid supports shared analysis workflows for muon depolarization spectra and related simulations.
- Technique: µSR uses spin-polarized muons implanted into materials, tracking their polarization through positron decay as a function of time.The technique is described as similar to nuclear magnetic resonance in its polarization tracking.
- Scientific scope: µSR is a local probe with applications from soft matter to quantum magnetism and can provide dynamic information complementary to neutron scattering.Its broad applicability is explicitly stated.
- Framework rationale: Because neutron instruments and muon facilities can share users, a familiar analysis framework is beneficial.The passage connects this shared user base to Mantid’s role.
- Analysis capabilities: Mantid includes integrated asymmetry, Fourier transform, maximum entropy, and time-domain methods for analyzing muon depolarization spectra.These are examples among a wider range of supported methods.
- Analysis capabilities: Mantid also supports simulations of muon data using Density Functional Theory or electronic calculations.Such simulations are presented as a way to obtain further insights into materials.
5. Development Practices
Mantid uses distributed software-development practices to coordinate contributions and enforce cross-platform quality. Issue tracking, Git branching, continuous integration, and extensive automated testing support collaborative maintenance.
- About thirty scientists and scientific software engineers collaborate across Europe and the United States using issue tracking for feature requests and defect reports.
- Mantid uses Git repositories and separate feature branches, merging them onto a develop branch to check cross-platform compatibility.
- Over 6000 automated unit tests run across supported operating systems whenever code reaches the develop branch.
- Over 150 daily integration system tests run against the most recent locally installed version.
6. Mantid Design
Mantid separates data containers from technique-independent algorithms within a modular C++/Python framework. Its workspaces represent diverse data types and instrument context, while algorithms, live processing, scripting, and visualization support neutron and muon analysis.
- Core design: Algorithms operate, where possible, across all data types without prior knowledge of the experiment, separating data from processing.Workspaces hold data and associated information, while algorithms manipulate them.
- Workspaces: Workspaces can load data from files or live streams, be manipulated by algorithms, and save results in NeXus or other formats.
- Instrument geometry: Instrument Definition Files provide XML descriptions validated against an XML schema, supporting geometry-aware analysis and visualization.Expanded constructive-solid-geometry descriptions can also support complex detector visualization and Monte Carlo simulations.
- Workspace types: Mantid supports MatrixWorkspaces, EventWorkspaces, multi-dimensional workspaces with 1 to 9 dimensions, MDEventWorkspaces, and TableWorkspaces.EventWorkspaces can provide histogram views while retaining event data; MDEventWorkspaces support adaptive searching, binning, and plotting.
- Algorithms: The algorithm layer contains over 500 algorithms spanning data handling, arithmetic, unit conversion, diffraction, scattering, reflectometry, spectrometry, and µSR.
- Event processing: Event data can remain in event form through reduction chains, with specialized operations for filtering and other processing of sparse data.The data is often retained as events until final presentation.
- Workflows: Core algorithms can be combined into instrument or science-area workflows exposed through Python, the command line, or custom reduction interfaces.Workflow algorithms can also process live event streams and detector histogram data directly.
- Python API: The Python API allows runtime algorithm plugins and extension with new algorithms or fit functions without recompilation or restart.
7. User Interface
MantidPlot provides a unified interface for visualizing, processing, scripting, and fitting data, while custom interfaces support multiple science areas and instruments. Its fitting tools cover simple line data and multidimensional datasets, including resolution-broadened scattering models.
- MantidPlot: MantidPlot combines visualization, data processing, Python scripting, generic fitting, algorithm discovery, validation, help, and execution logs in one interface.Algorithms open automatically generated parameter dialogs, with invalid inputs flagged for users.
- Custom Interfaces: Custom interfaces extend Mantid’s simple analysis workflow across multiple science areas and instruments.The supported science areas and instruments are listed in Table 1.
- Fitting: Mantid supports fitting mathematical functions and models to one-dimensional x,y,e data through a simple interface or Python scripting.The interface supports peak selection, while fitting remains available through scripting.
- Fitting: Fitted models can be batch processed across datasets, with results stored as TableWorkspaces for further manipulation and analysis.Results can also be plotted against a log parameter.
- Fitting: Multidimensional fitting compares resolution-broadened scattering models with experimental S(q, ω) datasets and fits scattering-model parameters.Mantid calculates an instrument resolution function, convolutes it with a chosen model, and performs the comparison.
8. Visualization
Mantid provides visualization tools for multidimensional data that cannot be easily represented through 1D or 2D projections. These include instrument-centered 3D and 2D views, interactive slicing, and ParaView-based multidimensional rendering.
- Overview: Mantid addresses the visualization of large multidimensional datasets generated by instruments surveying broad reciprocal-space regions.Such datasets cannot be easily visualized through only 1D or 2D projections.
- Instrument View: Instrument View renders an instrument in 3D, annotates components, and colors detectors according to total integrated counts.It also provides 2D detector projections and interfaces for masking, grouping, and viewing spectra.
- Slice Viewer: The Slice Viewer provides interactive 2D projections of multiple data types, including line integration and overplotted PeaksWorkspaces.It is illustrated with single-crystal NaCl data from the SXD diffractometer.
- VSI: VATES integrates ParaView widgets and rendering libraries into Mantid for multidimensional visualization through the VSI.An API translates Mantid data structures into VTK structures that can also be saved for visualization in ParaView.
- VSI: VSI offers Multi Slice, Splatter Plot, and Three Slice views for exploring multidimensional data and single-crystal diffraction peaks.Splatter Plot supports interactive peak selection, while Multi Slice places multiple orthogonal slices on the data.
9. Community Involvement and Expandability
Mantid extends beyond its core framework through Python-based user contributions, shared scripts, and documentation. Its visualization and analysis capabilities are demonstrated across neutron-scattering datasets and experimental configurations.
- Expandability: The Python API allows users to extend Mantid for different applications and submit generated Python applications to a shared script repository.MantidPlot supports uploading, downloading, and automatic updating of shared scripts.
- Community Involvement: The community repository enables users to contribute and share scripts, including applications used in ways not originally envisaged by the development team.Extensive documentation supports this ecosystem through validation, code completion, offline help, and online examples and tutorials.
- Visualization Examples: ParaView-based visualization is demonstrated with single-crystal YFeO3 data from the SEQUOIA spectrometer at SNS.Supplementary movies show slices and varying energy or momentum coordinates for the dataset.
- Visualization Examples: VSI Splatter Plot mode is demonstrated with single-crystal data measured on the SXD diffractometer at ISIS.The view is oriented toward visualizing peaks in single-crystal diffraction data.
- Visualization Examples: VSI also displays diffraction data from a triclinic lattice with non-orthogonal axes measured on the CNCS spectrometer at SNS.The example illustrates visualization across non-orthogonal reciprocal-space geometry.
10. Facility Integration
Mantid integrates with facility archives and data-acquisition systems to automate reduction and support near-real-time analysis. These integrations locate data, invoke reduction scripts, and process live measurements into scientifically useful results.
- Archive Integration: Mantid interfaces with ICAT to search raw or processed files across facility archives with different storage approaches.A small archive-search adapter can be written against the provided interface for each facility.
- Autoreduction: ICAT supports autoreduction by invoking a reduction script as soon as files are created and catalogued.The script uses file metadata and/or catalogue metadata to reduce raw data into a form useful to users.
- Live Data: Near-real-time Mantid interfaces at SNS and ISIS process live acquisition data into scientifically useful results.This level of analysis allows more efficient use of experiment time.
11. Conclusion
Mantid is an extensible Python framework for neutron scattering and µSR data manipulation, analysis, and visualization. It is used for data reduction at SNS and ISIS, with broader adoption and continued work on performance, usability, and documentation.
- Mantid provides an extensible Python framework for data manipulation, analysis, and visualization in neutron scattering and µSR experiments.
- Mantid is the main reduction software at SNS and ISIS and is partially used or considered for adoption at other neutron facilities.
- Development continues to improve Mantid’s performance, usability, and documentation through professional practices and community feedback.
- Source code is available on GitHub, with binary installers for Linux, OS X, and Windows.