PaperlayerView exact revision on arXiv

Source-linked AI summary

LHAPDF6: parton density access in the LHC precision era

Andy Buckley, James Ferrando, Stephen Lloyd, Karl Nordstrom, Ben Page, Martin Ruefenacht, Marek Schoenherr, Graeme Watt

arXiv:1412.7420v2hep-phhep-ex

TL;DR

LHAPDF 5’s static-memory, multiset, metadata, and release-management limitations hindered concurrent PDF use and reliable precision studies. LHAPDF 6 redesigns the system around dynamic C++ PDF objects and a universal data format, substantially reducing memory requirements while enabling efficient concurrent full-set use. The library is recommended for precision collider-physics PDF access, with scope limited to interpolated PDFs and related supported use cases.

  • Problem

    LHAPDF 5 imposed static-memory and multiset limitations, created metadata correctness risks, and coupled PDF availability to slow software releases.

  • Method

    LHAPDF 6 is a ground-up C++ redesign using dynamic PDF objects, centralized access through a universal grid format, and Python support.

  • Results

    LHAPDF 6 enables efficient simultaneous use of multiple full PDF sets while greatly reducing memory requirements compared with LHAPDF 5.

  • Takeaways & Limitations

    LHAPDF 6 is recommended as the production PDF-access library for high-precision collider physics beginning with LHC Run 2.

  • Takeaways & Limitations

    LHAPDF 6 is intentionally focused on interpolated PDFs and does not plan built-in QCD evolution, resolved virtual-photon structure functions, or transverse-momentum-dependent PDFs.

Abstract

from arXiv · show

The Fortran LHAPDF library has been a long-term workhorse in particle physics, providing standardised access to parton density functions for experimental and phenomenological purposes alike, following on from the venerable PDFLIB package. During Run 1 of the LHC, however, several fundamental limitations in LHAPDF's design have became deeply problematic, restricting the usability of the library for important physics-study procedures and providing dangerous avenues by which to silently obtain incorrect results. In this paper we present the LHAPDF 6 library, a ground-up re-engineering of the PDFLIB/LHAPDF paradigm for PDF access which removes all limits on use of concurrent PDF sets, massively reduces static memory requirements, offers improved CPU performance, and fixes fundamental bugs in multi-set access to PDF metadata. The new design, restricted for now to interpolated PDFs, uses centralised numerical routines and a powerful cascading metadata system to decouple software releases from provision of new PDF data and allow completely general parton content. More than 200 PDF sets have been migrated from LHAPDF 5 to the new universal data format, via a stringent quality control procedure. LHAPDF 6 is supported by many Monte Carlo generators and other physics programs, in some cases via a full set of compatibility routines, and is recommended for the demanding PDF access needs of LHC Run 2 and beyond.

1 Introduction

PDFs encode hadron momentum structure and enter collider cross-section calculations through convolutions with partonic cross-sections. LHAPDF standardises access to PDF sets, with LHAPDF 6 moving toward dynamic, concurrent PDF use.

  • PDF definitions: PDFs encode the process-independent momentum structure of partons within hadrons for collider cross-section calculations.At leading order, they represent the probability of finding a parton with a given flavour and momentum fraction; this interpretation changes at higher orders.
  • PDF definitions: PDF normalization is constrained by momentum conservation, while baryon-number conservation supplies flavour sum rules.
  • PDF definitions: Collinear PDFs are two-variable functions of momentum fraction x and factorization scale Q, with parton momenta aligned to the parent hadron.
  • LHAPDF: LHAPDF provides uniform access to many PDF fitting results, while LHAPDF 6 replaces Fortran static memory with C++ dynamic PDF objects and concurrent usage.
  • PDF use: In collider cross-section calculations, PDFs are convolved with partonic cross-sections and summed over contributing initial-state partons.

2 History and evolution of LHAPDF

LHAPDF evolved from PDFLIB replacement efforts but accumulated limitations from custom interpolation code, static memory, multiset handling, and release-coupled data provision. These constraints affected memory use, concurrency, correctness, maintainability, and advanced PDF uncertainty studies.

  • Origins: PDFLIB became unsustainable as growing PDF fits required storing interpolation data, prompting LHAPDF to pursue a scalable replacement.
  • Origins: LHAPDF’s original dynamic-evolution model also broke down because each parameterization required custom library code and its bundled QCDNUM became outdated.
  • LHAPDF 5 limitations: Multiset mode multiplied allocated interpolation space by NMXSET, enabling only a few concurrent sets and failing to integrate seamlessly with the library.
  • Performance problems: LHAPDF 5’s technical problems arose from interpolation routines and multiset mode rooted in Fortran static memory allocation.
  • Performance problems: The low-memory build reduced footprint but stored only one member per set, making repeated reinitialization prohibitively slow for PDF uncertainty reweighting.
  • Performance problems: Global state and shared memory made Fortran LHAPDF unsafe for multithreaded applications, restricting scalability on multicore systems.
  • Correctness problems: LHAPDF 5 could report incorrect validity ranges and metadata, including active-set mismatches for αS and x–Q boundaries in multiset mode.
  • Maintainability problems: Release-coupled PDF provision, wrapper maintenance, absent data-version tracking, and shared data files created logistical and reproducibility risks.

3 Design of LHAPDF 6

LHAPDF 6 redesigns PDF access around object-oriented C++ interfaces, dynamic object management, general parton content, and centralized metadata and interpolation mechanisms. Its design supports flexible PDF creation while retaining interpolated grid-based providers and configurable numerical behavior.

  • Core redesign: LHAPDF 6 is a ground-up C++ redesign that addresses the static-memory and retrofitted-feature problems of earlier Fortran versions.The new implementation is object oriented and targets the limitations identified in LHAPDF 5.
  • Core objects: The PDF interface represents complete multi-flavour parton-density objects, while PDFSet provides convenient access to set metadata and members.Single-flavour densities are not represented by a separate LHAPDF 6 user-interface type.
  • Parton content: LHAPDF 6 permits completely general parton flavours identified by standard PDG Monte Carlo ID codes, rather than special-case non-standard-flavour hacks.The gluon alias 0 for PDG code 21 is retained for backward compatibility.
  • Metadata: Cascading metadata attaches typed values to PDFs, PDF sets, or global configuration and resolves them from specific to general scopes.For example, αS(MZ) falls back from PDF to PDFSet to global configuration before an error or user-supplied default.
  • Object and memory management: Users may create any number of PDF objects at runtime without initializing whole sets or observing the NMXSET concurrent-set limit, but must release allocated memory.Factory functions beginning with “mk” return pointers whose deletion is the user’s responsibility.
  • Interpolation and thresholds: The current provider is GridPDF, which interpolates rectangular (x, Q2) grids and supports multiple Q2 subgrids for threshold discontinuities.Runtime-configurable interpolators allow controlled algorithm evolution, while generalized threshold handling is planned beyond the current heavy-quark-mass assumption.

4 Usage examples

LHAPDF 6 provides usage interfaces in C++, Python, and Fortran. Fortran access is supplied through a legacy compatibility layer, while the examples demonstrate PDF creation and value retrieval in each language.

  • Supported languages: LHAPDF 6 supports C++, Python, and Fortran usage, with Fortran provided through a legacy LHAPDF 5 API compatibility layer.The paper presents brief acquisition and use demonstrations in all three languages.
  • C++: C++ usage creates a PDF with mkPDF, evaluates xfxQ, and explicitly deletes the heap-allocated object.The example uses CT10/0 at x = 1e-4 and Q = 100.
  • Python: Python usage creates an MSTW2008nlo68cl member, evaluates xfxQ for its flavours, and accesses a PDFSet object.The example evaluates at x = 1e-3 and Q = 100.
  • Fortran: Fortran usage declares PDF variables, sets x and q, and initializes a named LHAPDF grid and member through compatibility routines.The example uses InitPDFsetByName("CT10.LHgrid") followed by InitPDF(0).

5 Data formats

LHAPDF 6 standardizes PDF metadata and interpolation data around a unified, directory-based format. Its cascading YAML metadata and separate member files support flexible configuration, faster lookup, and random access.

  • 5 Data formats: LHAPDF 6 uses one metadata system and one unified interpolation-grid format through GridPDF for currently active PDFs.This includes sets migrated from LHAPDF 5 and sets supplied directly to LHAPDF 6.
  • 5.1 Metadata format: Metadata uses YAML key–value dictionaries with boolean, string, integer, floating-point, and list value types.The same general syntax controls system behavior and stores PDF physical information.
  • 5.1 Metadata format: Each PDF data file begins with YAML, each set has a .info file, and global configuration is stored in lhapdf.conf.More specific metadata overrides values defined at more global levels.
  • 5.1.1 System-level metadata: System-level metadata is loaded from lhapdf.conf and supplies defaults such as the logcubic interpolator, continuation extrapolator, particle masses, and verbosity.The configuration is accessed through LHAPDF::getConfig().
  • 5.1.2 Set-level metadata: Each PDF set is a directory containing one .info file and member-specific data files, with common set metadata stored at set level.Set metadata is accessed through the lightweight PDFSet object.
  • 5.1.3 Member-level metadata: The directory structure enables faster set-level metadata lookup and random access to individual members than LHAPDF 5’s one-file-per-set structure.Member metadata can also be loaded without reading the larger data block.

5.2 PDF grid data format

LHAPDF 6 stores each PDF member in a separate, metadata-bearing grid file, enabling random member access and space-efficient distribution. Grid blocks encode knot coordinates, parton identifiers, and xf values for interpolation.

  • PDF grid data format: Each PDF member has its own zero-padded .dat file, with member 0 reserved for the central PDF.The set directory uses the member number in each filename, supporting direct access to individual members.
  • PDF grid data format: Splitting data into one file per member enables faster random access and allows installations to retain only selected members.This can substantially reduce storage for resource-constrained applications such as Grid deployments.
  • PDF grid data format: Each member file begins with YAML metadata, followed by Q-ordered subgrids separated by YAML document delimiters.The member header identifies the format and PDF type, while each subgrid is separately structured.
  • PDF grid data format: Every subgrid header lists x knots, Q knots, and ordered PDG parton IDs before the tabulated PDF values.The files use Q for human-readable scale coordinates, while interpolation operates in Q2 or log Q2.
  • PDF grid data format: Grid values are written as xf(x; Q) columns, with rows ordered by nested loops over x and Q knots.The parser checks the consistency of the grid format and its subgrid data.

5.3 αS interpolation data format

The αS interpolation format stores interpolation knots and scale positions in metadata, while the distribution system separates PDF data updates from software releases. Indexing and version metadata support reliable discovery and compatibility management.

  • 5.3 αS interpolation data format: αS interpolation uses equal-length AlphaS Vals and AlphaS Qs lists for log-cubic interpolation.Repeated Q boundaries mark subgrid transitions, with αS limits supplied from below and above.
  • 5.4 Index file: Global LHAPDF ID lookup uses pdfsets.index, which records each set’s ID, name, and latest data version.The index is needed for ID-based lookup but not for set-name plus member-number access.
  • 5.4 Index file: Each set’s .info file provides the first central-member ID through SetIndex metadata.The index and online PDF-set documentation are generated automatically by querying these metadata files.
  • 5.5 Distribution and updating: LHAPDF 6 decouples PDF-data releases from software releases, allowing new sets to be released independently.This separation was a major design goal of LHAPDF 6.
  • 5.5 Distribution and updating: PDF sets are distributed as compressed archives containing a set directory with .info metadata and .dat data files.Typical 50-member sets occupy 5–10 MB compressed and 20–30 MB expanded, while 100-member NNPDF sets are larger.
  • 5.5 Distribution and updating: The lhapdf management script can list and install PDF sets, while DataVersion tracks bug-fix updates.Official distributions include updated index files through the LHAPDF website and CERN AFS/CVMFS systems.
  • 5.5 Distribution and updating: Future sets may declare MinLHAPDFVersion when they require features introduced only in later LHAPDF 6 releases.This compatibility flag has not yet been needed in the described system.

6 PDF uncertainties

LHAPDF 6 adds automatic PDF-uncertainty and correlation utilities that hide set-specific formulae from users. The implementation supports Hessian, symmetric-Hessian, and replica error types, plus random replica generation from Hessian sets.

  • 6 PDF uncertainties: PDF uncertainty propagation requires set-specific formulas and confidence-level rescaling, creating opportunities for inconsistent implementations.Without automation, users or codes may duplicate formula logic and mishandle uncertainty normalization.
  • 6 PDF uncertainties: LHAPDF 6 implements uncertainty functions in C++ and selects formulas according to the PDF set’s ErrorType.The supported types are hessian, symmhessian, and replicas.
  • 6 PDF uncertainties: The uncertainty function returns central, asymmetric, and symmetric uncertainties together with the confidence-level rescaling factor.The default confidence level corresponds to 1-sigma, approximately 68.268949%.
  • 6 PDF uncertainties: Hessian, symmetric-Hessian, and replica sets use distinct prescriptions based respectively on eigenvector members, symmetric eigenvectors, or Monte Carlo samples.Replica defaults use the sample average and standard deviation, while an alternative interval uses the replica distribution.
  • 6 PDF uncertainties: The HERAPDF/ATLAS VAR model and parametrisation errors are not currently supported.This limitation reflects differences between the prescriptions used by those sets.
  • 6 PDF uncertainties: The correlation utility returns a cosine in [−1, 1], where values near 1, −1, and 0 indicate correlation, anticorrelation, and no correlation.The uncertainty inputs depend on the selected error prescription.
  • 6 PDF uncertainties: Hessian sets can be converted into Monte Carlo replicas using random Gaussian numbers and the randomValueFromHessian function.The resulting replicas support applications including Bayesian reweighting and combinations of PDF predictions.

7 PDF reweighting

PDF reweighting lets existing event samples be evaluated with alternative PDFs, but its validity depends on perturbative order, shower treatment, and αS consistency. LHAPDF 6 provides helper functions for one- and two-beam weight calculations.

  • 7 PDF reweighting: PDF reweighting changes an existing event sample by replacing the original PDF factors with factors from an alternative PDF.It is especially useful when many PDF members would otherwise require separate Monte Carlo samples.
  • 7 PDF reweighting: A single well-defined partonic initial state exists only at tree level, so higher-order reweighting requires information about subtraction counter-terms.The paper therefore places such reweighting more appropriately in NLO generator code.
  • 7 PDF reweighting: Parton-shower simulations also contain PDF factors in Sudakov form factors, whose consistent reweighting would require summing over emission histories.That procedure had not yet been formalised or implemented in the described programs.
  • 7 PDF reweighting: PDF reweighting should use PDFs with similar αS values, and LO-to-NLO reweightings are strongly discouraged.The caution applies because the strong coupling appears in both matrix-element and shower calculations but is often not reweighted consistently.
  • 7 PDF reweighting: LHAPDF 6 supplies single-beam and two-beam helper functions that calculate reweighting factors and can warn when αS values differ beyond a threshold.The default relative αS warning threshold is controlled by the aschk argument.

8 LHAPDF 5 / PDFLIB compatibility

LHAPDF 6 preserves access for existing Fortran and LHAPDF 5 C++ applications through localized compatibility interfaces, while adding version-detection tools for migration. These interfaces have been tested with widely used generators, although the improved native Fortran interface remains incomplete.

  • Compatibility interfaces: LHAPDF 6 provides C++ compatibility interfaces for the Fortran LHAPDF, PDFLIB, and LHAPDF 5 C++ interfaces.These interfaces are localized to LHAGlue.h and LHAGlue.cc.
  • Compatibility interfaces: The Fortran compatibility layer uses extern "C" linkage and numbered slots managed by a state machine for legacy PDF object creation and deletion.This bridges LHAPDF 6’s dynamic allocation with LHAPDF 5’s static slot model.
  • Compatibility interfaces: LHAPDF 6 preserves legacy PDF naming behavior by stripping obsolete .LHpdf and .LHgrid extensions and correcting the historical cteq6ll.LHpdf spelling.The misspelled name is translated to cteq6l1 during loading.
  • Compatibility interfaces: The Pythia6LambdaV5Compat metadata flag preserves the historically incorrect Λ(4)QCD behavior required by existing PYTHIA 6 tunes.The flag defaults to true in lhapdf.conf and can be changed through configuration or the metadata API.
  • Migration support: Three C++ version macros expose string, encoded-integer, and major-version information for portable preprocessor-based migration between LHAPDF 5 and 6.LHAPDF_VERSION_CODE encodes release 6.1.4 as 60104.
  • Uptake and prospects: Legacy interfaces were tested with PYTHIA 6, HERWIG 6, POWHEG-BOX, and aMC@NLO, while major C++ generators support LHAPDF 6 through its native API.The improved Fortran interface was still at an initial stage.

9 Benchmarking and performance

LHAPDF 6 replaces LHAPDF 5’s large static-memory design with dynamic allocation and improves performance in suitable access patterns. Its memory use becomes demand-driven, while CPU and disk-speed effects depend on interface usage and application context.

  • Memory requirements: 2.04 GB of static memory was declared by LHAPDF 5.9.1, making the library incompatible with LHC computing systems.Restricted-memory builds supported only basic event generation.
  • Memory requirements: 280 kB is LHAPDF 6’s total static memory footprint, with static uninitialised data reduced from more than 2 GB to 1.6 kB.Active PDF allocation is dynamic: a GridPDF typically requires a few hundred kB, while a whole set requires O(10 MB).
  • Memory requirements: LHAPDF 6 allows users to control PDF allocation and deallocation and to load individual PDF set members, unlike LHAPDF 5.This dynamic model addresses the static-memory limitation underlying LHAPDF 5’s large arrays.
  • CPU performance: LHAPDF 6 is significantly faster when one flavour is requested, but slightly slower when all flavours are evaluated at every (x, Q2) point.The difference arises because LHAPDF 5 returned all 13 flavour values together, whereas single-flavour LHAPDF 6 avoids twelve unused interpolations.
  • CPU performance: All Sherpa benchmark cases run faster with LHAPDF 6 than with version 5, with process- and PDF-specific speedups of factors of 2–6.Table 3 compares phase-space integration and CKKW-merged event generation using interfaces optimized for each version.
  • Future performance directions: LHAPDF 6 does not provide specific OpenMP or MPI interfaces, although it is largely safe for applications using multi-core frameworks.Further vectorization and GPU approaches involve numerical, memory, complexity, or portability trade-offs.
  • Initialization and disk space: Whole-set loading in LHAPDF 6 is as fast as in LHAPDF 5 after using a custom ASCII parser, while individual-file storage avoids scanning unused members.Typical PDF-set loading takes less than one second.
  • Initialization and disk space: LHAPDF 6 data sizes are broadly similar to LHAPDF 5, while compression can reduce storage by a typical factor of 3–4.For example, CT10nlo grows from 21 MB to 33 MB, whereas NNPDF 2.3 NLO decreases from 95 MB to 84 MB.

10 PDF migration and validation

The authors migrated more than 200 recent PDF sets into the universal GridPDF format and validated them through automated scans and author review. The comparison procedure targeted fractional deviations below 10^-3 for most PDFs and exposed discrepancies useful for debugging.

  • Migration: A Python migration script extracted LHAPDF 5 grid knots and PDF data into the new GridPDF format.The script was extended to support subgrids, member-specific metadata, and automated conversion of hundreds of PDFs.
  • Migration: More than 200 PDF sets were migrated, generally selecting the most recent set in each PDF family unless older versions were specifically requested.The approved sets span the ATLAS, CTEQ and CJ, HERAPDF, MRST, MSTW, and NNPDF collaborations.
  • Validation procedure: Validation scanned xf values across x ∈ [10^-10, 1] and Q ∈ [1, 10^4] GeV, using logarithmic grids and fixed x or Q values.The same C++ code was used for the migrated and original versions, with αS also scanned in log Q.
  • Validation procedure: 10^-3 was adopted as the ad hoc fractional-difference tolerance after consultation with PDF authors.The metric suppresses differences as xf approaches zero to reduce false alarms.
  • Validation outcome: For the CT10nlo central gluon PDF, v5 and v6 fractional differences remain below one part in 1000 except at the lowest Q point.Oscillations arise from small interpolation differences despite identical interpolation knots.
  • Validation outcome: The 10^-3 validation level was achieved almost everywhere for the majority of PDFs, while detected differences helped debug code, migration, and interpolation-grid stability.Validation plots were checked by the original set authors and the LHAPDF 6 team before release.

11 Summary and prospects

LHAPDF 6 became operational with expanded capabilities for PDF access, uncertainty calculations, and reweighting. Its unified design improves memory use and extensibility while retaining compatibility, although its scope remains intentionally focused on LHC applications.

  • Release and capabilities: LHAPDF 6.0.0 was officially released in August 2013 after a lengthy public testing period, while maintaining compatibility with LHAPDF 5 interfaces.The release also introduced more powerful models for dynamic PDF-memory allocation and parton-density metadata.
  • Release and capabilities: The unified data format and interpolation routines decouple new PDF-set releases from the slower LHAPDF software-release cycle.The design supports general parton content, including polarised partons, photon constituents, and fragmentation-function implementations.
  • Performance: Vastly reduced memory requirements allow efficient simultaneous use of multiple full PDF sets, which was unfeasible with LHAPDF 5 Grid-distributed builds.Single-flavour interpolation can also improve CPU performance in reweighting applications that evolve at most two parton flavours per event.
  • Release and capabilities: New tools support PDF uncertainty and reweighting calculations for increasingly complex experimental and phenomenological uses of PDFs.These tools respond to the increasing use of PDF reweightings in collider-physics workflows.
  • Scope and limitations: LHAPDF 6 intentionally excludes built-in QCD evolution, resolved virtual-photon structure functions, and transverse-momentum-dependent PDFs.External evolution libraries can generate grid files or be called at runtime, while the excluded PDF types require additional interpolation-space parameters.
  • Scope and limitations: Nuclear corrections to nucleon PDFs are not currently supported transparently, although external correction factors and hard-coded nuclear grids can be used.A near-future LHAPDF version was planned to address transparent nuclear corrections.
  • Conclusion: LHAPDF 6 is fully operational and recommended as the production LHAPDF version for high-precision collider physics beginning with LHC Run 2.The paper summarizes the release as offering significant performance and capability improvements over LHAPDF 5.
Loading 1412.7420v2…