PaperlayerView exact revision on arXiv

Source-linked AI summary

Parameter Sensitivity Analysis for Aerial LiDAR-Inertial Odometries in low-altitude flights

Robert Milijas, Jose Ramiro Martinez-de Dios, Stjepan Bogdan

arXiv:2609.12837v1cs.RO

TL;DR

LiDAR SLAM and LIO performance depends on scenario, sensor, and robot motion, making tuning intensive. The paper analyzes Cartographer and FAST-LIO2 parameter effects across aerial datasets, then provides tuning recommendations. With those recommendations, both algorithms achieve ATE within 5 cm of grid-search optimal performance in 94% of analyzed cases.

  • Problem

    LiDAR SLAM and LIO require extensive, time-consuming tuning because performance depends on environment, robot motion, and sensor, and settings may not transfer across scenarios.

    Source: §1 · p. 1

  • Method

    The study analyzes Cartographer and FAST-LIO2 parameter effects on aerial datasets using Pearson correlation and random-forest permutation importance across diverse LiDARs, environments, and flights.

    Source: §1 · p. 2, §3 · p. 4, §3.1 · p. 4

  • Results

    In 94% of analyzed cases, both algorithms achieved ATE values within 5 cm of the optimal performance found by exhaustive grid search using the proposed tuning recommendations.

    Source: §5 · p. 17

  • Takeaways & Limitations

    The recommendations provide starting parameter values and a tuning order, with a simplified procedure that focuses manual tuning on each algorithm’s two most influential parameters.

    Source: §1 · p. 2, §5 · p. 16

    The analysis focused on lower-altitude flights and did not consider parameter interactions.

    Source: §5 · p. 17

Abstract

from arXiv · show

LiDAR-based SLAM (Simultaneous Localization and Mapping) and LIO (LiDAR-inertial odometry) algorithms are often used for precise navigation of unmanned aerial vehicles, especially during interactions with the aerial robot's environment. However, the performance of these algorithms is greatly dependent on the scenario, LiDAR, and robot motion characteristics, often requiring an intensive tuning process to achieve the desired performance. To aid these tuning efforts, this paper analyzes the influence on performance of the parameters of an EKF-based LIO algorithm (FAST-LIO2) and the LIO module of a graph-based SLAM algorithm (Cartographer) on aerial LiDAR SLAM datasets recorded using different LiDARs in low-to-moderate-altitude flights in diverse environments. The analysis is conducted on the absolute trajectory error (ATE) resulting from processing the datasets with the LIO algorithms configured with each combination of parameters obtained in an exhaustive grid search. The relationship between individual parameters and the ATE results is assessed using Pearson's correlation, while the influence of each parameter is assessed using random forest permutation importance analyses with random forest models trained to predict the resulting ATE values based on the choice of parameters. The performed analysis obtains for Cartographer and FAST-LIO2: i) the identification of parameters with stronger influence in performance, ii) a simplified tuning procedure, and iii) tuning recommendations. Using the proposed tuning recommendations, both algorithms obtain on the analyzed datasets ATE values within 5 cm to the optimal performance found in the grid search procedure in 94% of the analyzed cases.

1. Introduction

LiDAR SLAM and LIO performance varies with environment, robot motion, and sensor, making tuning time-consuming and scenario-dependent. This work studies Cartographer and FAST-LIO2 to identify influential parameters and simplify tuning.

1. Introduction

  • Parameters tuned for some environments may fail in others, and strong symmetry or limited geometric information also challenge these methods.Tuning is described as a time-consuming deployment step, especially across environments or when changing sensor payloads.Source: §1 · p. 1
  • Pearson correlation and random-forest analyses identify high- and low-influence parameters and trends across LiDAR, environment, and flight conditions for Cartographer and FAST-LIO2.The analysis aims to narrow the parameter search space and facilitate tuning.Source: §1 · p. 2
  • The proposed procedure manually tunes each algorithm’s two most influential parameters and uses recommended values for the others, achieving performance comparable to exhaustive-search optima on analyzed datasets.The paper also provides and validates tuning recommendations and procedures for both algorithms.Source: §1 · p. 2
  • The paper provides over 68 GB of estimated trajectory data, 6,900 hours of trajectories, and 140,000 LIO performance data points to support further analysis.The processed data is to be made publicly available as supplementary material upon paper acceptance.Source: §1 · p. 2

2. Related work

Earlier SLAM and LIO parameter exploration often combined algorithm settings with hardware or compiler choices. This work instead examines LiDAR-based performance trends to inform parameter tuning for aerial robots.

2. Related work

  • Earlier KinectFusion and ElasticFusion design explorations sampled large parameter spaces and trained random-forest regressors, but reported no data on model performance.Related work also explored hardware, compiler, accuracy, runtime, and power-consumption factors.Source: §2 · pp. 2–3
  • Unlike RGB-D Pareto-search studies,,, this work seeks general LiDAR performance trends, including influential parameters uncorrelated with performance and a recommended tuning order.The analysis uses random-forest permutation-based feature importance to assess parameter influence.Source: §2 · p. 3
  • The analysis uses exhaustive parameter search across 60 LiDAR, environment, and flight combinations, focusing on accuracy for medium-large quadrotors rather than hardware, runtime, or power factors.The authors state that using real rather than modeled data supports correlation analysis and train/test validation of random-forest results.Source: §2 · p. 3
  • The study covers graph-based Cartographer and IKF-based FAST-LIO2; Cartographer’s loop-closure mechanisms were disabled to isolate its graph-based LIO module.Cartographer uses a bi-resolutional 3D grid map, while FAST-LIO2 uses an ikd-tree map representation.Source: §2 · p. 3

3. Methodology

The methodology evaluates parameter combinations across aerial LiDAR sequences, then analyzes their relationship to ATE and distills tuning recommendations. It combines diverse low-to-moderate-altitude datasets with correlation and random-forest analyses.

3.1. Overview of used aerial SLAM datasets

  • Two datasets cover 60 sequence–LiDAR combinations, including 16 PASTEL flights with three LiDAR models and varied environments and flight characteristics.NTU VIRAL complements PASTEL with higher average altitudes and greater obstacle distances.Source: §3 · p. 4, §3.1 · pp. 4–5

3.4. LIO evaluation pipeline

  • The evaluation grid tested selected low, medium, and high settings; over 140,000 trajectories were generated and scored using ATE.Cartographer grid values came from the authors’ LARICS settings, while FAST-LIO2 values doubled or halved its authors’ defaults.Source: §3 · p. 4, §3.4 · pp. 6–7

3. Methodology

  • Both algorithms’ tuning recommendations produced ATE values within 5 cm of grid-search optima in 94% of analyzed cases.The recommendations specify tuning order, parameter values, and policies.Source: §3 · p. 4

3.2. Cartographer parameters chosen for tuning

  • Cartographer’s LIO subsystem has more than 30 tunable parameters; the analysis excludes hardware-dependent and additional-filtering parameters, leaving others at defaults.Default settings were checked for coarse performance degradation and faster-than-real-time execution.Source: §3.2 · p. 5

3.3. FAST-LIO2 parameters chosen for tuning

  • FAST-LIO2’s search was narrowed to ten parameters, with hardware-dependent settings left at defaults and weakly influential parameters removed after coarse sensitivity tests.The study adds LiDAR_max_range as a preprocessing cutoff, zeroing points beyond the threshold.Source: §3.3 · pp. 5–6

3.5. Correlation analysis

  • Positive Pearson correlation associates larger parameter values with higher ATE, negative correlation with lower ATE, while weak correlation can signal ambiguity or little effect.For Cartographer’s LiDAR_max_range, effects varied across sequences and sensors; Figure 2 also compares probabilities of ATE changes exceeding 1 cm.Source: §3.5 · p. 7, Figure 2 · p. 8

3.4. LIO evaluation pipeline

  • PASTEL grid-search ATE averaged 14 cm (54%) below default results; NTU VIRAL results could not be compared with Cartographer or FAST-LIO2.Table 5 reports the best ATE for each algorithm, sequence, and LiDAR combination.Source: Table 5 · p. 7, §3.4 · p. 7

3.6. Random forest regressors and feature importance

  • Random forests predict ATE from parameter choices, and permutation importance ranks parameter influence; test RMSE averaged 58% below testing ATE standard deviation.Models were tested on the held-out 20% of performance data; Table 6 summarizes Cartographer test comparisons.Source: §3.6 · pp. 8–9, Table 6 · p. 9

4. Results

The results are interpreted in three stages: random-forest importance identifies influential parameters, correlation analysis informs manual tuning, and cross-case trends lead to recommendations that are validated. Cartographer’s permutation-importance table reports baseline and per-parameter RMSE values grouped by LiDAR.

4. Results

  • The analysis proceeds from random-forest importance, through correlation-based assessment of manual tuning needs and overarching trends, to recommendations that are ultimately validated.The interpretation begins by identifying high- and low-influence parameters for each sequence-LiDAR combination.Source: §4 · p. 9
  • Cartographer’s permutation-importance table groups results by LiDAR and reports baseline RMSE followed by RMSE values after permuting each parameter.The table covers each data sequence and LiDAR in the considered datasets.Source: Table 7 · p. 10

4.1. Cartographer parameter analysis

Cartographer recommendations prioritize LiDAR_max_range and Low_resolution for manual tuning, while giving starting values for other parameters. Across the analyzed scenarios, the recommended procedure approaches grid-search performance by manually tuning those two parameters.

4.1.1. Permutation-based feature importance data analysis

  • LiDAR_max_range is influential in 58% (35/60) of cases and most influential in 26; Low_resolution is influential in 50% (30/60) and most influential in 8.Voxel_filter_size and High_resolution are classified as low-influence and are recommended at their higher considered values rather than being tuned.Source: §4.1.1 · p. 11

4.1.2. Correlation data analysis - need for manual tuning

  • Sign-changing or near-zero correlations despite high permutation importance make LiDAR_max_range and Low_resolution the main parameters requiring manual tuning across many cases.Near-zero correlations are defined as −0.2 < r < 0.2; they indicate ambiguous influence on ATE.Source: §4.1.2 · p. 11, Table 8 · p. 12

4.1.3. Correlation data analysis - overarching trends

  • For influential cases, higher Rotation_weight and Submap_size generally correspond to lower ATE, while lower Translation_weight is recommended; LiDAR_max_range trends vary by sensor and setting.The authors recommend reducing LiDAR_max_range in confined spaces with 16-beam or low-vertical-field-of-view sensors, but larger values for the Ouster OS0-128 in the described PASTEL cases.Source: §4.1.3 · p. 11

4.1.4. Cartographer tuning recommendations

  • The recommendations rank LiDAR_max_range and Low_resolution first for manual tuning, followed by Translation_weight, Rotation_weight, and Submap_size, while assigning starting values to the parameters.The authors expect focusing on the two most influential parameters often to provide satisfactory results in similar scenarios.Source: §4.1.4 · pp. 11–12, Table 9 · p. 13

4.1.5. Validation of Cartographer tuning recommendations

  • Validation applies recommended parameter values in reverse tuning-importance order, then manually tunes LiDAR_max_range and Low_resolution for each sequence-LiDAR combination.The final ATE distribution’s minimum, median, and quartiles are less than 5 cm higher than those from optimal grid-search tuning; the maxima differ by approximately 12 cm.Source: §4.1.5 · pp. 12–13
  • 55/60 sequence-LiDAR combinations achieve ATE within 5 cm of grid-search optimal values after following the recommendations and manually tuning LiDAR_max_range and Low_resolution.The remaining exception is NTU VIRAL sequence 11, with an ATE of 48 cm; that sequence also has the largest grid-search minimum ATE, 36.5 cm.Source: §4.1.5 · pp. 13–14, Figure 4 · p. 13, Table 10 · p. 14

4.2. FAST-LIO2 parameter analysis

FAST-LIO2’s recommendations prioritize parameters for manual tuning and provide a simplified tuning procedure. Validation finds the recommended settings close to grid-search optima across the analyzed sequence-LiDAR combinations.

4.2.1. Permutation-based feature importance data analysis

  • Gyroscope_covariance has negligible influence across the analyzed dataset-LiDAR combinations and can remain at its default value in scenarios similar to NTU VIRAL and PASTEL.All other analyzed parameters are influential in more than half of the cases.Source: §4.2.1 · p. 14, Table 11 · p. 15

4.2.2. Correlation data analysis - need for manual tuning

  • LiDAR_max_range, Point_filter_number, and Filter_size_surfaces have near-zero correlations in 19/60 influential cases each, indicating the greatest need for manual tuning.Acceleration_covariance and Filter_size_map each have only five weak-correlation cases, indicating lower need for manual tuning.Source: §4.2.2 · p. 14, Table 12 · p. 16

4.2.3. Correlation data analysis - overarching trends

  • Larger LiDAR_max_range is generally preferred, while lower Filter_size_map is preferred in most influential cases; higher Acceleration_covariance is recommended for the noisier PASTEL Ouster OS1-16 IMU data.LiDAR_max_range correlations are negative in 36/38 influential cases, and Filter_size_map correlations are positive in 47/50 influential cases; the default map-filter value is recommended initially to limit computational burden.Source: §4.2.3 · p. 14

4.2.4. FAST-LIO2 tuning recommendations

  • The recommendations arrange parameters by decreasing need for manual tuning and advise starting with recommended values, then tuning from the top if performance is unsatisfactory.The proposed simplified process focuses on the two most influential parameters.Source: §4.2.4 · p. 15, Table 13 · p. 16

4.2.5. Validation of FAST-LIO2 tuning recommendations

  • Validation applies recommended settings in reverse importance order and compares ATE distributions across the sequence-LiDAR combinations with optimal grid-search results.Gyroscope_covariance has negligible influence in Step 1, while influence increases in Steps 2–4.Source: §4.2.5 · p. 15
  • 58/60 sequence-LiDAR combinations achieve ATE within 5 cm of grid-search optimal values after recommended tuning; the worst ATE is 23.5 cm in NTU VIRAL sequence 8.The Step 5 distribution’s extrema, quartiles, and median are within 5 cm of their respective optimal grid-search points.Source: §4.2.5 · p. 15

5. Conclusion and future work

The paper analyzes Cartographer and FAST-LIO2 parameter effects on trajectory accuracy in low-to-medium-altitude quadrotor flights using correlation and random-forest permutation importance. Its recommendations achieve ATE within 5 cm of grid-search optima in 94% of analyzed cases, while the study leaves interactions and higher-altitude flights for future work.

5. Conclusion and future work

  • The study assesses parameter influence on trajectory accuracy using Pearson’s correlation and random-forest permutation importance across two public datasets, three LiDAR models, and varied flights.It provides recommended starting values and a proposed tuning order.Source: §5 · p. 16
  • 94% of analyzed cases achieve ATE values within 5 cm of grid-search optimal performance using the recommendations for both algorithms.The proposed simplified procedure focuses on the two most influential parameters of each algorithm.Source: §5 · pp. 16–17
  • The analysis is limited to low-to-moderate-altitude flights and does not consider parameter interactions; high-altitude scenarios and interaction analysis remain future work.The authors also identify linking aerial LiDAR data characteristics to parameter choices as a future direction.Source: §5 · p. 17

CRediT authorship contribution statement

The statement credits Robert Milijas with data curation, methodology, validation, and writing. Jose Ramiro Martinez-de Dios and Stjepan Bogdan are credited with conceptualization, supervision, and funding acquisition, with different writing contributions.

Loading 2609.12837v1…