PaperlayerView exact revision on arXiv

Source-linked AI summary

R4Tun: LLM-guided adaptive segmental tunnel lining segmentation in point clouds

Xinghui Tao, Zehao Ye, Guangming Wang, Jelena Ninić, Brian Sheil

arXiv:2609.11360v1cs.CV

TL;DR

Tunnel-lining point-cloud segmentation must adapt to changing tunnel conditions, yet expert-tuned pipelines are parameter-sensitive. R4Tun adds LLM-guided bounded parameter adaptation using structured context to SAM4Tun, improving segmentation across tested tunnels while remaining a controlled, auditable mechanism rather than a deployable final-inspection system.

  • Problem

    Tunnel point-cloud segmentation must handle varied geometry and acquisition conditions, but expert-designed pipelines can degrade when reference-tuned parameters become mismatched.

  • Method

    R4Tun extends SAM4Tun with multi-agent LLM-guided bounded parameter adaptation using memory, intermediate state, and tunnel-specific knowledge without modifying the underlying algorithms.

  • Results

    The full m+s+k design raised mIoU from 0.18 to 0.43–0.48 and OA from 0.42 to 0.59–0.65, while near-reference regular subsets reached mIoU 0.784–0.796 across LLMs.

  • Takeaways & Limitations

    R4Tun provides controlled, label-free, cross-LLM adaptation with bounded changes and logged rationales in the tested SAM4Tun–Seg2Tunnel setting.

  • Takeaways & Limitations

    The study is limited by SAM4Tun’s fixed operators and a single expert reference, which bound adaptation when no contextual anchor resembles the target tunnel.

Abstract

from arXiv · show

Automated inspection of segmental tunnel linings requires adaptive segmentation from 3D point clouds, yet expert-tuned pipelines often degrade when tunnel conditions vary. This paper presents R4Tun, a large language model (LLM)-driven adaptation framework that extends an expert-designed pipeline (SAM4Tun) with bounded parameter tuning informed by structured context: memory ($m$), state ($s$), and knowledge ($k$). Evaluated on 30 selected Seg2Tunnel subsets (13 regular, 17 complex) across three LLMs, the full $m+s+k$ design raised mean Intersection-over-Union (mIoU) from 0.18 to 0.43--0.48 and overall accuracy (OA) from 0.42 to 0.59--0.65 relative to the static SAM4Tun baseline, with the near-reference regular (staggered) subsets reaching mIoU 0.784--0.796 across LLMs. Across 270 (30 tunnels $\times$ 3 different LLMs $\times$ 3 context settings) runs, the LLMs showed similar parameter-adjustment trends (with overlapping 95\% CIs on mean gains) and consistently adjusted a shared set of critical parameters. These results support R4Tun as a controlled, label-free, cross-LLM adaptation mechanism in the tested SAM4Tun--Seg2Tunnel setting, demonstrating consistent accuracy gains; we position R4Tun as a mechanism contribution rather than a deployable final-inspection system, in which each bounded parameter change is auditable via logged rationales.

1. Introduction

Tunnel-lining point-cloud segmentation must handle varying geometry and scanning conditions, but expert-designed pipelines can degrade when their parameters no longer match the tunnel. R4Tun addresses this by adding LLM-guided, bounded parameter adaptation with structured context while retaining the fixed SAM4Tun pipeline.

  • Motivation and contribution: Tunnel geometry, lining properties, acquisition settings, occlusions, and noise vary across projects, making direct extraction of lining components challenging.These differences motivate segmentation methods that can adapt beyond a single reference configuration.
  • Motivation and contribution: R4Tun extends the fixed SAM4Tun pipeline with LLM-guided, bounded parameter updates based on memory, intermediate state, and tunnel-specific knowledge.The framework is designed as a controlled adaptation mechanism, with logged rationales for review.
  • Related approaches: Prior approaches trade off auditability, adaptability, and data requirements: geometric rules are explicit, whereas supervised deep models require labels and retraining for new conditions.The paper positions LLM-guided adaptation as a middle path within expert-designed pipelines rather than as a replacement for them.
  • Motivation and contribution: SAM4Tun achieves segmentation without training data but remains sensitive to stage-specific parameters encoding assumptions about the reference tunnel.When those assumptions fail, performance can degrade without identifying which parameters failed or why.
  • Motivation and contribution: R4Tun introduces LLM reasoning to retune parameter-sensitive tunnel pipelines per tunnel while preserving engineer inspection and override through logged rationales.The paper also tests whether LLM-based adaptation adds gains beyond hand-coded rules using the same knowledge text.

3. Methodology

R4Tun adds an LLM-guided adaptation layer to the fixed SAM4Tun tunnel-lining segmentation pipeline, using structured context and bounded per-tunnel parameter updates. The pipeline unfolds, denoises, enhances, detects boundaries, and segments TLS point clouds while preserving the underlying algorithms.

  • Task definition: The task assigns each TLS point to background, key, adjacent, or standard segment classes, with an additional A4 class for seven-segment complex tunnels.The unit of analysis is a multi-ring tunnel subset from the Seg2Tunnel benchmark.
  • R4Tun adaptation: R4Tun adapts SAM4Tun parameters per tunnel using structured context, bounded updates, and logged LLM reasoning without modifying the underlying algorithms.The context combines memory, cumulative state, and shared knowledge; updates are clipped to admissible bounds and cross-parameter constraints.
  • Pipeline: The four-stage pipeline unfolds the tunnel, denoises artefacts, enhances geometric continuity, and segments the resulting representation using boundary prompts and SAM.Stage 4 reprojects SAM’s 2D masks into the 3D point cloud.
  • Stage 2: Denoising: Denoising retains dense tunnel surfaces within radial masks while rejecting isolated artefacts, preserving joint boundaries and ring edges for later processing.The cleaned surface supports curvature analysis and interpolation in the enhancement stage.
  • Stage 3: Enhancing: Enhancement inserts points according to local curvature and progressively upsamples the surface while preserving high-curvature joints and ring edges.Pixel-level interpolation then refines joint locations before projection into a panoramic depth map.
  • Stage 4: Segmenting: The segmenting stage detects ring-joint lines with Hough transforms, converts them into template prompts for key, adjacent, and standard segments, and sends them to SAM.SAM produces panoramic-image masks that are reprojected into 3D.
  • Experimental design: The evaluation compares fixed expert tuning with cumulative context ablations and a deterministic non-LLM adaptive control across regular and complex tunnels.The fixed SAM4Tun configuration applies one reference setting uniformly to all 30 tunnels, whereas R4Tun adapts parameters from tunnel characteristics and intermediate states.

4. Results

R4Tun improves SAM4Tun segmentation through bounded, context-informed adaptation, with gains varying by tunnel category and context component. The results also identify runtime costs, consistently adjusted parameters, and structural limits that tunnel-level tuning cannot remove.

  • Overall performance: Overall mIoU rises from 0.18 to 0.43–0.48 and OA from 0.42 to 0.59–0.65 under full m+s+k adaptation across LLMs.The improvement holds across regular and complex tunnel categories.
  • Overall performance: Regular-tunnel mIoU improves from 0.27 to 0.68–0.71, while near-reference staggered subsets reach 0.784–0.796 across LLMs.Continuous regular tunnels remain lower at 0.270–0.309, and complex tunnels improve from 0.04 to 0.15–0.20.
  • Ablation analysis: State provides the largest ablation increment, with m+s producing the strongest gains across all three LLMs and lifting mIoU 0.16–0.25 above rule-based adaptation.State supplies quantitative intermediate feedback from pipeline stages, whereas memory alone provides only a reference.
  • Ablation analysis: Knowledge adds up to 0.05 mIoU on top of m+s, mainly for complex tunnels where category-specific guidance supplements numerical state information.The mean increments are small, but their per-tunnel direction is positive.
  • Runtime and error analysis: Runtime increases to roughly 331–542 seconds per tunnel, while residual errors expose structural limits from one configuration per tunnel and fixed segment-offset labeling.Complex-tunnel adaptation reduces false negatives but can increase adjacent-class swaps because the hard-coded template cannot represent varying ring geometry.
  • Parameter analysis: The adaptation consistently adjusts 18 parameters across 270 runs, including 11 tunnel-responsive parameters and 7 shared baseline corrections.The tunnel-responsive parameters vary with geometry, while baseline corrections remain nearly identical across tunnels.

5. Discussion

R4Tun supports a mechanism-level claim: structured context improves bounded parameter adaptation across three LLMs, but the single-reference SAM4Tun setting limits absolute accuracy and deployment scope.

  • Practical scope: R4Tun is label-free and auditable, logging bounded parameter rationales for post-hoc expert review rather than replacing the underlying pipeline.The framework is positioned for assisted analysis, not autonomous inspection.
  • Key findings: State provided the main adaptation gains, adding 0.16–0.25 mIoU over memory alone across models and tunnel categories.The discussion attributes the strongest ablation contribution to intermediate pipeline state.
  • Key findings: mIoU rose from 0.18 to 0.43–0.48 and OA from 0.42 to 0.59–0.65 with full m+s+k versus static SAM4Tun.The directly comparable evaluation used no labels, training, or per-case tuning.
  • Interpretation: Near-reference regular-staggered tunnels reached mIoU 0.784–0.796 across LLMs, while complex cases remained closer to similarly low performance.The advantage is concentrated near the single calibrated reference; far-from-reference cases lack a matching anchor.
  • Limitations: Transferability is supported only conceptually beyond SAM4Tun and Seg2Tunnel, under assumptions including bounded parameters, comparable expert references, compact state, and encoded stage knowledge.The supplied evidence does not establish empirical transfer to other backbones or datasets.

6. Conclusions

R4Tun extends fixed SAM4Tun operators with LLM-guided, bounded parameter adaptation and logged rationales, improving segmentation consistently across tested models while remaining constrained by a single reference and fixed template.

  • Main findings: The full m+s+k design raised mIoU from 0.18 to 0.43–0.48 and OA from 0.42 to 0.59–0.65 across 30 tunnels and three LLMs.Near-reference regular-staggered subsets reached mIoU 0.784–0.796.
  • Main findings: State increased mIoU by 0.16–0.25 over memory alone, whereas memory or knowledge in isolation produced only small increments.This identifies intermediate pipeline state as the dominant context contributor.
  • Main findings: The three LLMs showed overlapping effect ranges, shared 18 critical parameter adjustments, and left 90.9% of critical parameters unchanged across repeats.These results support consistent adaptation behavior across models.
  • Practical implications: R4Tun operates without labelled retraining, logs a rationale for each parameter change, and adds 96–307 seconds per tunnel.The workflow shifts effort toward upfront reference calibration and stage-specific knowledge authoring.
  • Limitations: Single-reference anchoring leaves regular-continuous and complex subsets at mIoU 0.15–0.31, while fixed offset-and-order labelling causes persistent class swaps.The evidence is confined to the SAM4Tun–Seg2Tunnel setting, and transferability elsewhere remains unvalidated.
  • Future work: Multiple expert anchors, dynamic ring-level labelling, and evaluation on other parameter-controllable backbones are proposed as future directions.These directions target the single-anchor gap, structural ceiling, and untested transferability.

Declaration of generative AI use

The authors evaluated three LLMs as experimental systems and also used LLMs for language editing and manuscript-structure improvement.

  • Declaration of generative AI use: Opus-4.6, GPT-5.4, and Gemini-3-Flash were evaluated as the main experimental systems.The LLMs were also used for language editing and improving manuscript structure.

1. Baseline parameter tables

Tables 12–15 provide the SAM4Tun baseline parameter values used as the reference configuration for all adaptation experiments.

  • Baseline parameter tables: Tables 12–15 report the SAM4Tun baseline parameter values used as the reference configuration for all adaptation experiments.

2. Characteriser fields

Characteriser fields describe each tunnel and populate the per-stage state supplied to the agents.

  • Characteriser fields describe each tunnel and populate the per-stage state provided to the agents.

3. Non-LLM rule-based pseudocode

The non-LLM baseline deterministically selects stage parameters from tunnel characteristics and reference settings, applying rule-based conditions for denoising while other stages follow the same pattern.

  • The non-LLM baseline reproduces stage-specific parameter selection through deterministic Python lookups using the same knowledge documents as the LLM agents.The denoising logic is shown explicitly, while the other stages follow the same pattern.
  • Denoising selection reads diameter, density, and unfolded radial percentiles before applying conditional parameter rules.
  • For diameters above 6.5, the rule assigns the continuous family and sets mask_r_high to max(p99, 2.85) with smooth_win = 6.
  • For the large family, the rule sets mask_r_low to p10, mask_r_high to p99 + 0.05, and smooth_win to 5.

4. Context components: denoising agent example

The denoising agent receives memory, state, and stage-specific knowledge that connect tunnel differences to reference parameters and bounded tuning constraints.

  • Cross-stage template: The same four-block context template extends across unfolding, enhancing, and segmenting, with stage-specific knowledge documents.
  • Memory: Memory pairs reference and target tunnel characteristics and includes SAM4Tun reference parameters as a known-good baseline.
  • State: State supplies denoising with unfolded characteristics from the preceding unfolding stage as a reference/target pair.
  • Knowledge: Knowledge documents define each parameter’s validated range, default, and inter-parameter constraints for the denoising stage.
  • Cross-tunnel variation: Tunnel conditions vary in scale, geometry, segment layout, joint assembly, and scanning configuration, motivating structured cross-tunnel context.

5. Worked CoT trace

A worked denoising trace shows the agent comparing target and reference characteristics, diagnosing parameter mismatches, validating bounded adaptations, and emitting stage-ready JSON.

  • The denoising agent emits one schema-conformant JSON object that the stage consumes unchanged after the five-step protocol.
  • The trace begins by comparing target and reference diameter, radial percentiles, and nearest-neighbour distance to identify scale and density differences.
  • The trace adapts denoising parameters for a large, sparse target by widening radial gates, raising the cutoff, and relaxing z_step within empirical bounds.The selected values are mask_r_low = 2.37, mask_r_high = 3.80, default_cutoff_z = 3.7, and z_step = 0.005.
  • The agent validates parameter ordering, empirical ranges, diameter consistency, key presence, and type conformity before output.
  • GPT-5.4 and Gemini 3 Flash adjust the same parameter keys in qualitatively the same direction for this tunnel.

6. Runtime, API calls, and cost

The section reports compute, API-cost, repeatability, and performance-distribution analyses for R4Tun across 30 tunnels and three LLMs. Additional tables examine per-class IoU separately for regular and complex tunnels.

  • Runtime, API calls, and cost: Table 19 reports per-tunnel input/output token counts and indicative USD API costs for one full m+s+k run, averaged over 30 tunnels and four stage calls per tunnel.The reported token counts follow vendor APIs, while USD figures use vendor list prices at submission and are indicative.
  • Repeatability: Table 20 compares adapted parameters and mIoU between two reinferences of each tunnel under the full m+s+k condition at temperature 0.The repeatability metrics cover 30 tunnels, two runs per tunnel, and each LLM.
  • Per-class performance: For regular tunnels, rules achieve per-class IoU comparable to SAM4Tun, while all LLM conditions improve roughly uniformly; complex-tunnel rules recover most segment classes from near-zero but not K-block.The complex-tunnel rules baseline includes 17 tunnels, including three failed tunnels scored as zero; Opus 4.6 is representative of the reported LLM pattern.
  • Performance distribution: Table 23 summarizes the distribution of mIoU across tunnels using means across the three LLMs for each condition.The table is paired with the reported regular- and complex-tunnel performance analyses.
Loading 2609.11360v1…