HIDE: Hierarchical Isolation via Documentation and Evidence for Rustc Bugs

Motivation

• Rustc bugs can block builds or undermine trust in compiled software, making responsible-file identification necessary for repair.The paper frames isolation as challenging in a large, evolving repository with strong source-file dependencies.Source: §1 · p. 1

Evaluation

• 42.08% Accuracy@1 and 80.42% Accuracy@10 exceed the strongest baseline results by 5.83% and 6.04%, respectively.The evaluation covers 160 reproducible rustc bugs and compares HIDE with Agentless, SoapFL, and GraphLocator.Source: §1 · p. 2

Approach

• HIDE combines compiler documentation with bug evidence to rank suspicious rustc source files.Its Layered Documentation Aggregation and Hierarchical Bug Isolation stages address distributed functionality, search-space size, and context overload.Source: §1 · p. 2

Layered Documentation Aggregation

• Layered Documentation Aggregation summarizes individual files and how files with similar functions work together.This documentation is intended to capture compiler functionality distributed across source files.Source: §1 · p. 2

Hierarchical Bug Isolation

• Hierarchical candidate selection reduces the files requiring detailed analysis, while Scoped Candidate Competition and Evidence-Guided Code Slicing limit prompt contents.The mechanisms limit the number of candidates and the amount of source code included, respectively.Source: §1 · p. 2

2.1 Rustc Architecture and Query System

• Rustc source is organized into crates, modules, and files, from groups of compiler functions down to concrete checks, analyses, or transformations.The example progresses from the rustc_const_eval crate to its transform module and validate.rs.Source: §2.1 · p. 3
• Query traces record computations observed during compilation, complementing source organization by providing execution evidence for bug isolation.Queries take an input key and return a result; a query implementation may request other queries, creating dependencies.Source: §2.1 · p. 3

2.2 Motivating Example

• Bug 118990 exposes distributed functionality: explaining the incorrect MIR validation report requires relating validation, storage-liveness information, and when validation runs.The relevant files are validate.rs, storage_liveness.rs, and pass_manager.rs; the failure involves cleanup of a temporary whose storage is not active.Source: §2.2 · pp. 3–4
• The bug's query trace includes routine MIR construction and borrow checking, so their presence alone does not identify the responsible files.Following their implementations and dependencies can reach many files, making exhaustive detailed inspection costly.Source: §2.2 · p. 5
• Including many candidate files and their complete source can burden analysis with unrelated code and obscure whether cleanup would occur.The motivating example calls for comparing smaller candidate groups and focusing on bug-related code.Source: §2.2 · p. 5

3 Approach

• HIDE generates file-, module-, and crate-level documentation, then uses bug context and those docs to narrow candidates from crates to modules to files.The documentation captures both individual file functionality and how files or modules work together.Source: §3 · p. 5

3.1 Layered Documentation Aggregation

• Layered Documentation Aggregation synthesizes file responsibilities and aggregates file docs into module docs, then module docs into crate docs.The resulting documentation describes compiler functionality and how files or modules work together.Source: §3.1 · pp. 6–7

3.2 Hierarchical Bug Isolation

• Hierarchical Bug Isolation combines the shared bug context with level-specific documentation and passes candidate selections and assessments through successive levels.The shared context includes the issue description, failure program, compiler log, and query trace.Source: §3.2 · pp. 7–8

3.2.1 Crate-Level Bug Isolation.

• Crate-Level Bug Isolation assesses crates using their documentation and the bug context, ranks them, and passes the top-Kc candidates to module search.The next component searches only modules within those selected crates.Source: §3.2.1 · pp. 8–9

3.2.2 Module-Level Bug Isolation.

• Module-Level Bug Isolation ranks modules using module docs, bug context, and crate assessments, then selects files separately within each selected module.Scoped Candidate Competition limits files considered together and preserves candidates from multiple modules.Source: §3.2.2 · pp. 9–10

3.2.3 File-Level Bug Isolation.

• File-Level Bug Isolation slices candidate files using weighted bug-context terms and failure-marker bonuses, then analyzes and ranks the resulting candidates.Terms from compiler logs and query traces receive greater weight than terms from issue descriptions and failure programs.Source: §3.2.3 · p. 10

• The listed query trace is mir_drops_elaborated_and_const_checked, and the compiler-log entry is delay_span_bug .../validate.rs.Source: §1 · p. 11

2. Evidence-Term Extraction

• Execution evidence receives greater weight when the evidence terms are used to rank candidates for Scoped Candidate Competition.The mechanism returns the specified number of candidates, or up to K_s, with a brief selection reason.Source: §2 · pp. 11, 13
• Top Window Selection ranks each file’s code windows by score and extracts the highest-ranked windows within a per-file code budget.Files without matching evidence terms are ranked using file documentation and other available evidence, without code slices.Source: §2 · p. 11
• File-level analysis combines file documentation and available code slices with bug context and crate- and module-level assessments.The resulting per-file assessments are compared across modules to rank files and select the top K_f.Source: §2 · p. 11
• The shared prompt template specifies isolation level, bug context, candidate documentation, available assessments and code slices, and output size.Inputs vary by level, from crate documentation to file candidates analyzed with code slices and prior assessments.Source: §2 · p. 12, Table 2 · p. 12
• Hierarchical Bug Isolation uses generated documentation and bug context to narrow candidates across crates, modules, and files.Scoped Candidate Competition and Evidence-Guided Code Slicing limit, respectively, the number of candidates and source code included in prompts.Source: §2 · p. 13

4 Experimental Setup

• The evaluation asks how HIDE compares with baselines in effectiveness and efficiency, performs across LLMs, and benefits from its components and mechanisms.These are the four research questions guiding the evaluation.Source: §4 · p. 13

4.1 Datasets

• The dataset contains 160 reproducible ICE, completeness, and soundness bugs; runtime-behavior and diagnostic bugs are excluded to keep reproduction oracles uniform.Of 195 bugs screened, 35 were excluded for missing information or inconsistent behavior; bug context includes issue description, failure program, compiler log, and query trace.Source: §4.1 · p. 13
• File-level ground truth comes from compiler source files modified by each bug’s fixing pull request, excluding tests, documentation, and other non-compiler files.A bug is successfully isolated when at least one ground-truth file appears in the returned top-K list.Source: §4.1 · p. 14

4.2 Baselines

• HIDE is compared with Agentless, SoapFL, and GraphLocator, representing semantics-based, execution-based, and dependency-based approaches.All methods use a shared dataset and protocol while preserving each baseline’s core workflow, prompts, and reported parameter settings.Source: §4.2 · p. 14

4.3 Evaluation Metrics

• Effectiveness is measured by Accuracy@k, the proportion of bugs with at least one ground-truth file among the top-k results; efficiency uses mean time and tokens per bug.Higher Accuracy@k and lower time and token counts indicate better performance on their respective metrics.Source: §4.3 · p. 15

4.4 Experimental Design

• RQ1 and RQ2 use GPT-5.4 for HIDE and baselines, while RQ3 tests HIDE with three LLMs and RQ4 compares GPT-5.4 ablations.Candidate-selection parameters were chosen on 20% of the dataset by Accuracy@10, with tokens and then time breaking ties; the selected settings were K_c=4, K_m=12, K_s=4, and K_f=10.Source: §4.4 · p. 15, §4.5 · p. 15

5.1 RQ1: Effectiveness

• 42.08% Accuracy@1 and 80.42% Accuracy@10; HIDE leads all reported Accuracy@k metrics, with gains of 5.83% and 6.04% over the strongest baselines, respectively.GraphLocator is the strongest baseline at Accuracy@1; Agentless is strongest at Accuracy@3, @5, and @10.Source: Table 5 · p. 16, §5.1 · pp. 16–17

5.2 RQ2: Efficiency

• 154 seconds per bug makes HIDE the fastest approach; it uses 145,413 tokens, fewer than Agentless and SoapFL but more than GraphLocator.HIDE uses 45,771 more tokens than GraphLocator but finishes 291 seconds sooner.Source: Figure 7 · p. 17, §5.2 · pp. 17–18

5.3 RQ3: Generalizability

• 76.04%–80.42% mean Accuracy@10 across three LLMs contrasts with larger cross-model differences at smaller k.Gemini-3.1-Pro-Preview has the highest means at Accuracy@1, @3, and @5; GPT-5.4 is highest at Accuracy@10.Source: Table 6 · p. 18, §5.3 · pp. 18–19
• With three runs, GPT-5.4 has the smallest observed accuracy variation, but the spans do not establish inherent differences in model randomness.Its Accuracy@1 span is 1.25%; the authors caution that three runs describe only observed variation under the tested settings.Source: §5.3 · pp. 18–19

5.4 RQ4: Ablation Study

• Removing Module-Level Bug Isolation reduces Accuracy@10 by 8.13%, versus 0.62% at Accuracy@1; removing both isolation components reduces Accuracy@10 by 10.00%.Removing Crate-Level Bug Isolation alone reduces Accuracy@10 by 3.76%.Source: §5.4 · p. 19
• Removing Crate-, Module-, or File-Level Documentation Generation reduces Accuracy@10 by 4.38%, 5.63%, or 13.13%, respectively.The file-level removal also causes the largest decrease at every reported metric within the documentation group.Source: §5.4 · pp. 19–20
• Removing Scoped Candidate Competition reduces Accuracy@5 and Accuracy@10 by 9.38% and 10.00%; removing Evidence-Guided Code Slicing reduces Accuracy@10 by 9.38%.The slicing removal has smaller decreases at Accuracy@3 and Accuracy@5: 0.62% and 1.88%, respectively.Source: §5.4 · pp. 19–20
• All eight ablated variants score below the full approach at every reported Accuracy@k, supporting the evaluated components and mechanisms.File-Level Documentation Generation produces the largest Accuracy@10 decrease among the variants.Source: §5.4 · pp. 19–20, Table 7 · p. 20

• Three independent runs reduce reliance on a single LLM output but cannot fully characterize variation, and ablation results may remain affected.Source: §6 · p. 20
• Baseline adaptation may affect performance even though the comparison preserves each baseline’s core localization procedure, prompt design, and reported settings.Source: §6 · p. 20
• Public issues and fixes may have appeared in undisclosed model training data, despite excluding fixing pull requests from method inputs.Source: §6 · pp. 20–21
• Accuracy@k records whether any ground-truth file appears in the top k, without requiring all responsible files or evaluating ranking quality beyond that cutoff.Source: §6 · p. 21
• The evaluation covers 160 reproducible ICE, completeness, and soundness bugs; its results may not extend to non-reproducible bugs, performance regressions, or poor diagnostics.Source: §6 · p. 21

7.1 Extension to Method-Level Bug Isolation

• 32.50% Accuracy@1 and 60.00% Accuracy@10: HIDE leads the strongest baselines by 6.25% and 3.75% on method-level rustc isolation.Source: §7.1 · p. 21
• Method-level accuracy rises by only 1.87% from Accuracy@5 to Accuracy@10, so returning more method candidates adds limited benefit in this evaluation.Source: §7.1 · p. 21

7.2 Transferability to GCC

• 50.00% Accuracy@1 and 85.50% Accuracy@10: HIDE exceeds SoapFL’s 49.50% and 80.50% on 200 sampled GCC bugs.The evaluation samples six GCC modules; the reported findings support transferability under that setting.Source: §7.2 · p. 22, Table 10 · p. 22, Table 9 · p. 22

8.1 Bug Isolation

• Semantics-, execution-, and dependency-based methods provide useful evidence, but may miss compiler-specific file roles, exclude responsible candidates early, or fail to explain cross-file functionality and query execution.Source: §8.1 · p. 23
• HIDE combines file-role documentation, hierarchical narrowing across crates, modules, and files, and prompt-level candidate and code limits to address distributed functionality, search cost, and context overload.Source: §8.1 · p. 23
• Compiler-specific methods compare executions by mutating programs, varying compiler options, or skipping steps; each strategy depends on producing an execution without the failure.Step-skipping can be difficult to apply directly to rustc because queries are requested as needed and may request other queries.Source: §8.1 · pp. 23–24
• Unlike execution-contrast approaches, HIDE uses the original bug context and compiler documentation to narrow candidates without requiring a failure-free execution.Source: §8.1 · p. 24

8.2 Rust Software Engineering

• Rust safety, maintenance, and compiler-testing studies address program safety, application repairs, or bug discovery, whereas HIDE ranks rustc files after a failure is observed.Source: §8.2 · pp. 24–25

• With GPT-5.4, HIDE achieves 42.08% Accuracy@1 and 80.42% Accuracy@10, outperforming Agentless, SoapFL, and GraphLocator at every reported Accuracy@k.The evaluation covers 160 reproducible rustc bugs; HIDE also has the lowest average execution time.Source: §9 · p. 25
• Future work will broaden the evaluated compiler bug types, extend HIDE to more compilers, and improve its method-level localization.Source: §9 · p. 25