PaperlayerView exact revision on arXiv

Source-linked AI summary

Rust's Type Checker Implementation Is Unsound: An Empirical Study on Soundness Bugs in rustc

Yusung Sim, Sukyoung Ryu, Jaemin Hong

arXiv:2608.28713v1cs.SEcs.PL

TL;DR

Rust’s type checker is intended to prevent memory bugs, yet rustc contains soundness bugs that accept programs which should be rejected. The paper studies 30 reported issues, their lifecycles and triggering features, and potential testing oracles and semantic documentation. It finds recurring memory-safety risks and difficult type-system interactions, while current tools and documentation remain incomplete for comprehensive soundness testing.

  • Problem

    Soundness bugs in rustc have not been studied in depth, and existing testing tools lack an oracle for deciding whether programs should be accepted or rejected.

  • Method

    The authors empirically analyze 30 rustc soundness issues and assess AddressSanitizer, Miri, Chalk, a-mir-formality, and Rust semantic documents as potential testing oracles.

  • Results

    The study finds that implied bounds and trait objects are especially relevant to memory bugs, while associated types and lifetime–trait interactions challenge sound type checking; AddressSanitizer and Miri detect memory manifestations, but Chalk and a-mir-formality are immature.

  • Takeaways & Limitations

    Comprehensive type-soundness testing for rustc still requires better oracles and more precise semantic specifications than current tools and documentation provide.

  • Takeaways & Limitations

    The dataset may omit additional soundness issues because conservative filtering excluded some reports, particularly issues involving feature-gated features.

Abstract

from arXiv · show

Rust is claimed to be a type-sound language capable of preventing various undesirable behaviors, including memory bugs. However, rustc, the official Rust compiler, is not immune to defects; it contains soundness bugs, where the compiler accepts programs that should be rejected during type checking. In this work, we present an empirical study of 30 issues that report potential soundness bugs in rustc, collected from the GitHub issue tracker between January 1, 2022 and September 1, 2025. We analyze each issue in depth, focusing on its affected feature, symptom (how the feature is mishandled), consequence (the resulting undesirable behavior), triggering features, community consensus regarding whether it is a bug, and lifecycle, including introduction, discovery, and fix. Furthermore, we investigate existing artifacts, including implementations such as AddressSanitizer, Miri, Chalk, and a-mir-formality, alongside documentation such as the Rust Reference, the FLS, and Rust RFCs to assess their potential as oracles for testing the type soundness of rustc. Our key findings indicate that: (1) Certain soundness bugs, typically triggered by implied bounds or trait objects, compromise memory safety. (2) Sound type checking is challenged by edge cases involving associated types and the interaction between lifetimes and traits. (3) Most bugs persist from the initial introduction of the relevant features and require significant time to be discovered. (4) While AddressSanitizer and Miri can detect soundness bugs that lead to memory bugs, a-mir-formality and Chalk are currently immature despite their potential to identify other bug categories. (5) Existing documentation frequently fails to provide precise explanations of the language semantics.

1 Introduction

Rust relies on a sound type checker to prevent undesirable behaviors, but soundness bugs in rustc can undermine that guarantee. This study analyzes 30 reported issues and evaluates potential testing oracles and semantic documentation.

  • Motivation: Rust adopts type checking to prevent undesirable behaviors, including memory bugs, and its ecosystem relies on the type checker being sound.Rust systems software and legacy-system integration depend on this assumption.
  • Research gap: Soundness bugs accept programs that should be rejected, threatening Rust’s core type-soundness guarantee.Among 301 previously collected rustc bugs, 22 were identified as soundness bugs.
  • Approach: The study examines 30 rustc soundness issues across affected features, symptoms, consequences, triggers, community consensus, and lifecycles.Lifecycle analysis covers issue introduction, discovery, and fix.
  • Approach: The authors evaluate AddressSanitizer, Miri, Chalk, and a-mir-formality as potential oracles for deciding whether issue programs should be rejected.The artifacts span memory-bug detection, interpretation, trait-system modeling, and executable type-system modeling.
  • Approach: They also examine the Rust Reference, FLS, and RFCs because Rust lacks a full specification for its language semantics.The study assesses whether these documents can support testing type soundness.
  • Findings: The findings identify memory-safety risks from implied bounds and trait objects, difficult lifetime–trait and associated-type cases, persistent bugs, immature formal artifacts, and unclear documentation.AddressSanitizer and Miri can detect bugs manifesting as memory bugs, whereas Chalk and a-mir-formality remain immature for other categories.

2 Background: Language Features of Rust

This section introduces Rust features that recur in soundness issues, including lifetimes, traits, associated types, opaque and dynamic trait types, and higher-rank lifetime polymorphism. These features express validity, abstraction, dispatch, and relationships among types and lifetimes.

  • Lifetimes: References carry lifetimes describing how long they remain valid, and Rust rejects uses that outlive the referenced value.A reference has type &'a T, where 'a is its lifetime.
  • Lifetimes: Lifetime bounds such as 'a: 'b express that one lifetime outlives another, while implied bounds can be introduced automatically from type well-formedness.For example, &'a &'b i32 requires 'b: 'a, and foo is equivalent to a version with that bound stated explicitly.
  • Traits: Traits define shared behavior, and generic functions can require trait implementations through type parameters, where clauses, or impl trait parameters.These forms allow functions to call methods guaranteed by a trait bound.
  • Traits: Associated types let each trait implementation specify a type used by trait methods, with syntax such as T::A or <T as X>::A.Different implementations of X can assign different types to A, while bounds can constrain A to a specific type.
  • Opaque and dynamic types: Return-position impl trait hides a function’s single concrete return type while exposing only its trait interface; it does not permit multiple underlying return types.Trait objects instead support values of different implementing types through dynamic dispatch and are typically placed behind pointers.
  • Polymorphism: Higher-rank lifetime polymorphism allows a function pointer to accept references with different lifetimes, unlike rank-1 polymorphism.A higher-rank function pointer can be called with references created in separate scopes.
  • Polymorphism: Higher-rank lifetime bounds can also require that references to a type implement a trait for every lifetime.The example uses for<'a> &'a T: X to call a trait method on references created in multiple scopes.

3 Collecting Soundness Issues

The authors build a 30-issue dataset by filtering rustc’s issue tracker and manually validating candidate soundness reports. They then remove duplicates, compare the dataset with prior work, and classify the retained issues by features, consequences, and status.

  • Collection scope: The study covers rustc issues reported from January 1, 2022, through September 1, 2025, spanning the Rust 2021 and 2024 editions.The period covers more than three and a half years.
  • Filtering: The authors first filter GitHub issues using type-related area labels, then retain reports marked C-bug or I-unsound and exclude labels unrelated to soundness.Excluded categories include documentation and tool issues, diagnostics issues, and alternative symptoms such as crashes or hangs.
  • Manual validation: 320 issues remained after automatic filtering, and manual inspection identified 27 soundness issues using code-rejection claims and compiler-caused buggy behavior.Issues caused by external changes, such as standard-library breaking changes, were excluded.
  • Deduplication: Duplicate checking removed four within-dataset duplicates while retaining two duplicates of pre-2022 issues, producing the final 30-issue dataset.The retained older duplicates did not create redundancy within the study’s timeframe.
  • Dataset comparison: Only 4 issues overlap with Liu et al.’s dataset, while 19 are unique to this study and 18 are unique to the prior study.The comparison attributes differences to timeframe, categorization, duplicate treatment, and filtering decisions.

4 Soundness Issues in rustc

The study identifies five affected features and fourteen triggering features among rustc soundness issues, with lifetime-checking failures frequently enabling memory bugs. These failures arise across implied bounds, associated types, higher-rank traits, arrays, closures, async functions, and related contexts.

  • Missing Lifetime Checks: 14 issues involve improper lifetime checks, and 8 of them actually break memory safety.The remaining six produce inconsistent type-checking results without reference misuse.
  • Implied Bounds: Implied-bound failures can let a short-lived reference be treated as 'static, enabling memory bugs when the reference is later dereferenced.The compiler accepts a function-pointer assignment despite an implied 'a: 'static requirement; explicitly adding the bound makes it reject the assignment.
  • Associated Types and Traits: Higher-rank trait bounds can make an associated output type usable with incompatible lifetimes, allowing a non-static reference where 'static is expected.The issue arises because the compiler permits F::Output to denote references with both 'static and arbitrary lifetimes.
  • Array Lengths: A zero-length array bypasses a lifetime check that rejects the otherwise identical length-1 array, although the accepted function returns no reference.The behavior is inconsistent with developers’ expectation that array type checking should not depend on length.
  • Other Contexts: Other lifetime-checking gaps occur with associated constants or types, nested impl traits, trait bounds, struct construction in closures, and async functions.These cases include issues where missing checks do not lead to memory bugs, as well as the anonymous-lifetime case in async functions.

4.2 Traits

Trait-related soundness issues include overlapping implementations, failed dyn-compatibility checks, orphan implementations, and incorrect trait solving. Associated types and trait-object interactions can turn these inconsistencies into memory bugs.

  • Overlapping Implementations: 4 issues allow overlapping implementations; 2 cause memory bugs, while 2 cause only inconsistency.Overlapping implementations create ambiguity in method resolution and are normally rejected.
  • Overlapping Implementations: Trait-object overlaps can assign different associated types to the same object, allowing a value with one lifetime or type to be treated as another.One issue lets a reference with lifetime 'a be returned as &'static i32; another similarly causes memory bugs.
  • Trait Objects: A missing dyn-compatibility check can make one return type be interpreted as two different types, leading to a memory bug.The issue involves an associated type of a supertrait referenced through Self.
  • Orphan Implementations: An associated-type expression can disguise an orphan implementation, causing rustc to accept an impl that should be rejected.Writing <T as Y>::A, equivalent to T in the example, bypasses the orphan check.
  • Incorrect Trait Solving: Trait solving can accept cyclic requirements involving associated-type bounds, even though a directly applied bound causes rejection.The compiler concludes that S implements Unpin despite a cycle between S: X<A = S> and S: Unpin.

4.3 Type Inference

Type-inference soundness issues arise when rustc selects a type or hidden RPIT type despite multiple valid choices. These cases are treated as inconsistencies and may produce unexpected runtime behavior.

  • Type Arguments: 2 issues arbitrarily choose a type argument when multiple types are possible, although inference should succeed only when the type is unique.The contrast is between foo(0), where i32 is uniquely determined, and baz(), where any type is valid.
  • Type Arguments: Arbitrary type selection can make runtime behavior deviate from developer expectations.The paper classifies such selection as inconsistent because no unique valid type determines the result.
  • Type Arguments: The try propagation operator can trigger arbitrary type inference when a function’s return value is otherwise unused.baz is accepted with ?, whereas the analogous bar call is rejected because its type argument is unconstrained.
  • Type Arguments: A function with a lifetime parameter can receive an arbitrary type argument when coerced to a function pointer.This is another reported case of inference choosing among multiple possible types.
  • RPIT Hidden Types: 4 issues arbitrarily choose an RPIT hidden type, including cases involving recursive calls, PhantomData, and associated types.Such issues are considered inconsistent and may lead to unexpected runtime behavior.

4.4 Type Well-formedness

Rustc can accept an RPIT containing a type that is not well-formed, even though the same type is rejected in another context. This inconsistency does not cause a memory bug because the invalid value cannot be constructed.

  • Type Well-formedness: Rustc accepts an RPIT using S<i32> even though S<i32> is not well-formed because no type implements Y.The same type is rejected when used in an argument-position impl trait.
  • Type Well-formedness: The inconsistency does not cause a memory bug because a value of S<i32> cannot be constructed.

4.5 Type Cast

Rust accepts casting the constructor of a data-carrying enum variant to an integer, treating that constructor as a function and yielding its address.

  • Type Cast: Casting a data-carrying enum constructor such as F::B to an integer is accepted, unlike casting the variant F::B(0).The Rust Reference intentionally treats F::B as a function, so the cast produces its address.

5 Findings from Soundness Issues

The study finds that rustc soundness issues arise across lifetimes, traits, inference, and associated features, with some compromising memory safety. Bugs often persist undetected for years, while feature interactions and incomplete validation complicate prevention and diagnosis.

  • Symptoms and Consequences: Lifetimes affect 15 issues, traits 7, and type inference 6; missing lifetime checks are the most common symptom with 14 issues.The issues mainly lead to inconsistencies in 17 cases and memory bugs in 11 cases.
  • Symptoms and Consequences: 11 issues lead to memory bugs, including 8 caused by missing lifetime checks, 2 by overlapping impls, and 1 by missing dyn-compatibility checks.The findings connect lifetime-checking failures and type confusion from trait-related mechanisms to memory-safety violations.
  • Triggering Features: Associated types trigger 15 issues, followed by implied bounds and RPITs with 7 each, and higher-rank polymorphism with 5.Associated types are often used to exploit edge cases in other features rather than causing soundness failures alone.
  • Triggering Features: Most lifetime issues require trait-related triggers, including associated types in 6 cases, showing that interactions between lifetimes and traits challenge sound type checking.The authors encourage studying these features together rather than separately.
  • Triggering Features: All 7 issues involving implied bounds and all 6 involving trait objects lead to memory bugs, making their treatment critical for memory safety.Implied bounds require validation at both definition and use sites, while trait objects complicate overlapping-impl detection through automatically introduced built-in impls.
  • Issue Lifecycles: Soundness issues took an average of 1,291 days to discover, with a maximum of 3,552 days, and often persisted across multiple compiler releases.The study attributes delayed discovery to rare combinations of language features and silent successful compilation.

6 Potential Oracles for Testing Type Soundness

The paper evaluates implementations and documentation as potential oracles for testing rustc’s type soundness. AddressSanitizer and Miri detect memory manifestations, while Chalk and a-mir-formality offer broader but currently limited formal checking potential.

  • Implementations: AddressSanitizer and Miri can serve as oracles for soundness bugs that manifest as memory bugs, but not for bugs without undefined behavior.They check runtime memory safety rather than type-checking decisions directly.
  • Implementations: Both AddressSanitizer and Miri detected memory bugs in all eleven evaluated soundness issues that produce memory bugs.The authors therefore describe them as promising while noting that complementary tools are needed.
  • Implementations: Chalk provides a logic-based trait solver that can act as an oracle for trait-related soundness bugs, but other type-system features remain outside its scope.Its latest evaluated version correctly identified some orphan and dyn-compatibility violations, while lacking support for at least one issue involving associated-type bounds.
  • Implementations: a-mir-formality models the Rust type system broadly and has potential as a comprehensive oracle, but its syntax and feature coverage require further development.The study manually rewrote issue code into its syntax and recommends prioritizing implied bounds, impl traits, and trait objects.
  • Documentation: Existing Rust documentation contains semantic holes that must be resolved before constructing a mechanized specification from it.The Reference, FLS, and RFCs leave unclear areas including well-formedness, implied bounds, higher-rank coercions, and interactions between HRTBs and associated types.
  • Documentation: The documentation does not explain how type paths involving HRTBs and associated types should be resolved.The Reference and FLS describe these concepts separately or incompletely rather than specifying their interaction.

7 Threats to Validity

The study’s validity is constrained mainly by issue selection and dataset size, while researcher bias was mitigated through independent investigation, discussion, cross-referencing, and objective checks. The authors nonetheless retain confidence that the findings reflect real soundness issues.

  • External validity: The dataset may omit soundness issues because GitHub-label filtering and conservative inclusion criteria can miss relevant or poorly documented cases.The authors supplemented the dataset with seven previously excluded issues but acknowledge that other issues may remain missing.
  • External validity: Feature-gated issues were excluded, potentially omitting additional soundness bugs because unstable features may contain more compiler bugs than stable features.The authors found 10 of 223 issues excluded by an F- label also carried I-unsound.
  • Construct validity: Characterizing features, symptoms, consequences, and bug status is vulnerable to researcher bias and human error.Two authors independently investigated issues, discussed discrepancies, and used external resources and objective status checks as mitigations.
  • Conclusion validity: The small dataset limits the generalizability of the conclusions.The authors frame the study as valuable despite this conclusion-validity threat because it reports and analyzes actual soundness issues.
  • Overall assessment: The study’s findings remain useful for guiding future testing and formalization research despite limits on generalizability.The authors identify actionable insights for testing the Rust type checker and formalizing its type system.

8 Related Work

Prior work has examined compiler bugs across Rust and other ecosystems, including soundness caused by feature interactions. This study differs by focusing specifically on soundness bugs in rustc and by relating them to testing and formalization artifacts.

  • Rust compiler studies: Earlier research on rustc and rust-gcc found type checking to be among the most bug-prone compiler components.That work reported type-system errors, including trait-bound and opaque-type issues, as prominent bug categories.
  • Other compiler studies: Researchers have also studied compiler bugs in GCC, LLVM, WebAssembly, JVM-targeting languages, Solidity, and deep-learning compilers.These studies cover broader compiler-defect domains beyond the specific rustc soundness focus here.
  • Soundness studies: Soundness studies of Java and Scala showed that interactions among language features can enable unsoundness without relying on built-in downcasting.The reported examples used null pointers with existential types to enable casts between arbitrary types.
  • Rust formalization: Formalizations such as RustBelt and Borrow Calculus provide verified models of subsets of Rust that may support future mechanized specifications.These efforts formalize ownership, borrowing, linearity, or related core-language properties and prove corresponding soundness results.

9 Conclusion

The study analyzes 30 rustc soundness issues across their characteristics and lifecycles, and evaluates potential testing oracles and semantic documentation. It finds recurring roles for associated types and lifetime–trait interactions.

  • Conclusion: The study examines 30 rustc soundness issues across affected features, symptoms, consequences, triggers, community consensus, and lifecycles.It also evaluates AddressSanitizer, Miri, Chalk, a-mir-formality, and Rust semantic documents as potential testing resources.
Loading 2608.28713v1…