PaperlayerView exact revision on arXiv

Source-linked AI summary

Toward Understanding Operating System Defects

Hongyao Zuo, Jiali Li, Jiajun Jiang

arXiv:2608.20643v1cs.SE

TL;DR

Operating-system defects can have severe consequences, yet prior evidence has been limited across systems and defect characteristics. The paper studies 1,500 defects from Android, Linux, and HarmonyOS using a multi-dimensional classification and cross-OS analysis, finding substantial distribution differences across systems while repair-related code-element distributions are largely similar. These findings support targeted OS debugging and quality-assurance research within the study’s data and labeling boundaries.

  • Problem

    Comprehensive evidence on OS defect characteristics across different systems, especially emerging architectures, remains limited despite their potentially severe consequences.

  • Method

    The study manually classifies 1,500 defects from Android, Linux, and HarmonyOS across five dimensions and performs joint and cross-OS distribution analyses.

  • Results

    Defect distributions across OS layers, affected functions, trigger scenarios, and severity differ largely across systems, whereas repaired code-element distributions are largely similar.

  • Takeaways & Limitations

    The findings indicate that OS quality-assurance approaches should account for system-specific defect characteristics and support targeted debugging techniques.

  • Takeaways & Limitations

    The conclusions are constrained by differing bug-report sources across systems and the subjectivity of manual defect labeling.

Abstract

from arXiv · show

Operating systems (OS) serve as the foundation for all other software systems, and thus defects in OSes can lead to severe conquences, such as system crashes and data corruption, affecting billions of users. This broad impact underscores the necessity and importance of ensuring OS quality. Understanding the characteristics of OS defects is a fundamental step in this quality assurance task, as it facilitates the design of effective defect detection and debugging approaches. In this work, we conduct a large-scale study of 1,500 defects from three distinct and representative operating systems (Android, Linux, and HarmonyOS) spanning both mobile and desktop environments. To the best of our knowledge, this is the largest study of its kind in this domain. By analyzing the distribution of OS defects across multiple classification dimensions, including the OS layer where defects occur, the functions they affect, how they are triggered, their severity, and the code elements involved in their repair, as well as performing joint analysis between dimensions and cross-OS similarity comparisons, we summarize several major findings that contribute to a comprehensive understanding of OS defects across systems. Based on these findings, we provide a series of actionable implications for better OS defect detection and debugging, offering guidelines for future research aimed at improving OS quality assurance.

I. INTRODUCTION

The study addresses limited cross-OS evidence by systematically characterizing defects across Android, Linux, and HarmonyOS. It analyzes 1,500 defects through a multi-dimensional framework to support more targeted quality assurance.

  • OS defects can cause data corruption, system crashes, and security breaches affecting billions of users, making systematic defect understanding critical.
  • Existing studies often examine one OS or one defect type, leaving comprehensive comparisons across mobile, desktop, and emerging OS families limited.
  • The study analyzes 1,500 real-world defects from Android, Linux, and HarmonyOS across mobile and desktop computing domains.
  • Its classification framework captures defect triggers, affected functions, repair-related code elements, and distributions across systems.
  • The findings yield implications for improving OS defect detection, debugging, and testing, while the released dataset supports future reliability and security research.

II. RELATED WORK

Prior OS-defect research provides important subsystem and vulnerability insights but remains fragmented by defect type, system, and maturity. This study motivates a cross-system, multi-dimensional comparison using representative public defect data.

  • Earlier research analyzed kernel defect rates, regression-prone subsystems, large bug-report collections, and mobile security patterns, but did not provide a unified cross-OS view.
  • The study identifies four gaps: limited comprehensive defect-characteristic analysis, predominantly single-system studies, insufficient attention to emerging systems, and scarce joint multi-system data.
  • Study Design: Candidate systems were selected using availability, maturity, computing-domain, and implementation-language criteria to support meaningful comparisons.
  • Data Sources: The dataset draws on Android Security Bulletins, Linux Launchpad reports, and HarmonyOS repositories and disclosures.
  • Data Preparation: Filtering removed duplicates, reports without open-source patch diffs, and changes exceeding 500 lines before selecting the latest 500 valid defects per system.

B. Classification and Labeling Process

The researchers built and refined a unified defect-classification process through pilot annotation, independent iterative labeling, consensus resolution, and reliability tracking.

  • A multi-dimensional framework was designed to capture defect characteristics consistently across all three operating systems.
  • Thirty pilot defects, sampled as 10 from each system, were used to develop and refine classification categories through independent annotation and discussion.
  • Two annotators independently labeled 50 defects per iteration, while practical experience and inter-rater reliability guided continuous framework refinement.

C. Defect Classification Framework

Each defect is classified across five dimensions: OS layer, affected function, trigger scenario, severity, and repair-related code level. The framework operationalizes layer categories by the location of the root-cause code.

  • The framework classifies each defect by OS layer, affected function, trigger scenario, severity, and code level involved in repair.
  • OS-layer assignment is based on where the root-cause code resides, typically inferred from the modified code’s directory path.
  • The six OS-layer categories are Kernel, System Library, Runtime/Virtual Machine, Framework, System Service, and Application.
  • Kernel defects concern core hardware-control functionality, while System Library defects affect foundational interfaces used by upper layers.
  • Runtime/Virtual Machine, Framework, System Service, and Application categories distinguish defects by execution infrastructure, framework logic, cross-application services, and system-included application code.

2) Affected Function:

The affected-function taxonomy organizes OS defects into eight categories spanning security, resources, processes, storage, devices, networking, compatibility, and graphics.

  • Affected-function categories: The affected-function dimension comprises eight categories covering security and permission control, graphics and rendering, memory and resource management, process and task scheduling, storage, devices, networking, and compatibility.
  • Memory and Resource Management: Memory and Resource Management covers allocation, deallocation, leaks, and CPU/GPU resource scheduling, with memory leaks and out-of-bounds access as typical manifestations.
  • Process and Task Scheduling: Process and Task Scheduling covers process lifecycle, priority scheduling, task stacks, and foreground/background transitions affecting execution order and CPU allocation.
  • Other affected functions: Storage, device, network, and compatibility categories address data access, hardware and HAL adaptation, connection and transmission behavior, and cross-platform or cross-version consistency.
  • Security and Permission Control: Security and Permission Control covers permission validation, encryption, malicious-behavior protection, and unauthorized access to sensitive data.

3) Trigger Scenario:

The trigger-scenario taxonomy classifies defects by the conditions and environments required to reproduce them, including operational, boundary, concurrency, state, environment, permission, and localization factors.

  • Trigger-scenario categories: The trigger-scenario dimension comprises seven categories based on the manners and scenarios in which defects are triggered.
  • Regular operation: Regular Operation Trigger defects occur during ordinary functionality use without special environments, permissions, or boundary values.
  • Boundary conditions: Boundary Condition Trigger defects arise under extreme inputs, resource exhaustion, or exceptional flows when source code mishandles boundary conditions.
  • Context-dependent triggers: Concurrency/Multi-thread, State Dependency, and Environment Dependency triggers require concurrent execution, system-state changes, or external factors such as networks, hardware, versions, and dependencies.
  • Localization: Multi-Language/Localization Trigger defects involve language, regional, cultural, encoding, time-zone, or formatting adaptations.

4) Severity Level:

The study defines severity using developer-assigned labels and classifies repair changes by fine-grained code elements, with code-level categories allowing multiple assignments per patch.

  • Severity level: Severity considers security risk, usability impact, urgency of fixes, and other aspects requiring substantial system-implementation expertise.
  • Severity level: Developer-provided severity labels are adopted for each operating system to support reliable classification.
  • Code-level repair elements: The code-level dimension classifies the fine-grained elements modified in patches that remove defects.
  • Code-level repair elements: Unlike other dimensions, code-level categories are not mutually exclusive because one defect can require multiple code changes.
  • Code-level repair elements: The taxonomy includes entire classes, entire functions, function signatures, if blocks, exception-handling flows, and function calls among its repair categories.
  • Counting rule: Elements nested inside newly introduced or removed compound blocks are excluded from repeated counting.

IV. RESULT ANALYSIS

Defect distributions differ substantially across Android, Linux, and HarmonyOS in affected OS layers, functions, and trigger scenarios, reflecting distinct system profiles.

  • OS Layers: Android defects concentrate in the System Service Layer at 38.80%, Linux defects in the Application Layer at 54.20%, and HarmonyOS defects in the Kernel Layer at 71.80%.HarmonyOS Runtime or Virtual Machine defects also reach 8.80%.
  • Affected Function: Android’s leading affected function is Security and Permission Control at about 60.00%, Linux’s is Compatibility and Adaptation at nearly 41.20%, and HarmonyOS leads in Memory and Resource Management at 22.80% and Network and Communication at 20.00%.
  • Affected Function: Android and Linux show stronger concentration in particular affected-function categories, whereas HarmonyOS defects are distributed more evenly across categories.
  • Trigger Scenario: Android’s leading triggers are Boundary Condition at 37.00% and Permission/Identity Dependency at 31.20%, while Linux is dominated by Environment Dependency at 62.00%.
  • Trigger Scenario: HarmonyOS has the highest Boundary Condition Trigger rate at 52.60% and an elevated Concurrency/Multi-Thread Trigger rate of 24.00%.Multi-Language/Localization adaptation issues are rare across all three systems.

4) Severity:

Severity distributions differ across Android, Linux, and HarmonyOS, while repair patterns are comparatively similar across systems.

  • Severity distribution: Android’s high-severity share reflects security-focused defect reporting, while Linux reserves High for crashes, corruption, or breaches.Linux’s large unclassified share is associated with decentralized triage and older tools that do not enforce severity selection.
  • Severity distribution: HarmonyOS’s Medium-severity majority reflects a default classification for functional issues, including integration and performance bugs with workarounds.This classification approach avoids labeling such issues as High or Low.
  • Repair characteristics: Across systems, repairs most frequently modify if statements, function calls, assignments, and variables, with small fixes affecting no more than ten lines most common.The study interprets this prevalence of small patches as consistent with minimal repair.
  • Cross-system comparison: Defect distributions differ substantially across OS layers, affected functions, trigger scenarios, and severity, but repair code-element distributions are largely similar.This contrast separates system-specific defect characteristics from more consistent repair structures.

1) OS Layers vs Affected Functions:

The relationships between OS layers and affected functions vary markedly across Android, Linux, and HarmonyOS, despite strong within-system concentrations.

  • OS Layers vs Affected Functions: 60.00% of Android defects involve Security and Permission Control, with these issues spanning System Service, Application, and Framework layers.About 28.60% of all defects are System Service security or permission issues, while 13.60% and 12.20% are associated with Application and Framework layers, respectively.
  • OS Layers vs Affected Functions: 41.20% of Linux defects lead to Compatibility and Adaptability issues, and 25.60% of all defects in this category occur at the Application layer.Linux Framework-layer defects mainly affect Graphics and Rendering functions.
  • OS Layers vs Affected Functions: 71.80% of HarmonyOS defects occur in the Kernel layer, whose defects affect all identified function categories with a relatively even distribution.Device Management and Driver functions are dominantly affected among HarmonyOS kernel defects.
  • OS Layers vs Trigger Scenarios: Application-layer triggers differ across systems: Linux is dominated by Environment Dependency Issues, while Android and HarmonyOS more often involve Permission/Identity Dependency and Boundary Condition triggers.In Linux, Environment Dependency Issues account for 29.80% of 54.20% of Application-layer defects.
  • OS Layers vs Trigger Scenarios: Kernel defects in Android and HarmonyOS are dominated by Boundary Condition and Concurrency/Multi-thread issues, whereas Linux kernel defects most often involve Environment Dependency.The study attributes this contrast partly to Linux’s longer maturation and extensive code review.
  • OS Layers vs Trigger Scenarios: Boundary-condition defects are prevalent across all three systems and can appear at most OS layers, making condition-related detection and repair a common priority.This prevalence persists despite system-specific layer–trigger correlations.

3) Affected Functions vs Trigger Scenarios:

Boundary conditions affect every identified function category, while concurrency, regular-operation, and state-dependency triggers also show cross-system patterns; overall defect distributions remain heterogeneous.

  • Affected Functions vs Trigger Scenarios: Boundary Condition issues affect all identified functions, but their most frequent affected functions differ across Android, Linux, and HarmonyOS.Android emphasizes Security and Permission Control and Memory/Resource Management; Linux emphasizes Security and Permission Control and Storage Management; HarmonyOS emphasizes Memory/Resource Management, Network/Communication, and Security/Permission.
  • Affected Functions vs Trigger Scenarios: Missing boundary checks, including null-pointer checks, are identified as a major root cause affecting memory and resource-management correctness and safety.The study connects boundary-condition issues to both shared prevalence and system-specific affected-function patterns.
  • Affected Functions vs Trigger Scenarios: Concurrency/Multi-thread, Regular Operation, and State Dependency triggers affect diverse functions and exhibit comparable patterns across Android, Linux, and HarmonyOS.These cross-system regularities extend beyond boundary-condition defects.
  • Affected Functions vs Trigger Scenarios: Security and permission-related functions can be affected by regular operations as well as boundary conditions and permission or identity checks.The finding broadens the set of trigger scenarios relevant to OS security analysis.
  • Cross-OS Similarity Analysis: The study quantifies cross-OS similarity using Spearman correlation across OS layer, affected-function, and trigger-scenario distributions, excluding severity.Figure 4 heatmaps classify correlation strength from weak or no correlation through very strong correlation.
  • Cross-OS Similarity Analysis: Overall defect distributions show no substantive monotonic correlation: the largest absolute coefficient is -0.11 between Linux and Android.The result indicates substantial divergence in overall defect composition.
  • Cross-OS Similarity Analysis: HarmonyOS and Android have a strong negative OS-layer correlation of -0.66, while Linux shows little correlation with either system.Affected-function and trigger-scenario correlations are also limited, including a strongest trigger correlation of 0.41 between HarmonyOS and Linux.
  • Cross-OS Similarity Analysis: Defect distributions differ substantially across systems, supporting more targeted debugging techniques rather than assuming cross-OS patterns generalize.This conclusion follows the reported cross-system similarity analysis.

V. IMPLICATIONS

The study translates system-specific defect distributions into debugging priorities for Android, Linux, and HarmonyOS.

  • Android: Android debugging should prioritize permission management, system services, and edge-case handling because defects concentrate in Security and Permission Control and the System Service Layer.Static analysis of permission flows is specifically recommended.
  • Linux: Linux debugging should prioritize environment configuration, cross-platform adaptation, and application-layer implementation because Compatibility and Adaptation defects dominate and over 60.00% are environment-triggered.Environment snapshot and replay, dependency manifests, containerization, and automated cross-platform testing are recommended.
  • HarmonyOS: HarmonyOS troubleshooting should focus on resource scheduling, boundary checks, and multithread synchronization because defects concentrate in Memory and Resource Management and the Kernel Layer.Stress testing with HDC shell commands is recommended for low-memory and concurrent-application scenarios.

B. Regarding Defect Repair

Operating-system defect repairs are usually small and localized, but a substantial subset requires coordinated changes beyond the scope of many existing repair techniques. The study’s repair findings are supported by a large, manually labeled dataset, with explicit validity controls and data-source boundaries.

  • Fewer than 30 lines of code modification are required for most defect-fixing changes across all three systems.These repairs typically affect only a few files and functions.
  • Minimal modifications are recommended because focused edits can address root causes while reducing review complexity and fault surface.Developers should avoid unrelated refactoring or cleanup during defect fixes.
  • A significant portion of repairs spans multiple methods or files, exposing a mismatch with automated repair techniques designed mainly for single-method or single-file changes.The finding motivates repair methods that propagate fixes across function boundaries, shared data structures, or module interfaces.
  • Manual labeling was supported by independent annotation, iterative Cohen’s Kappa tracking above 0.8, third-author adjudication, and open-sourced data.Later labeling iterations reached Kappa values from 0.9 to 1.0.
  • The study analyzes 1,500 defects across Android, Linux, and HarmonyOS using multiple defect dimensions and summarizes five key findings.The dataset covers representative mobile and desktop operating systems.
Loading 2608.20643v1…