PaperlayerView exact revision on arXiv

Source-linked AI summary

Ghera: A Repository of Android App Vulnerability Benchmarks

Joydeep Mitra, Venkatesh-Prasad Ranganath

arXiv:1708.02380v1cs.CRcs.SE

TL;DR

Android vulnerability-detection tools lack comprehensive benchmark suites for rigorous evaluation. The paper presents Ghera, a repository of paired vulnerable and exploiting apps, and identifies desirable benchmark characteristics. Ghera currently captures 25 known vulnerabilities, but does not cover all framework areas or known vulnerabilities.

  • Problem

    The authors found few Android benchmarks for evaluating vulnerability-detection tools, and existing examples did not explicitly capture known vulnerabilities.

    Source: §2.1 · p. 2

  • Method

    The authors created Ghera by identifying potential vulnerabilities through prior research, discussions, documentation, and Android source, then building and testing benign and exploiting apps.

    Source: §3.1 · p. 5, §3.2 · p. 5

  • Results

    Ghera captures 25 known Android app vulnerabilities in benchmarks spanning four platform areas.

    Source: §5 · p. 7

  • Takeaways & Limitations

    The authors plan to extend Ghera and use it in an ongoing evaluation effort, and hope community contributions improve tool evaluation.

    Source: §5 · p. 7

    Ghera covers four framework areas, but known vulnerabilities in other areas and additional vulnerabilities within those areas may be absent.

    Source: §3.5 · p. 7

Abstract

from arXiv · show

Security of mobile apps affects the security of their users. This has fueled the development of techniques to automatically detect vulnerabilities in mobile apps and help developers secure their apps; specifically, in the context of Android platform due to openness and ubiquitousness of the platform. Despite a slew of research efforts in this space, there is no comprehensive repository of up-to-date and lean benchmarks that contain most of the known Android app vulnerabilities and, consequently, can be used to rigorously evaluate both existing and new vulnerability detection techniques and help developers learn about Android app vulnerabilities. In this paper, we describe Ghera, an open source repository of benchmarks that capture 25 known vulnerabilities in Android apps (as pairs of exploited/benign and exploiting/malicious apps). We also present desirable characteristics of vulnerability benchmarks and repositories that we uncovered while creating Ghera.

1 INTRODUCTION

Android apps handle sensitive personal data, and their security matters to users; evaluating vulnerability-detection tools rigorously and reproducibly is therefore important. The paper introduces Ghera and identifies desirable benchmark characteristics.

1.1 Motivation

  • Android apps use sensitive personal data, while known vulnerabilities can reflect developers’ lack of security knowledge or understanding.The paper presents vulnerable apps as a way to demonstrate how security can go awry and help developers learn about known vulnerabilities.Source: §1.1 · pp. 1–2
  • Tool verdicts need rigorous, reproducible evaluation, and a common benchmark baseline supports fair, comparable comparisons across techniques.Such comparisons can help developers assess efficacy, scale, and ease of use.Source: §1.1 · p. 1

1.2 Contributions

  • The authors identify a gap in comprehensive, tool-agnostic repositories and introduce Ghera, with 25 vulnerability benchmarks pairing benign and malicious apps across four Android framework areas.The areas are Inter Component Communication (ICC), Storage, System, and Web.Source: §1.1 · p. 2, §1.2 · p. 2
  • A retrospective on creating Ghera yields desirable vulnerability-benchmark characteristics that the authors say also apply to benchmarks more generally.The paper presents Ghera alongside these characteristics.Source: §1.2 · p. 2

2 DESIRABLE CHARACTERISTICS OF VULNERABILITY BENCHMARKS

The paper derives benchmark and repository characteristics from the need for rigorous vulnerability-tool evaluation and a review of existing efforts. It describes how these characteristics affect evaluation fairness, focus, reproducibility, and scope.

2.2.1 Tool and Technique Agnostic.

  • Tool- and technique-agnostic benchmarks do not depend on how tools detect vulnerabilities, supporting fair evaluation and comparison.The paper contrasts these with DroidBench, whose cases are tailored to taint-analysis tools.Source: §2.2.1 · p. 2

2.2.2 Authentic.

  • Authentic benchmarks truly contain the vulnerabilities they claim to capture, allowing them to serve as ground truth for accuracy evaluations.The paper notes that AndroZoo offers evidence of malicious behavior but not evidence that apps contain vulnerabilities.Source: §2.2.2 · pp. 2–3

2.2.3 Feature Specific.

  • Feature-specific benchmarks isolate vulnerability causes, helping evaluate whether tools detect vulnerabilities arising from particular features.They can also help assess whether a tool’s explanations identify the cause of a detected vulnerability.Source: §2.2.3 · p. 3

2.2.4 Contextual.

  • Contextual benchmarks focus evaluation on settings such as real-world scale, experimentation, or specific libraries; lean cases can aid debugging before fat benchmarks test scale.The authors report that establishing tool veracity was easier in evaluations using lean and fat benchmarks than in those using only fat benchmarks.Source: §2.2.4 · p. 3

2.2.5 Ready-to-Use. / 2.2.6 Easy-to-Use.

  • Ready-to-use, easy-to-use benchmarks reduce setup friction: artifacts reproduce vulnerabilities directly, while simple setup expedites evaluation and can encourage adoption.Ready-to-use artifacts avoid dependence on interpreting instructions or developer skill; source projects can also ease building and evaluation.Source: §2.2.5 · p. 3, §2.2.6 · p. 3

2.2.7 Version Specific. / 2.2.8 Well Documented.

  • Version-specific benchmarks identify compatible platform versions, avoiding evaluations that mislabel detection on versions where a vulnerability no longer exists.Documentation should also describe vulnerabilities, features, compatible versions, reproduction and exploitation instructions, and, when possible, source code.Source: §2.2.7 · pp. 3–4, §2.2.8 · p. 4

2.2.9 Dual.

  • Dual benchmarks pair a vulnerability with its exploit, supporting evaluations that exercise vulnerabilities, demonstrations, exploit evaluation, and authenticity checks.The authors report finding no dual benchmarks in their exploration.Source: §2.2.9 · p. 4

2.3.1 Open. / 2.3.2 Comprehensive. / 2.4 Discussion

  • Repository characteristics include openness for access and contributions, and broad coverage that lets evaluations consider most known vulnerabilities from one or few repositories.The authors found no single repository covering almost all Android app vulnerabilities, and note that not all desirable characteristics are equally easy to achieve.Source: §2.3.1 · p. 4, §2.3.2 · p. 4, §2.4 · p. 4

3 GHERA

Ghera was built as a tool-agnostic repository of Android app vulnerability benchmarks, organized around common Android capabilities and documented with reproducible vulnerable/exploiting app pairs. The paper describes its design and structure, assesses its benchmark characteristics, and notes limits in coverage and scalability evaluation.

3.1 Design Choices

  • Ghera’s design goals were unique, tool-agnostic, easy-to-use, well-documented benchmarks with source code, while making the repository open and as comprehensive as possible.The authors grouped vulnerabilities by Android capabilities and developed and tested benign/exploiting app pairs on Android 4.4 through 7.1.Source: §3 · p. 5, §3.1 · p. 5

3.2 Structure and Content

  • Each benchmark folder contains benign and malicious app source, a README describing the vulnerability and exploit, build instructions, and tested Android versions.Web benchmarks omit a malicious-app folder because MitM attacks exploit the benign app through a web server.Source: §3.2 · p. 5

3.3 Work Flow

  • The illustrated workflow creates an emulator, builds and installs both apps, launches them in sequence, and executes a command; other benchmarks follow a similar workflow with minor changes.Instructions assume emulators, though users can instead connect an Android device.Source: §3.3 · pp. 5–6

3.2 Structure and Content

  • 25 benchmarks capture unique vulnerabilities across ICC (13), Storage (2), System (4), and Web (6), with source-code/configuration sizes ranging from 490 to 1,510 lines.Storage benchmarks are larger in bytes because they rely on two external APK files totaling 13,144 KB.Source: §3.2 · p. 5, Table 1 · p. 6

3.4 What about characteristics?

  • Benchmarks are version-specific and documented with vulnerability and exploit descriptions, reproduction instructions, and tested Android versions; where possible, each pairs a vulnerable app with an exploiting app.The paper also characterizes this pairing as a dual benchmark characteristic.Source: §3.4 · p. 6
  • Ghera’s benchmarks are described as open, tool-agnostic, authentic, feature-specific, contextual, ready-to-use, and easy-to-use.The repository is public and accepts contributions; vulnerability presence is checked through exploitation rather than detection tools, and benign apps use few unrelated framework features.Source: §3.4 · p. 6

3.5 Limitations & Threats to Validity

  • Because Ghera currently contains only lean benchmarks, it cannot evaluate vulnerability-detection-tool scalability, which would require fat benchmarks.Source: §3.5 · p. 7
  • Ghera is not comprehensive: it covers four framework areas, while known vulnerabilities may remain uncovered both in other areas and within those four.The authors also caution that their process-based claims of feature specificity and contextuality do not account for bias or oversight.Source: §3.4 · p. 6, §3.5 · p. 7

4 FUTURE WORK

The paper proposes extending Ghera’s vulnerability coverage and adding real-world app contexts. It also identifies code-pattern extraction and tool evaluation as ways to use the repository in future work.

4 FUTURE WORK

  • Add benchmarks for uncaptured vulnerabilities in the four existing areas and for Android API areas such as Networking and Camera.Source: §4 · p. 7
  • Add real-world apps containing vulnerabilities represented by existing benchmarks as context-specific benchmarks to help evaluate techniques for scale.Source: §4 · p. 7
  • Extract benchmark code patterns to identify similar instances, develop IDE plugins to help developers avoid vulnerabilities, and measure those vulnerabilities’ prevalence in real-world apps.Source: §4 · p. 7
  • Create benchmarks from Android app vulnerabilities listed on CVE and evaluate existing analysis tools to compare them and explore possible accuracy improvements from combining techniques.The authors report that they are pursuing an evaluation of existing tools.Source: §4 · p. 7

5 SUMMARY

Ghera is an open repository of Android app vulnerability benchmarks covering 25 known vulnerabilities across four platform capabilities. The paper also documents desirable benchmark and repository characteristics as guidance for rigorous, reproducible evaluations.

5 SUMMARY

  • Ghera captures 25 known Android app vulnerabilities across four Android platform capabilities.The authors plan to extend the repository and use it in an ongoing tool-evaluation effort.Source: §5 · p. 7
  • The authors hope the community will use and contribute to Ghera to improve evaluations of Android app vulnerability detection tools and techniques.Source: §5 · p. 7
  • The paper documents desirable characteristics of vulnerability benchmarks and repositories to guide rigorous, reproducible evaluations.Source: §5 · p. 7

A CATALOG OF BENCHMARKS

The catalog organizes Ghera's current benchmarks by vulnerability category and describes each vulnerability and its exploit.

A CATALOG OF BENCHMARKS

A.1 Inter Component Communication

Android apps use four component types for interface, background work, event handling, and data management; Intents facilitate communication between components.

A.1 Inter Component Communication

  • Android apps have four basic component types: Activities, Services, Broadcast Receivers, and Content Providers.Source: §A.1 · p. 8
  • Activities display the user interface, Services perform background operations, Broadcast Receivers handle event notifications, and Content Providers manage app data.Source: §A.1 · p. 8
  • Components communicate within and across apps through Intents, while intent-filters specify which kinds of intents they can process.Source: §A.1 · p. 8

A.1.1 Dynamically registered broadcast receiver provides unrestricted access.

A dynamically registered broadcast receiver with a non-null intent filter is automatically exported, allowing other apps to reach it and potentially trigger unintended actions.

A.1.1 Dynamically registered broadcast receiver provides unrestricted access.

  • Providing a non-null intent filter when dynamically registering a broadcast receiver automatically exports it for access by other apps.Source: §A.1.1 · p. 8
  • A malicious app can broadcast a message to trigger the dynamically registered receiver.Source: §A.1.1 · p. 8
  • The receiver may then unintentionally perform an action on behalf of the malicious app.Source: §A.1.1 · p. 8

A.1.2 Empty pending intent leaks privilege.

An empty pending intent lets its recipient choose an action that runs with the sending app’s authority.

A.1.2 Empty pending intent leaks privilege.

  • An app can create a pending intent with no specified action, leaving the recipient able to set one.The selected action is executed in the context of the app that sent the pending intent.Source: §A.1.2 · p. 8
  • A malicious app can associate its chosen action with the empty pending intent, causing that action to run in the sender’s context when processed.Source: §A.1.2 · p. 8

A.1.3 Low priority activity prone to hijacking.

Android orders matching activities by priority, allowing a higher-priority malicious activity to appear before a benign one.

A.1.3 Low priority activity prone to hijacking.

  • Android displays activities sharing an intent-filter in priority order when an activity is started.Source: §A.1.3 · p. 8
  • A malicious app can register a matching activity with higher priority, so it is displayed before the benign activity.Source: §A.1.3 · p. 8

A.1.4 Service started by implicit intent is prone to hijacking.

Android selects the highest-priority service matching an implicit intent, so a malicious app can hijack a request intended for a benign service.

A.1.4 Service started by implicit intent is prone to hijacking.

  • For an implicit intent matching multiple services, Android chooses the service with higher priority.Source: §A.1.4 · p. 8
  • A malicious app can register a higher-priority service with the same intent-filter, causing it to start instead of the intended benign service.Source: §A.1.4 · p. 8

A.1.5 Implicit pending intent leaks information.

The section catalogs Android vulnerabilities in which intents, app components, storage, permission checks, and WebView behavior expose information or enable malicious actions.

A.1.5 Implicit pending intent leaks information; A.1.6 Content provider with inadequate path-permission leaks information.

  • A higher-priority matching component can intercept an implicit intent inside a pending intent, while inadequate path-permissions leave provider subdirectories and files accessible.Source: §A.1.5 · p. 8, §A.1.6 · p. 8

A.1.7 Apps have unrestricted access to Broadcast receivers registered for system events; A.1.8 Ordered broadcasts allow malicious data injection; A.1.9 Sticky broadcasts are prone to leaking sensitive information and malicious data injection.

A.1.10 Task affinity makes an app vulnerable to phishing atacks; A.1.11 Task affinity and task re-parenting enables phishing and denial-of-service; A.1.12 Content Provider API allow unauthorized access.

  • Task affinity and re-parenting can let a malicious activity supersede benign activities for phishing or denial-of-service, while provider call methods can expose data without specific permissions.Source: §A.1.10 · p. 9, §A.1.11 · p. 9, §A.1.12 · p. 9

A.2.1 External storage allows data injection atack; A.2.2 Writing sensitive information to external storage enables information leak.

A.3.1 checkCallingOrSelfPermission method leaks privilege; A.3.3 enforceCallingOrSelfPermission method leaks privilege.

  • Permission checks using checkCallingOrSelfPermission or enforceCallingOrSelfPermission can pass when the protected app itself holds the permission, rather than the requester.Source: §A.3.1 · pp. 9–10, §A.3.3 · p. 10

A.3.2 checkPermission method leaks privilege; A.3.4 enforcePermission method leaks privilege.

  • In the app’s main thread, getCallingPID and getCallingUID return that app’s IDs, potentially misdirecting checkPermission and enforcePermission checks.Source: §A.3.2 · p. 10, §A.3.4 · p. 10

A.4.1 Incorrect hostname verification enables Man-in-the-Middle (MitM) atack; A.4.2 Incorrect trust validation enables Man-in-the-Middle attack.

  • Weak custom hostname verification or trust validation can allow connections to malicious servers or trust unknown certificates, enabling MitM attacks.Source: §A.4.1 · p. 10, §A.4.2 · p. 10

A.4.3 Allowing execution of unverified JavaScript code in WebView exposes app’s resources; A.4.4 Ignoring SSL errors in WebViewClient enables Man-in-the-Middle (MitM) atack; A.4.5 Lack of validation of resource load requests in WebView allows loading malicious content; A.4.6 Lack of validation web page load requests in WebView allows loading malicious content.

Loading 1708.02380v1…