PaperlayerView exact revision on arXiv

Source-linked AI summary

An Investigation into the Use of Common Libraries in Android Apps

Li Li, Tegawendé F. Bissyandé, Jacques Klein, Yves Le Traon

arXiv:1511.06554v1cs.SE

TL;DR

Android analyses may include common-library code that is not part of an app’s functionality, and prior work lacked a comprehensive shared library set. The authors mine apps from Google Play to build common- and ad-library collections and report that the sets benefit piggybacking detection and malware detection.

  • Problem

    Common-library code can add noise to Android analyses, while prior work had not produced a comprehensive library set for researchers.

    Source: §I · pp. 1–2

  • Method

    The authors analyze about 1.5 million Android apps, identify recurring same-name packages with similar code, and label harvested libraries as advertisements or other common libraries.

    Source: §I · p. 2, §III · p. 3

  • Results

    The authors report that their library sets benefit piggybacking detection and machine-learning-based malware detection.

    Source: §VIII · p. 10

  • Takeaways & Limitations

    The harvested libraries can serve as whitelists that reduce both false positive and false negative rates in piggybacked-app detection.

    Source: §I · p. 2

    Well-used libraries such as unit3d can themselves be used as compromising points for malicious apps.

    Source: §VI · p. 10

Abstract

from arXiv · show

The packaging model of Android apps requires the entire code necessary for the execution of an app to be shipped into one single apk file. Thus, an analysis of Android apps often visits code which is not part of the functionality delivered by the app. Such code is often contributed by the common libraries which are used pervasively by all apps. Unfortunately, Android analyses, e.g., for piggybacking detection and malware detection, can produce inaccurate results if they do not take into account the case of library code, which constitute noise in app features. Despite some efforts on investigating Android libraries, the momentum of Android research has not yet produced a complete set of common libraries to further support in-depth analysis of Android apps. In this paper, we leverage a dataset of about 1.5 million apps from Google Play to harvest potential common libraries, including advertisement libraries. With several steps of refinements, we finally collect by far the largest set of 1,113 libraries supporting common functionalities and 240 libraries for advertisement. We use the dataset to investigates several aspects of Android libraries, including their popularity and their proportion in Android app code. Based on these datasets, we have further performed several empirical investigations to confirm the motivations behind our work.

I. INTRODUCTION

Common-library code can interfere with Android static analyses, yet prior library lists were limited or unavailable. This paper builds and shares large common- and ad-library collections and investigates their use in app analyses.

I. INTRODUCTION

  • Static analyses can include substantial third-party library code unrelated to an app’s functionality, creating a barrier to code exploration.The paper identifies repackaging detection, malware detection, and code analysis as affected tasks.Source: §I · p. 1
  • Prior findings attributed over 60% of Android app code to common libraries, which can distort similarity-based repackaging detection.The paper notes that some recent approaches filter libraries using heuristics.Source: §I · p. 1
  • Library code can add noise to malware-detection features, and some approaches assume advertisement libraries are trustworthy and ignore their results.Source: §I · p. 1
  • Libraries may also consume analysis resources on irrelevant code: prior estimates suggest they account for over half of CPU and memory use.The passage frames this as an implication of findings that libraries contribute 60% of app code.Source: §I · p. 1
  • Existing common-library collections were either unavailable to the community or limited to between 9 and 103 libraries.The paper also notes that a reported collection of over 600 libraries was not publicly available and might not represent other datasets.Source: §I · p. 2
  • The authors collect 1,113 common libraries from about 1.5 million apps, identify 240 ad libraries, and report benefits for piggybacking detection.They also investigate library use and report significant differences between benign and malicious apps.Source: §I · p. 2

II. MOTIVATING EXAMPLE

Common libraries can mislead piggybacking comparisons in opposite ways: they can make unrelated apps appear similar or mask a true piggybacking relationship. The paper motivates filtering them and outlines a process for identifying common libraries and labeling ad libraries.

II. MOTIVATING EXAMPLE

  • Common libraries can skew pairwise comparisons in either direction, motivating library filtering in piggybacking detection.The paper frames piggybacking as modifying and re-signing an existing app, typically with a new payload, before distributing it as a new app.Source: §II · p. 2

A. Mislabeling Legitimate apps as Piggybacked

  • Filtering common libraries reduces the motivating legitimate-app pair’s similarity from 86% to 0%, avoiding a false positive at the 80% threshold.The apps share packages such as com.android and com.adobe.Source: A. Mislabeling Legitimate apps as… · p. 3

B. Missing True Piggybacked Apps

  • For the known piggybacked pair, filtering raises similarity from 47% to 84%; the authors aim to provide a comprehensive whitelist amid limited or rarely shared alternatives.The pair’s library replacement redirected the legitimate app’s revenues to another developer.Source: B. Missing True Piggybacked Apps · p. 3

III. IDENTIFICATION OF COMMON LIBRARIES

  • The approach ranks recurring package names across apps, checks same-name candidates for code similarity, then labels harvested libraries as advertisements or not.The process is intended to harvest common libraries and identify advertisement libraries among them.Source: §III · p. 3

A. Step 1: Candidate Libraries Extraction

Candidate libraries are extracted from recurring package names under an assumption of unmodified reuse, then confirmed through code similarity to distinguish libraries from ordinary app packages. Confirmation filters package candidates and compares app and package similarity while accounting for repackaging and piggybacking.

A. Step 1: Candidate Libraries Extraction

  • Common-library candidates are recurring packages assumed to be used by many apps without developer modification.Package names are extracted and clustered by frequency.Source: A. Step 1: Candidate Libraries… · p. 3
  • Extraction considers the first three package-name segments, or the full name when shorter, and excludes android.support packages.The authors exclude android.support because these packages are part of the Android framework and pervasive in apps.Source: A. Step 1: Candidate Libraries… · p. 3

B. Step 2: Common Libraries Confirmation

  • Recurring names alone can reflect piggybacking, obfuscation, or simplistic naming, so code similarity refines candidates into common libraries.Pairwise comparison is costly, so heuristics first remove packages deemed irrelevant.Source: B. Step 2: Common Libraries Confirmation · pp. 3–4
  • Confirmation excludes packages appearing in 10 or fewer apps, one-segment names, single-letter obfuscated segments, and prefixes of other packages.These filters aim to reduce candidate volume and naming collisions before pairwise code comparison.Source: B. Step 2: Common Libraries Confirmation · p. 4
  • For scalability, the process randomly compares 10 app pairs per package name rather than every possible pair.Each package candidate appears in at least 10 apps, which would yield at least 45 pairs.Source: B. Step 2: Common Libraries Confirmation · p. 4
  • Method similarity abstracts statement types to reduce obfuscation effects, while package and app comparisons apply separate thresholds tp and ta.The method comparison also considers SDK API method names; package similarity must exceed tp, while app similarity must remain below ta.Source: B. Step 2: Common Libraries Confirmation · pp. 4–5

C. Step 3: Identification of Ad Libraries

Advertisement libraries are a pervasive Android library type with implications for app revenue and security analysis. The paper proposes mining common libraries to identify ads through package names and library characteristics.

C. Step 3: Identification of Ad Libraries

  • Ad libraries are pervasive because they are a major way app developers are rewarded, and they may be inserted during piggybacking to redirect revenue.Their presence can also lead antivirus products to flag apps as adware.Source: C. Step 3: Identification of Ad… · p. 5
  • Android security approaches may process ad-library code separately; MUDFLOW uses a whitelist of only 12 libraries.MUDFLOW does not report potential sensitive-data leaks through ad libraries because they may be legitimate.Source: C. Step 3: Identification of Ad… · p. 5
  • The paper proposes mining its common-library set for ad libraries using both package-name matching and semantic characteristics.Source: C. Step 3: Identification of Ad… · p. 5

1) Keywords matching:

The authors identify ad libraries using name-based screening and characteristic-based checks. The latter examine internet use, declared components, and View-based classes.

1) Keywords matching:

  • Matching “ad” in package names risks false positives from ordinary words, so the authors filter against 13,385 English words containing “ad.”Examples of misleading segments include shadow, gadget, load, adapter, and adobe.Source: 1) Keywords matching · p. 5

2) Ad features investigations:

  • Manual investigation of ad libraries informs characteristic-based screening, including whether a package uses internet-related APIs.Because apps may request INTERNET for their own needs, the permission alone does not establish that a library is an ad library.Source: 2) Ad features investigations · p. 5
  • Ad libraries may declare components for ad interactions, so a library package associated with a declared component is flagged as a potential ad library.The paper gives an activity used to support video ads as an example.Source: 2) Ad features investigations · p. 5
  • Because ads are visualized through Android views, the authors check common libraries for classes extending android.view.View.Source: 2) Ad features investigations · p. 5

D. Implementation details

The implementation disassembles apps and analyzes code with Soot, using a Google Play dataset of about 1.5 million apps. The reported common-library set varies with similarity thresholds, with the selected precise set containing 1,113 libraries.

D. Implementation details

  • The implementation disassembles apps with Apktool, represents packages using smali-file path prefixes, and ranks clustered packages by frequency.Packages larger than a threshold become library candidates.Source: D. Implementation details · p. 5
  • Soot performs code-similarity analysis and ad-library confirmation at the Jimple level, with Dexpler translating Dalvik bytecode into Jimple.Source: D. Implementation details · p. 5

A. Dataset

  • The dataset contains 1,455,516 apps collected from Google Play over several months.The authors report that nearly 21% were flagged by at least one VirusTotal product and nearly 4% by at least five.Source: A. Dataset · p. 6

B. Overall Results

  • The refinement process reduces 676,674 distinct package names to 16,732 packages for pairwise similarity analysis.It first filters out 656,949 names used by at most 10 apps, then dismisses 2,993 more through refinement.Source: B. Overall Results · p. 6

1) Results of Common Libraries:

2) Results of Ad Libraries:

The authors refine harvested common libraries into a 240-library set of highly precise ad libraries. Among common libraries, com.google.ads is the most used, appearing in nearly 17% of the dataset’s apps.

2) Results of Ad Libraries:

  • Keyword matching harvested 275 ad libraries, while intersecting three ad characteristics identified 822; merging the sets yielded AD1050.The three characteristics were INTERNET permission, declared View gadgets, and declared components.Source: 2) Results of Ad Libraries · p. 6, Figure 4 · p. 6
  • Manual refinement confirmed 240 libraries as ad libraries, forming the highly precise AD240 set.The authors describe AD1050 as potentially containing false positives before this refinement.Source: 2) Results of Ad Libraries · pp. 6–7

C. Popularity of common libraries

D. Proportion of Library code in App code

In a random sample of 10,000 apps, common-library code had a median share of 0.41, and 42.9% had more library code than app-specific code. The empirical investigations also compare library use in benign and antivirus-flagged apps and test library features for classification.

D. Proportion of Library code in App code

  • The 10,000 sampled apps had a median library-code share of 0.41; 4,293 (42.9%) had p >= 0.5.The share was computed as library size divided by whole-app size.Source: D. Proportion of Library code in App… · p. 7
  • The paper also reports that 42% of sampled packages contained more common-library code than app-specific code, with an average library-code share of 41%.This summary appears alongside the proportion results.Source: D. Proportion of Library code in App… · p. 7

V. EMPIRICAL INVESTIGATIONS

  • The investigations ask whether libraries distinguish benign from antivirus-flagged apps, improve Android analysis, and how exhaustive the ad-library set is versus VirusTotal.The analysis includes piggybacking detection and machine-learning-based malware detection.Source: §V · p. 7

A. RQ1: Benign vs. Antivirus-flagged Apps

B. RQ2: Improvements to Analysis Approaches

Excluding common libraries from app pairs changed some piggybacking similarity classifications: it removed some apparent positives and revealed two verified piggybacked pairs among prior negatives.

1) Piggybacking detection:

  • The experiment compared 1,100 pairs initially above 80% similarity with 761 pairs below 50%, then recalculated similarity after excluding CL91 libraries.The authors treated the initially high-similarity FPData pairs as piggybacked and the low-similarity FNData pairs as not piggybacked.Source: 1) Piggybacking detection · p. 8
  • 71 of 1,100 FPData pairs (6.5%) fell below the 80% similarity threshold after library exclusion, and 10 manually checked pairs were not piggybacked.The remaining 1,029 pairs stayed above the threshold.Source: 1) Piggybacking detection · p. 8
  • Among the 761 FNData app pairs, 110 (14%) had higher similarity scores; 2 exceeded the 80% threshold, and manual verification confirmed both as piggybacked pairs.Source: 1) Piggybacking detection · p. 8

2) Machine learning for Malware detection:

In the MUDFLOW-based case study, excluding the authors’ harvested libraries increased accuracy, while excluding a broader, less precise set slightly reduced it. The ad-library comparison found 62% completeness for AD240 and 82% for AD1050, with gaps for some adware keywords.

2) Machine learning for Malware detection:

  • The malware-detection case study used MUDFLOW’s dataset and a simplified 10-fold cross-validation, one-class classification evaluation.The dataset contains sensitive-data-leak information for 15,096 malicious and 2,800 benign apps.Source: 2) Machine learning for Malware… · p. 8
  • Accuracy increased when E7 excluded the harvested 1,113 libraries versus E5, while E8’s broader 5,509-library set slightly reduced accuracy relative to E7.E6, which excluded MUDFLOW’s 12 ad libraries, had the same accuracy as E5.Source: 2) Machine learning for Malware… · p. 9, Table V · p. 9

C. RQ3: Completeness of our harvested ad libraries

VI. THREATS TO VALIDITY AND DISCUSSION

The study notes that library identification is difficult and that obfuscation and unaccounted library versions constrain its results, while also reporting how libraries relate to malicious apps.

VI. THREATS TO VALIDITY AND DISCUSSION

  • Without a convention for marking library packages, identification is challenging; the study mined about 1.5 million apps collected over several months.Source: §VI · p. 9
  • Obfuscation awareness is incomplete and may leave results incomplete; dataset scale reduces its influence, while SDK API-method features remain a proposed validation path.The authors plan future experiments to validate that possibility.Source: §VI · pp. 9–10
  • Ignoring library versions may threaten similarity analysis because similarity can vary significantly across versions, though dataset variety and scale somewhat mitigate this threat.Source: §VI · p. 10
  • Well-used libraries such as unit3d can be compromise points, while malware writers also masquerade as reputed libraries, a fraud similarity analysis detected as outliers.DroidKungFu used the package name com.google.update for a malicious payload.Source: §VI · p. 10

VII. RELATED WORK

Related work describes security and privacy risks from libraries and their effects on app analysis. The paper presents publicly accessible library sets and reports that whitelisting them can benefit piggybacking detection and machine-learning malware detection.

VII. RELATED WORK

  • Reported library threats include malicious modification, masquerading, aggressive behavior, privacy leakage, cross-app tracking, excessive permissions, and unsafe remote-code execution.Prior work also associates free ad libraries with lower app ratings.Source: §VII · p. 10
  • The library list can provide a starting point for API studies and help other approaches refine their results.Source: §VII · p. 10
  • Clone detection and machine-learning malware detection have used library whitelists described as incomplete or limited, including a 73-library list and a 12-ad-library list.Source: §VII · p. 10
  • Earlier identification work includes Wang et al.’s clustering of over 600 libraries and AdDetect’s semantic ad-library classification, which reported no community-benefiting findings.Source: §VII · p. 10

VIII. CONCLUSION

  • The study collected 1,113 common libraries and 240 ad libraries from about 1.5 million apps, describing both sets as the largest publicly accessible in Android research.Source: §VIII · p. 10
  • The authors report that whitelisting the harvested libraries can improve Android analysis performance, with benefits shown for piggybacking detection and machine-learning malware detection.Source: §VIII · p. 10
Loading 1511.06554v1…