PaperlayerView exact revision on arXiv

Source-linked AI summary

A First Look at Deep Learning Apps on Smartphones

Mengwei Xu, Jiawei Liu, Yuanqiang Liu, Felix Xiaozhu Lin, Yunxin Liu, Xuanzhe Liu

arXiv:1812.05448v4cs.LGcs.CY

TL;DR

The study addresses limited empirical evidence about how smartphone apps adopt and use deep learning in practice. It analyzes 16,500 Android apps with a static tool, finding that early adopters are top apps and that deep learning commonly supports core functionality, with photo beauty the leading use case.

  • Problem

    The study fills an empirical gap between mobile deep learning research and practice by examining which real-world Android apps adopt deep learning, how they use it, and what their models look like.

  • Method

    The authors build a static analysis tool that dissects Android apps, detects popular deep learning frameworks, and analyzes their deep learning functions and models.

  • Results

    Early adopters are top apps; 81% of deep learning apps use deep learning for core functionality, and photo beauty is the most popular use case at 44.5%.

  • Takeaways & Limitations

    The findings provide implications for app developers, hardware designers, and researchers, including encouraging relevant apps to embrace deep learning and motivating higher-level abstractions.

  • Takeaways & Limitations

    The analyzer can miss apps using non-popular frameworks or no detectable string patterns and can falsely classify apps that integrate but do not use deep learning.

Abstract

from arXiv · show

We are in the dawn of deep learning explosion for smartphones. To bridge the gap between research and practice, we present the first empirical study on 16,500 the most popular Android apps, demystifying how smartphone apps exploit deep learning in the wild. To this end, we build a new static tool that dissects apps and analyzes their deep learning functions. Our study answers threefold questions: what are the early adopter apps of deep learning, what do they use deep learning for, and how do their deep learning models look like. Our study has strong implications for app developers, smartphone vendors, and deep learning R\&D. On one hand, our findings paint a promising picture of deep learning for smartphones, showing the prosperity of mobile deep learning frameworks as well as the prosperity of apps building their cores atop deep learning. On the other hand, our findings urge optimizations on deep learning models deployed on smartphones, the protection of these models, and validation of research ideas on these models.

1 INTRODUCTION

This paper presents a first large-scale empirical study of deep learning adoption in real-world Android apps, using a static analyzer to examine 16,500 popular apps. It finds rapid adoption, core functionality use, growing mobile-framework traction, lightweight but underoptimized models, and weak model protection.

  • Key findings: 211 deep learning apps represented only 1.3% of apps but contributed 11.9% of downloads and 10.5% of reviews.Their number increased by 27% over three months.
  • Key findings: 81% of deep learning apps used deep learning for core functionality, and this group grew by 23% over three months.The paper defines core functionality as frequently invoked and essential to operation.
  • Key findings: Photo beauty was the most popular use case, appearing in 94 apps, or 44.5% of deep learning apps.Photography apps accounted for 61 apps, or 29%.
  • Key findings: TFLite adoption grew by 258% over three months while mobile-optimized frameworks gained traction alongside full-fledged frameworks.TensorFlow remained popular because of its momentum.
  • Model characteristics: Only 6% of deep learning models used studied optimizations, despite quantization reducing cost by up to two orders of magnitude with little accuracy loss.The paper also reports median model memory of 2.47 MB and inference computation of 10M FLOPs.
  • Model characteristics: Only 39.2% of discovered models were obfuscated and 19.2% encrypted, leaving remaining models trivial to extract and vulnerable to unauthorized reuse.The findings motivate model protection, smartphone-specific optimization, and validation of research ideas on deployed lightweight models.
  • Study scope and approach: 16,500 popular Android apps were studied with a tool that identifies deep learning usage and extracts models for inspection.The study analyzes apps from 33 Google Play categories.

2 BACKGROUND

Deep learning systems deploy trained models and frameworks to perform inference for tasks such as face recognition and language translation. Smartphone inference can run on-device or in the cloud, with this study focusing on on-device execution.

  • Deep learning components: Deep learning models consist of organized neuron layers, including convolution, pooling, and fully-connected layers.Different layer organizations define model categories such as CNNs and RNNs.
  • Deployment workflow: Smartphone developers commonly train models on cloud servers before packaging them with apps for runtime inference.Apps invoke deep learning frameworks to execute tasks such as face recognition and language translation.
  • Inference settings: Inference may be offloaded to cloud servers or executed solely on smartphones.Cloud inference leverages data-center resources, while this work studies on-device deep learning.

3 GOAL AND METHODOLOGY

The study characterizes Android apps, frameworks, and models through two Google Play snapshots and a semiautomatic static-analysis tool. Its pipeline identifies deep learning apps, extracts their models, and analyzes model formats and deployment characteristics.

  • Scope: The analysis focuses on Android because Android represented 88% of smartphone shipments in the second quarter of 2018.The paper treats Android as a proxy for the broader smartphone app population.
  • Study design: The study examines top Android apps from Google Play, using 16,500 apps across 33 categories and snapshots from June and September 2018.The two datasets were collected three months apart and overlapped by more than two-thirds.
  • Analyzer pipeline: The semiautomatic analyzer identifies deep learning apps and supports large-scale analysis of their adoption.It uses extracted app packages and metadata from Google Play.
  • Analyzer pipeline: Model Extractor takes identified deep learning apps as input and outputs the model files they use.It validates framework-specific model formats after scanning decomposed app assets.

4 APPLICATION ANALYSIS

Among the most popular Android apps, deep learning is increasingly adopted, usually supports core functionality, and appears across diverse uses, while mobile frameworks and app deployments show measurable resource costs.

  • Characteristics of DL apps: 27.1%: DL apps increased from 166 to 211 between June and September 2018.Among apps present in both snapshots, 48 of 190 newer DL apps were checked in, while only 5 checked out.
  • Characteristics of DL apps: DL apps were substantially more popular than non-DL apps, with median downloads of 5,000,000 versus 100,000 and reviews of 41,074 versus 1,036.Their median category ranking was 89 among 500 apps, while both groups had the same median rating of 4.3.
  • The roles of DL in apps: Image processing dominated smartphone DL usage at 149 apps, compared with 26 for text and 24 for audio.Photo beauty and face detection were the most common detailed uses; recommendation appeared in 11 apps.
  • The roles of DL in apps: 171 out of 211 (81%) apps used DL to support core features.Photo beauty and face detection were treated as essential in photo and camera apps, while recommendation was often complementary.
  • DL frameworks: TensorFlow, TFLite, and ncnn were the most popular frameworks in September 2018, used by 51, 31, and 28 DL apps respectively.Five of the six dominant frameworks were open-source, and four were optimized for smartphones.
  • DL frameworks: DL framework libraries averaged 7.5MB, nearly six times the size of non-DL libraries, while 24 apps embedded multiple DL frameworks.Such multi-framework use added around 5.4MB, or 13.6% of total APK size on average.

5 MODEL ANALYSIS

The study analyzes extracted mobile DL models structurally, by resource footprint, and by security. It finds lightweight models, sparse use of optimizations, and substantial exposure of models without protection.

  • 5 MODEL ANALYSIS: The analysis covers model structures, resource footprints, and security, using extracted models from major mobile DL frameworks.Model Extractor identifies DL models in apps and supports framework-specific model formats.
  • Model structures: 87.7% of extracted models are CNNs and 7.8% are RNNs, with CNNs mainly serving image/video and text-classification tasks.RNNs are mainly used for text and voice processing, including prediction, translation, and speech recognition.
  • Model structures: 87.7% of models include convolutional layers, while pooling and softmax appear in 76.5% and 69.1% of models, respectively.The median and mean numbers of convolutional layers among models containing them are 5 and 14.8.
  • Model structures: 6.32% of models are quantized into 8-bit, while the remaining models are non-optimized.The study focuses on quantization and sparsity across five popular frameworks.
  • Resource footprint: Extracted models are small, with a median storage size of 1.6MB and mean size of 2.5MB; TensorFlow models are larger than TFLite and ncnn models.TensorFlow, TFLite, and ncnn have median sizes of 3.2MB, 0.75MB, and 0.86MB, respectively.
  • Resource footprint: In-the-wild models have median memory usage of 2.47 MB and computational complexity of 10M FLOPs, making mobile execution inexpensive.Figure 8 compares extracted models with classical CNN architectures using memory and computation costs.
  • Security: Among 120 analyzed models, 47 (39.2%) are obfuscated and 23 (19.2%) are encrypted, leaving most models exposed without protection.Encrypted models are also counted as obfuscated, so the two sets overlap.

6 LIMITATIONS AND FUTURE WORK

The study is limited by static-analysis errors, two dated Android-only snapshots, and the absence of dynamic measurements. Future work extends coverage across time and platforms and adds dynamic analysis.

  • Analyzer limitations: The analyzer can miss apps using unrecognized DL frameworks or native implementations and falsely classify apps that integrate but do not invoke DL.Planned remedies include code-pattern mining and more advanced static analysis of library API calls.
  • Future work: The empirical datasets contain only two snapshots from June and September 2018, limiting the study’s temporal coverage.The authors propose updating the study at additional time steps, such as every three months.
  • Future work: The study analyzes Android apps only, leaving iOS and Android Wear for future comparison.The authors expect cross-platform comparisons to provide additional implications for researchers and developers.
  • Future work: The current analysis is static and therefore does not directly characterize runtime measures such as end-to-end latency and energy consumption.The authors propose running models on off-the-shelf mobile devices through dynamic analysis.

7 RELATED WORK

Prior mobile DL research develops applications and reduces deployment overhead, but is usually conducted in laboratory settings using classical models. This study addresses that gap through large-scale analysis of real-world apps and models.

  • Mobile DL: Prior mobile DL work spans novel applications and optimizations including compression, hardware customization, lightweight models, distillation, and cloud offloading.These efforts target the overhead of DL on resource-constrained mobile devices.
  • Mobile DL: Existing optimization studies commonly use laboratory environments and classical models such as VGG and ResNet rather than real-world workloads.The paper positions its analysis as a bridge between academic literature and industry products.
  • ML/DL as cloud services: Cloud ML services provide another deployment mode, with prior work studying their effectiveness and the relationship between user control and risk.Examples include services from Amazon, Google, and Microsoft Azure.
  • Empirical study of DL: Prior empirical DL studies often assist development by analyzing program defects, network topology, geometry, and decision boundaries.The cited defect analysis is limited to open-source, small-scale projects.
  • DL model protection: Model-protection research includes watermarking, secure execution environments, and framework mechanisms, but prior techniques remain impractical or insufficient for in-the-wild deployment.The paper reports that many deployed models remain exposed without secure protection.

8 CONCLUSIONS

The paper presents the first large-scale empirical study of DL adoption in real-world smartphones, using a static tool to analyze Android apps. It finds that early adopters are top apps and DL often serves critical core functions, with implications for mobile ecosystem stakeholders.

  • 8 CONCLUSIONS: The study mines and analyzes large-scale Android apps with a static tool to understand real-world DL adoption.Its stated purpose is to bridge DL research and practice.
  • 8 CONCLUSIONS: Early adopters of mobile DL are top apps, and DL plays a critical, core role in those apps.The findings provide implications for developers, hardware designers, and researchers.
Loading 1812.05448v4…