Source-linked AI summary
A Hybrid Deep Learning Architecture for Privacy-Preserving Mobile Analytics
Seyed Ali Osia, Ali Shahin Shamsabadi, Sina Sajadmanesh, Ali Taheri, Kleomenis Katevas, Hamid R. Rabiee, Nicholas D. Lane, Hamed Haddadi
TL;DR
Cloud-based IoT analytics can expose personal data to unwanted inference, while fully local or encrypted processing may be resource-intensive. The paper splits complex DNNs between device and cloud and uses Siamese fine-tuning to create task-specific exclusive features; evaluations report reduced unnecessary sensitive information at small processing cost, with a privacy-utility-performance trade-off.
Problem
Cloud-based IoT analytics can expose collected personal data to unwanted secondary inference, while complete local processing and encryption have resource limitations or user-experience penalties.
Method
The framework separates a pre-trained DNN into a device-side feature extractor and cloud-side classifier, using Siamese fine-tuning to produce an exclusive feature for the main task.
Results
The evaluations report that Siamese fine-tuning greatly reduces unnecessary potentially sensitive information at a small processing cost while maintaining an acceptable utility-privacy-performance trade-off.
Takeaways & Limitations
Privacy-preserving mobile analytics can collaborate across device and cloud without uploading raw data or running the whole deep model on the user device.
Takeaways & Limitations
The framework assumes the service provider releases a feature extractor module that is publicly verifiable in terms of privacy.
Abstract
from arXiv · showhide
Internet of Things (IoT) devices and applications are being deployed in our homes and workplaces. These devices often rely on continuous data collection to feed machine learning models. However, this approach introduces several privacy and efficiency challenges, as the service operator can perform unwanted inferences on the available data. Recently, advances in edge processing have paved the way for more efficient, and private, data processing at the source for simple tasks and lighter models, though they remain a challenge for larger, and more complicated models. In this paper, we present a hybrid approach for breaking down large, complex deep neural networks for cooperative, privacy-preserving analytics. To this end, instead of performing the whole operation on the cloud, we let an IoT device to run the initial layers of the neural network, and then send the output to the cloud to feed the remaining layers and produce the final result. In order to ensure that the user's device contains no extra information except what is necessary for the main task and preventing any secondary inference on the data, we introduce Siamese fine-tuning. We evaluate the privacy benefits of this approach based on the information exposed to the cloud service. We also assess the local inference cost of different layers on a modern handset. Our evaluations show that by using Siamese fine-tuning and at a small processing cost, we can greatly reduce the level of unnecessary, potentially sensitive information in the personal data, and thus achieving the desired trade-off between utility, privacy, and performance.
1 INTRODUCTION
The paper proposes hybrid privacy-preserving analytics that splits a DNN between the user device and cloud, using Siamese-trained features to retain task-relevant information while limiting unwanted inference.
- Motivation: The framework targets the trade-off between resource-intensive on-device analytics and privacy-invasive cloud services.Complete offloading creates privacy risks, while local processing and encryption can impose resource or user-experience costs.
- Hybrid Architecture: Initial DNN layers run on the user device as a feature extractor, while remaining layers stay in the cloud as a classifier.The device uploads an exclusive feature rather than raw personal data for subsequent cloud processing.
- Evaluation: The framework is evaluated on gender classification and activity recognition, with gender and face recognition treated as privacy-sensitive secondary tasks in different settings.The evaluation uses CNN models and assesses privacy using the proposed measure, transfer learning, and deep visualization.
- Privacy Mechanism: Siamese architecture is used to build a feature extractor that preserves privacy at the point of cloud offloading.The approach aims to retain information needed for the primary task while discarding irrelevant information that could support secondary inference.
- Evaluation: The authors propose a new measure to evaluate privacy and verify the feature extractor module.This measure is part of the framework’s evaluation of information exposure to the cloud service.
2 HYBRID USER-CLOUD FRAMEWORK
The hybrid framework uploads a specially crafted exclusive feature instead of raw data, but designing it requires balancing primary-task utility, privacy against secondary inference, and minimal client-side cost.
- Framework Workflow: The user performs minimal client-side processing to extract an exclusive feature and uploads it to the cloud instead of raw data.The cloud then applies the classifier to the uploaded feature and returns the expected result.
- Feature Requirements: The exclusive feature must preserve information relevant to the primary measure while hiding unnecessary information that could reveal sensitive measures.These requirements define the utility-privacy objective of the feature extractor.
- Layer Separation: Layer separation splits a pre-trained DNN at an intermediate layer into a device-side feature extractor and a cloud-side classifier.The intermediate-layer output becomes the feature passed between the two modules.
- Framework Workflow: The service provider supplies the feature extractor, the user computes the exclusive feature, and the cloud processes it to return the result.This workflow supports collaboration without requiring the user to run the complete model locally.
- Design Challenge: Feature-extractor design is challenging because removing information can reduce main-task performance while client-side processing must remain minimal.The framework focuses on feed-forward neural networks such as CNNs and MLPs.
3 DEEP PRIVACY EMBEDDING
The framework splits a pre-trained DNN between the user device and cloud, then applies embedding methods to reduce sensitive information in the shared feature while preserving the primary task.
- Layer Separation: The layer-separation mechanism splits a DNN at an intermediate layer, sending initial layers to the device and retaining the remaining classifier layers in the cloud.The intermediate feature is the output of the pivot layer; higher-layer outputs are more specific to the primary task and contain less irrelevant information.
- Layer Separation: Simple Embedding uses the unchanged initial DNN layers through the intermediate layer, with the exclusive feature equal to the intermediate feature.
- Dimensionality Reduction: PCA reduces the intermediate feature’s dimensionality while attempting to preserve the primary signal and reducing communication overhead.The reduced embedding adds a reduction matrix to the feature extractor and a reconstruction matrix before the classifier; the procedure does not significantly affect primary-task accuracy.
- Siamese Fine-tuning: Siamese fine-tuning maps samples with the same primary class close together and samples with different classes far apart, making sensitive attributes harder to infer.A contrastive loss is added to the primary classification loss as a regularization term, and fine-tuning occurs once in the cloud before layer separation.
- Noise Addition: Noise addition increases uncertainty about unauthorized tasks, but higher variance can reduce primary-task accuracy when points leave the correct-class region.Siamese fine-tuning reduces primary-class intra-class variance and increases inter-class variance, allowing higher-variance noise without a noticeable primary-task performance drop.
- Advanced Embedding: Advanced Embedding combines Siamese fine-tuning, dimensionality reduction, and noise addition before the resulting feature is uploaded for cloud processing.The cloud performs Siamese fine-tuning, while the device applies PCA projection and noise addition to produce the exclusive feature.
4 PRIVACY VERIFICATION
The paper evaluates privacy by testing sensitive information retained in noisy exclusive features, using likelihood rank, transfer learning, and deep visualization. These methods assess sensitive-class inference, task specificity, and reconstructability.
- The framework uses statistical privacy measurement, transfer learning, and deep visualization to evaluate the exclusive feature.These methods can be applied by the user or a third party to assess whether a feature extractor should be trusted.
- Privacy is linked to sensitive information retained in the exclusive feature, but direct information-theoretic estimation is intractable in high-dimensional spaces.The proposed alternative uses sensitive-variable classification and likelihood rank rather than accuracy alone.
- For each noisy feature, the method computes conditional likelihoods for all sensitive classes and ranks the likelihood of the true class.The noisy features are formed by adding noise to extracted features with known sensitive labels.
- The privacy score normalizes the true-class likelihood rank by the total number of classes and averages individual scores for total privacy.The measure is used to construct an accuracy-privacy trade-off.
- Transfer learning copies and freezes the first i layers of a main-task network before training the remaining layers for sensitive-variable inference.Lower sensitive-variable prediction accuracy indicates that the extracted feature is more specific to the main task.
5 EXPERIMENTS
Experiments evaluate privacy-preserving embeddings for gender classification and activity recognition, showing that Siamese fine-tuning improves privacy while preserving main-task accuracy. Mobile results indicate that selecting intermediate layers enables feasible on-device processing with reduced cost.
- Experiment Results: Siamese fine-tuning preserves main-task accuracy while improving privacy compared with alternative embedding methods.The evaluation covers gender classification and activity recognition, with gender or identity treated as sensitive secondary information.
- Transfer Learning: 2.3% face-recognition accuracy is achieved with Siamese embedding at the Conv5-3 intermediate layer for gender classification.Higher layers contain less identity-related information, and Siamese embeddings outperform simple embeddings for privacy across configurations.
- Transfer Learning: Higher intermediate layers provide more privacy, while dimensionality reduction further improves privacy protection.This pattern appears for both gender classification and activity recognition, although higher layers impose greater processing overhead.
- Privacy Measure: Increasing noise variance improves privacy while reducing main-task accuracy, with Siamese fine-tuning allowing slower accuracy degradation.The accuracy-privacy trade-off is evaluated for gender classification and activity recognition using noisy embeddings.
- Visualization: Advanced embedding provides stronger identity privacy than simple embedding because reconstructed images make identities harder to distinguish.Gender remains recognizable in reconstructions, while advanced embedding best distorts identity information.
- Mobile Evaluation: Modern-handset evaluation shows that the framework is feasible and reduces mobile-device cost compared with running the full VGG-16 model on-device.The cost advantage is especially pronounced for the heavier gender-classification network.
6 RELATED WORK
Related work addresses privacy through dataset publishing, model sharing, training privacy, visual transformations, and mobile inference optimization. This paper instead focuses on protecting users’ personal data while they use an already-trained cloud service.
- Learning with privacy: Dataset publishing methods such as randomized noise and k-anonymity are limited for high-dimensional data and vulnerable to attacks.The cited limitations include the curse of dimensionality and unreliable protection under varied attacks.
- Learning with privacy: Differential privacy and federated learning primarily address privacy of training data or shared models rather than users’ data during service use.These methods concern model sharing or distributed training, whereas this paper studies personal data uploaded to an already-trained service.
- Learning with privacy: The proposed method targets unauthorized use of personal data by sending an exclusive task-specific feature instead of raw data to the cloud.The feature is designed to discard irrelevant information, including information that may be sensitive to users.
- Privacy in image analytics: Prior visual-privacy approaches often transform or publish face images but do not protect the privacy of a newly submitted face image against future analyses.The paper distinguishes its service-delivery setting from dataset de-identification work.
- Deep learning on mobile devices: Mobile deep-learning research reduces inference cost through lighter models, accelerators, or GPUs, but efficiency gains can reduce accuracy or increase battery consumption.These trade-offs motivate distributing complex neural-network computation between the handset and cloud.
7 CONCLUSION AND NEXT STEPS
The framework combines on-device feature extraction with cloud classification and Siamese fine-tuning to improve privacy while preserving utility. The authors identify broader task, model, optimization, and training extensions as future work.
- Conclusion: The proposed architecture splits a deep neural network between an on-device feature extractor and a cloud classifier.Siamese fine-tuning produces features suited to the main task but inappropriate for secondary tasks.
- Conclusion: Siamese fine-tuning helps protect privacy by removing sensitive information from extracted features.The approach targets unauthorized secondary inferences while retaining information needed for the primary task.
- Conclusion: Mobile-phone implementation reduced computational complexity on the user side and communication cost between the device and cloud.
- Next Steps: The current framework is limited to supervised classification during inference, with regression, recurrent networks, optimized trade-offs, and privacy-preserving model training left for future work.The authors specifically identify multi-class classification, test-phase inference, and naive hyperparameter balancing as current boundaries.