PaperlayerView exact revision on arXiv

Source-linked AI summary

Can We Use Split Learning on 1D CNN Models for Privacy Preserving Training?

Sharif Abuadbba, Kyuyeon Kim, Minki Kim, Chandra Thapa, Seyit A. Camtepe, Yansong Gao, Hyoungshick Kim, Surya Nepal

arXiv:2003.12365v1cs.CRcs.LGcs.NEstat.ML

TL;DR

The paper asks whether split learning can provide privacy-preserving training for 1D CNNs on sensitive sequential ECG data. It implements and evaluates split 1D CNNs, finding comparable accuracy but substantial privacy leakage; mitigation attempts reduce leakage while significantly harming accuracy. The authors therefore conclude that split learning alone is insufficient for protecting raw sequential data in this setting.

  • Problem

    The paper examines whether split learning can be applied to sequential ECG data and protect privacy when training 1D CNN models.

  • Method

    The authors implement split learning for 1D CNN ECG classification and assess privacy leakage using visual invertibility, distance correlation, and Dynamic Time Warping.

  • Results

    98.9% test accuracy is achieved by the two-layer 1D CNN, matching the three-layer model and supporting split learning’s applicability without classification-accuracy degradation.

  • Takeaways & Limitations

    Directly applying split learning to 1D CNNs can exhibit high privacy leakage, while adding client-side layers or differential privacy reduces leakage but significantly affects accuracy.

  • Takeaways & Limitations

    The leakage analysis covers one sensitive health application, ECG signals from the MIT-BIH dataset; other 1D CNN applications and datasets remain to be investigated.

Abstract

from arXiv · show

A new collaborative learning, called split learning, was recently introduced, aiming to protect user data privacy without revealing raw input data to a server. It collaboratively runs a deep neural network model where the model is split into two parts, one for the client and the other for the server. Therefore, the server has no direct access to raw data processed at the client. Until now, the split learning is believed to be a promising approach to protect the client's raw data; for example, the client's data was protected in healthcare image applications using 2D convolutional neural network (CNN) models. However, it is still unclear whether the split learning can be applied to other deep learning models, in particular, 1D CNN. In this paper, we examine whether split learning can be used to perform privacy-preserving training for 1D CNN models. To answer this, we first design and implement an 1D CNN model under split learning and validate its efficacy in detecting heart abnormalities using medical ECG data. We observed that the 1D CNN model under split learning can achieve the same accuracy of 98.9\% like the original (non-split) model. However, our evaluation demonstrates that split learning may fail to protect the raw data privacy on 1D CNN models. To address the observed privacy leakage in split learning, we adopt two privacy leakage mitigation techniques: 1) adding more hidden layers to the client side and 2) applying differential privacy. Although those mitigation techniques are helpful in reducing privacy leakage, they have a significant impact on model accuracy. Hence, based on those results, we conclude that split learning alone would not be sufficient to maintain the confidentiality of raw sequential data in 1D CNN models.

I. INTRODUCTION

The paper investigates whether split learning can support privacy-preserving training for 1D CNNs on sequential ECG data. It finds that split learning preserves classification accuracy but can leave substantial privacy leakage, while proposed mitigations reduce leakage at an accuracy cost.

  • Motivation: Split learning divides neural-network computation between client and server so raw data need not be shared directly.The client runs initial layers and sends activated outputs from the split layer to the server.
  • Research Questions: The study uses personal ECG signals because they can reveal disease status and uniquely identify individuals, making raw-data privacy important.
  • Research Questions: The authors investigate whether split learning applies to sequential data and whether it protects privacy when training 1D CNNs.
  • Findings: Split learning on 1D CNNs achieves the same classification accuracy as the non-split model for ECG-based heart-abnormality detection.The contribution reports comparable accuracy between split and non-split models.
  • Findings: The proposed privacy assessment uses visual invertibility, distance correlation, and Dynamic Time Warping to evaluate leakage from split-layer activations.The authors report high privacy leakage when split learning is directly applied to 1D CNNs.
  • Motivation: 1D CNNs use vector-shaped convolution filters to extract features from sequential or time-series data, unlike the matrix-shaped filters used by 2D CNNs.

B. Split Learning

Split learning partitions a CNN between a client and a server, which collaboratively train without accessing each other’s model part. In 1D CNNs, the convolution filter is vector-shaped rather than matrix-shaped.

  • Split Learning: Split learning divides a CNN into client and server parts that collaboratively train without accessing each other’s model part.
  • 1D and 2D CNNs: A 1D CNN uses a vector-shaped convolution filter, whereas a 2D CNN generally uses a matrix-shaped filter.

C. Privacy Preserving Machine Learning

The paper applies split learning to 1D CNN ECG classification and examines whether intermediate activations protect sensitive sequential data. It finds comparable classification accuracy but identifies privacy leakage through activations sent to the server.

  • Experimental design: The implementation evaluates two- and three-convolutional-layer 1D CNN architectures using MIT-BIH ECG data and split points after the convolutional layers.The dataset contains 26,490 samples representing five heartbeat classes.
  • ECG classification: 98.9% test accuracy was achieved by the two-layer non-split model, matching the three-layer model and exceeding the prior 97.5% result.The evaluation used 5-class ECG classification on the MIT-BIH dataset.
  • Privacy-preserving setup: Split learning divides the 1D CNN between client and server, exchanging activations and gradients without sharing raw ECG data.The client holds the initial layers, while the server holds the remaining layers.
  • Implementation contribution: The study provides client- and server-side 1D CNN split-learning implementation strategies and specifies exchanged information for practitioners.The algorithms describe socket-based communication and the activation and gradient flow.

1) Client.:

The client-side procedure processes each ECG batch through its local layers, sends the split-layer activation to the server, and resumes backpropagation after receiving the corresponding gradient.

  • Client procedure: The client connects to the server and synchronizes training configurations before processing batches.The client owns the raw ECG dataset and executes the first part of the split model.
  • Client procedure: For each batch, the client forward propagates through layer l and sends activation a(l) to the server.This is the principal client-to-server data transfer during forward propagation.
  • Client procedure: After receiving ∂E/∂a(l), the client continues backpropagation toward the first hidden layer.The server supplies the gradient needed for the client-side update.

2) Server.:

The server continues the forward and backward passes from the received activation, enabling joint split-model training while exposing an activation that may leak raw data.

  • Server procedure: The server continues forward propagation after receiving the client’s split-layer activation.The server holds the remaining layers from l+1 through L.
  • Server procedure: The server computes the loss using the final activated output and the label sent by the client.The resulting loss initiates server-side backpropagation.
  • Server procedure: The server backpropagates to layer l+1 and sends ∂E/∂a(l) back to the client.This gradient lets the client complete its own backward pass.
  • Training result: The split and non-split models achieve the same classification accuracy in the tested architectures.The authors answer the first research question affirmatively.
  • Privacy result: The privacy assessment finds that split-layer activations can be used to reconstruct raw data, answering the second research question unfavourably.The assessment combines visual invertibility with distance correlation and DTW.

A. Threat Model

The paper models an honest-but-curious server that may reconstruct raw client data from split-layer activations. It assesses this risk using visual inspection, distance correlation, and DTW, finding that activations after two or three convolutional layers can support reconstruction.

  • Threat model: The threat model assumes an honest-but-curious server seeks to reconstruct client raw data from split-layer activations.The server has no access to the client device and does not collude with clients.
  • Visual invertibility: Visual inspection compares raw ECG samples with reconstructed signals from channels after the second convolutional layer.The figure places raw inputs in the top row and reconstructed channel outputs in the bottom row.
  • Similarity metrics: Distance correlation and DTW quantify similarity or dependence between raw ECG inputs and split-layer activation channels.Distance correlation requires matching vector dimensions, so the raw signals are average-downsampled; DTW does not require downsampling.
  • Distance correlation: 0.89 distance correlation after two layers indicates high leakage, while three layers reduce the top-channel value by 0.03 to 0.86.Some channels remain highly dependent after three layers, and the correlation can be exploited to reconstruct raw data.
  • Conclusion: Activations after two and three convolutional layers can be used to reconstruct raw data, answering the privacy question unfavourably.The conclusion is based on visual invertibility, distance correlation, and DTW analyses.

V. MITIGATE THE SHORTCOMING?

The paper evaluates adding client-side convolutional layers and differential privacy to reduce leakage from 1D CNN split-layer activations. Both approaches improve privacy for some channels but fail to protect all channels and reduce model accuracy, especially under strong differential privacy.

  • Mitigation strategies: The evaluation applies two mitigations: adding hidden convolutional layers before splitting and applying differential privacy to transmitted activations.Privacy reduction is measured with distance correlation and DTW, alongside post-mitigation model accuracy.
  • A. Adding More Hidden Layers: 0.89 to 0.69: increasing client-side convolutional layers slightly reduces top-channel distance correlation, but some channels remain above 0.5.The experiment varies the client-side depth from two to eight convolutional layers.
  • A. Adding More Hidden Layers: DTW means for some channels improve from 0 to 600+, while many channels remain near zero and potentially expose raw ECG data.The DTW scale treats 0 as high leakage risk and 600+ as low leakage risk.
  • B. Applying Differential Privacy on Split Layer: ϵ = 1 reduces similarity but degrades classification accuracy from 98.9% to 50%.The strongest tested differential privacy level is expected to protect output channels, but it substantially damages accuracy.
  • A. Adding More Hidden Layers: 84 times versus less than 5 times: added layers substantially improve DTW for least-correlated filters but are ineffective for highly similar filters.The comparison is reported when eight convolutional layers run on the client.
  • Conclusion: Neither mitigation efficiently protects all activation channels, and both incur accuracy reductions that make the privacy question difficult to answer favourably.The paper reports that mitigation effectiveness depends on the technique and convolution filters.

VI. DISCUSSION AND FUTURE WORK

The paper finds that directly applying split learning to 1D CNNs can leak raw sequential data, while mitigation reduces leakage at an accuracy and client-computation cost. It proposes further investigation of channel-selective sharing and notes unresolved applicability to recurrent models.

  • Mitigation trade-offs: Adding client-side convolutional layers and applying differential privacy reduce privacy leakage but significantly reduce model accuracy.The paper identifies both techniques as countermeasures, with differential privacy causing especially substantial accuracy reduction.
  • Mitigation trade-offs: Increasing client-side layers also increases computational overhead and diminishes split learning’s benefit for resource-restricted devices.Split learning reduces client computation by assigning only a small fraction of model parameters to the client; deeper client-side models weaken that advantage.
  • Privacy evaluation: The evaluation framework measures privacy leakage through visual invertibility, distance correlation, and Dynamic Time Warping.The authors propose these three metrics as a general framework for assessing leakage from split-layer activations.
  • Future work: Future work could target only revealing activation channels or share only protected channels through a multiple-horizontal split.The paper reports that added hidden layers protect many, but not all, channels without significant accuracy degradation.
  • Future work: Efficiently applying the current vertical split mechanism to LSTM or RNN models remains unresolved because these networks are sequential.This limitation motivated the authors to use a vertically splittable 1D CNN for time-series data.
  • Scope: The study examines leakage in one sensitive health application using ECG signals from the MIT-BIH dataset, leaving other 1D CNN applications and datasets untested.It also notes that privacy leakage in 2D CNN applications remains to be addressed.

VII. RELATED WORK

Related work shows that split-learning privacy concerns extend beyond the paper’s 1D CNN setting, including reconstruction, inference, and security attacks. Differential privacy affects highly and weakly correlated channels differently, while prior 2D CNN results rely on deeper networks.

  • Differential privacy: The accuracy comparison examines differential privacy across epsilon values from 10, the weakest setting, to 1, the strongest setting.The supplied figure caption defines the direction of privacy strength for the tested epsilon range.
  • Reconstruction leakage: Prior work argued that client-side data would be difficult to reconstruct because the server cannot access split-layer weights, but this paper reports reconstruction is possible for 1D CNNs.The contrast is between an earlier analysis focused on 2D CNNs and the present 1D CNN findings.
  • Comparison with 2D CNNs: A prior 2D CNN study found high split-layer leakage and reduced distance correlation by scaling weights before the split.The scaling approach was effective there because 2D CNNs had many hidden layers before the split layer.
  • Other threats: Split learning may also face membership-inference and backdoor attacks, but the extent of resistance remains future work.The related work connects membership inference to observed gradients and backdoors to maliciously manipulated training data.
  • Differential privacy: Differential privacy reduces distance correlation by 63% for highly correlated channels but by about 23% for lower-correlation channels.The comparison concerns the two most correlated and two least correlated channels identified in Fig. 13.
  • Differential privacy: Differential privacy improves DTW similarity by 158 times for low-similarity channels and by about 113 times for highly correlated channels.The figure compares distributions for the least and most correlated channels.

VIII. CONCLUSION

The paper implements and evaluates split learning for 1D CNN-based ECG classification, finding substantial feature-value privacy leakage and a trade-off between leakage reduction and model-accuracy deterioration.

  • Contribution: The study introduces split learning for 1D CNN models handling sensitive ECG time-series data to detect heart abnormalities.It presents the first implementation described in the paper for this model and application setting.
  • Evaluation: The privacy assessment framework uses visual invertibility, distance correlation, and DTW to evaluate leakage from split-layer features.The framework is applied extensively to an ECG dataset.
  • Findings: Directly applying split learning to 1D CNNs can expose raw sequential data through feature values.The conclusion characterizes the observed leakage as a possibility of high privacy leakage.
  • Findings: Adding CNN layers and applying differential privacy reduce leakage while causing a trade-off with joint-model accuracy, especially under differential privacy.The conclusion describes the accuracy deterioration as substantial for differential privacy.

APPENDIX

The appendix describes the ECG dataset preparation and the client-server training flow for a vertically split 1D CNN, including forward propagation, server-side loss, and returned gradients.

  • Dataset: The MIT-BIH data contain 48 records from 47 patients, with about 110K ECG signals and 30-minute, two-channel excerpts.Each record includes annotations giving time positions and beat types.
  • Dataset: Each heartbeat contains P-wave, QRS-complex, and T-wave components used to characterize cardiac cycles.The P wave reflects atrial depolarization, while the QRS complex and T wave reflect ventricular depolarization and repolarization.
  • Preprocessing: The preprocessing extracts 201-sample beats around R-peaks, applies min-max normalization, downsamples to 128 values, and denoises with a level-3 biorthogonal wavelet.Segments containing another R-peak in the sampled interval are discarded.
  • Classification: The classification targets are five heartbeat types, leaving 96K samples after excluding labels outside those categories.The selected classes are N, L, R, A, and V, based on morphological labels.
  • Training flow: During split training, the client forwards batches through layer l, the server continues propagation, computes loss, and returns the input gradient for client backpropagation.The client and server synchronize training parameters, batch counts, and related settings before training.
  • Training flow: The client-side 1D convolution uses activated outputs from the preceding layer, while the server can continue forward propagation from the split activation.Backpropagation likewise allows the client to continue after receiving the gradient at the split layer.
Loading 2003.12365v1…