PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Distillation based Indoor Localization for IoT Networks

Yaya Etiabi, Marwa Chafii, El Mehdi Amhoud

arXiv:2205.11440v2eess.SPcs.LGcs.NI

TL;DR

IoT localization needs to reduce federated training communication, while existing FD methods focus on classification rather than regression. This paper develops regression-capable FD for indoor localization and reports lower communication load than FL, alongside scalability as a practical advantage.

  • Problem

    Existing FD frameworks target classification rather than regression, while IoT localization faces bandwidth constraints and privacy-sensitive data exchange.

    Source: §I · pp. 1–2, B. FL based IoT localization frameworks · p. 3

  • Method

    The paper develops an FD framework for regression and applies it to an IoT indoor localization system.

    Source: §I · p. 2

  • Results

    The framework reduces communication load by ∼98% compared with FL-based frameworks.

    Source: §V · p. 13

  • Takeaways & Limitations

    The authors report that the framework’s scalability makes it suitable for large IoT network applications.

    Source: §V · p. 13

    The framework remains limited by segmentation impairments and device heterogeneity in computation and communication capabilities.

    Source: §V · p. 13

Abstract

from arXiv · show

Federated distillation (FD) paradigm has been recently proposed as a promising alternative to federated learning (FL) especially in wireless sensor networks with limited communication resources. However, all state-of-the art FD algorithms are designed for only classification tasks and less attention has been given to regression tasks. In this work, we propose an FD framework that properly operates on regression learning problems. Afterwards, we present a use-case implementation by proposing an indoor localization system that shows a good trade-off communication load vs. accuracy compared to federated learning (FL) based indoor localization. With our proposed framework, we reduce the number of transmitted bits by up to 98%. Moreover, we show that the proposed framework is much more scalable than FL, thus more likely to cope with the expansion of wireless networks.

I. INTRODUCTION

Indoor localization must contend with unreliable RSSI measurements and costly, privacy-sensitive data exchange, while existing federated distillation methods target classification rather than regression. The paper motivates and proposes regression-capable FD for IoT localization, emphasizing lower communication demands than FL.

A. RSSI fingerprinting based localization

II. RELATED WORK

  • Machine learning offers data-based localization without empirical channel models, while RSSI fingerprinting uses recorded signal patterns to predict locations.The paper situates its proposal among data-driven and federated approaches to localization.Source: §I · pp. 1–2, A. RSSI fingerprinting based… · p. 2

I. INTRODUCTION

  • Existing FD frameworks target classification, leaving regression tasks such as localization without an established FD approach.The paper proposes a regression FD framework and an IoT localization use case.Source: §I · p. 2

B. FL based IoT localization frameworks

C. Federated distillation

The paper adapts federated distillation to regression-based IoT localization by sharing segment-level prediction knowledge rather than raw RSSI data. Its system trains local models and aggregates their knowledge so clients can make nearly consistent location predictions.

A. Localization system model

  • Localization clients train on locally collected RSSI data and exchange model outputs, enabling their models to predict nearly the same positions from new measurements.A node sends RSSI measurements to a nearby FD client, which returns its estimated position.Source: A. Localization system model · p. 3

B. Federated distillation system architecture

2) Clients:

  • Clients periodically upload local average estimations per target segment; the server averages them and returns global estimates for client loss updates.Clients train with private datasets, while the communication system carries models and target estimates between clients and server.Source: 2) Clients · p. 3, 3) Communications system · p. 3

C. Problem formulation

  • The localization DNN maps RSSI measurements from access points to the two-dimensional coordinates where those measurements were recorded.The considered network uses a three-layer neural network with input, hidden, and output layers.Source: C. Problem formulation · p. 4
  • Regression FD partitions continuous target values into segments and shares clients’ average predictions for each segment instead of exchanging a dataset-sized vector.The approach assumes known or settable target boundaries; segment resolution determines the number of segments.Source: C. Problem formulation · p. 5
  • The server’s target-boundary choices can affect privacy because deriving global minimum and maximum values requires knowledge of local dataset boundaries.The paper suggests secure multiparty computation or order-preserving encryption for requesting local bounds, or having the server set bounds arbitrarily.Source: C. Problem formulation · p. 5

D. FD for regression algorithm

The regression FD algorithm first trains local models, then iterates through communication rounds in which clients share segment-level estimates and update their losses using server aggregates.

D. FD for regression algorithm

  • The teacher knowledge for each student is the aggregated knowledge of the remaining students, so students and teacher learn during the process.Unlike standard knowledge distillation, this configuration has no pretrained teacher model.Source: D. FD for regression algorithm · p. 5
  • Clients begin by training local models because no teacher predictions are available to form the distillation regularizer at the initial communication round.The paper describes a short local-training phase before distillation begins.Source: D. FD for regression algorithm · p. 5
  • Each client trains on private data, uploads average estimates per segment, receives server-averaged estimates, and updates its loss until convergence.The server computes a global average for each segment from the clients’ local averages.Source: D. FD for regression algorithm · p. 5

E. Convergence analysis

The paper formulates the regression FD objective using mean square error and relates its convergence analysis to knowledge distillation and co-distillation through neural tangent kernel theory.

E. Convergence analysis

  • The global optimization problem uses mean square error after applying the target-segmentation technique.The paper writes the loss as MSE(y, ŷ) in the global objective.Source: E. Convergence analysis · p. 5
  • The paper states that convergence can be established through asymptotic analyses of knowledge distillation and co-distillation using neural tangent kernel theory.These are identified as foundational building blocks of federated distillation.Source: E. Convergence analysis · p. 5

7 ClientLocalTraining (Dk, Wk, {Lk

The algorithm excerpts show per-client loops for local training and server knowledge distillation.

7 ClientLocalTraining (Dk, Wk, {Lk

  • ClientLocalTraining includes a loop over clients.The excerpt supplies only the loop header.Source: §7 · p. 6

23 ServerknowledgeDistillation ({¯ℓk

  • ServerknowledgeDistillation includes a loop over clients.The excerpts supply loop headers for this stage but no further procedure details.Source: §23 · p. 6

1) Convergence analysis of Knowledge distillation:

The analysis derives asymptotic behavior for knowledge distillation under stated neural-network assumptions and relates student prediction error to teacher quality.

1) Convergence analysis of Knowledge distillation:

  • The analysis translates gradient-descent weight dynamics into continuous-time dynamics of the logits to study convergence.It assumes Lipschitz-continuous, differentiable hidden-layer activations with Lipschitz-continuous first derivatives.Source: 1) Convergence analysis of Knowledge… · pp. 6–7
  • The student network output converges asymptotically under mild initialization-eigenvalue assumptions, bounded inputs, and bounded weights.The analysis also discusses the infinite-width regime.Source: 1) Convergence analysis of Knowledge… · p. 7
  • For an imperfect teacher, the student’s prediction error increases monotonically with the regularization coefficient λ.The passage links student learning accuracy to teacher quality.Source: 1) Convergence analysis of Knowledge… · p. 7

2) Federated distillation:

In federated distillation, each student uses the ensemble predictions of the other students as teacher knowledge. The analysis establishes asymptotic convergence of student outputs to the labels.

2) Federated distillation:

  • Each student minimizes a local loss using the ensemble prediction of the remaining students as its teacher.The students possess their own data, making the distillation online.Source: 2) Federated distillation · p. 7
  • As communication rounds increase, each student’s output converges asymptotically to y.The result is stated for every client k.Source: 2) Federated distillation · p. 7
  • Numerical simulations and experimental data are reported to confirm the FD system’s convergence property.The passage points to Section IV for those evaluations.Source: 2) Federated distillation · p. 8

1) Energy consumption:

The energy analysis focuses on transmission energy and compares FD’s prediction exchange with FL’s model-parameter exchange, assuming equal communication rounds.

1) Energy consumption:

  • FD devices transmit predictions rather than model parameters, reducing the communication cost relative to exchanging model parameters.The passage says this design addresses model-size-related energy consumption, while slightly increasing the local-computation issue because of the regularization term.Source: 1) Energy consumption · p. 8
  • Assuming the same number of communication rounds for FL and FD, the analysis focuses on transmission energy as the dominant part of overall system energy consumption.The uplink is modeled as a shared Gaussian multiple-access channel, while downlink communication is assumed noiseless.Source: 1) Energy consumption · pp. 8–9
  • S × N_o is much smaller than W, making the FD model more communication-efficient than FL.W denotes the considered neural network’s total number of parameters.Source: 1) Energy consumption · p. 9
  • The authors report that the FD constraint does not affect the system too much because Nbits < Bmax is likely to hold, as shown by Section IV results.The constraint is discussed in the context of each device’s transmission capacity.Source: 1) Energy consumption · p. 9

IV. PERFORMANCE EVALUATION

The evaluation models RSSI-based indoor positioning and describes how simulated RSSI data are generated. Data collection pairs access-point measurements with reference-position labels and repeats measurements over time to capture channel variation.

1) System model and data collection:

  • The localization system predicts an IoT device’s position from its RSSI measurements using data crowdsourced by FD clients.The network has M randomly distributed access points, and each of C clients collects RSSI data to train a local model.Source: 1) System model and data collection · p. 9
  • Each RSSI recording from an access point is labeled with its reference position, with measurements repeated across k time intervals.The system considers M access points and N reference positions to capture channel variations and impairments.Source: 1) System model and data collection · p. 9

2) Simulated RSSI data:

  • Simulated RSSI measurements follow a log-distance path-loss model, with shadowing represented by Xσ and reference-distance loss derived from free-space propagation.The path-loss exponent is β, and the free-space reference loss depends on signal frequency f and the speed of light c.Source: 2) Simulated RSSI data · p. 9

B. Simulation Results

The evaluation compares FD with FL, centralized learning, and standalone learning across simulated and experimental regression tasks. FD substantially reduces communication, while its localization accuracy is slightly better than standalone learning but lower than FL and centralized learning in the reported comparisons.

1) Set-up:

  • The simulation uses a 20 × 20 m² area, 10 access points, 100 reference positions, and five FD clients.The path-loss exponent and shadowing parameter are set to 3.23 and 2, respectively, and the number of repetitions is 10.Source: 1) Set-up · p. 9

2) Results:

  • In simulation, FD improves localization accuracy over standalone learning but remains less accurate than FL and centralized learning.The student models converge to the same MAE, and the evaluation also reports estimation errors and network-level RMSE.Source: 2) Results · p. 10, Figure 3 · p. 10
  • 99.85% of FL transmission energy is saved in simulation, while FD is 1.6× less accurate than FL.FD transmits 3,200 bits per round versus 2,080,320 bits for FL under the stated settings.Source: 2) Results · p. 10

C. Experimental validation and benchmark

D. Scaling Capability

Scaling tests show FD sharply reduces transmitted bits relative to FL, while segmentation and regularizer choices shape accuracy. The conclusion identifies segmentation impairments and device heterogeneity as improvement targets.

D. Scaling Capability

  • FD transmits 1.46% of FL’s bits, making it about 98.54% more communication-efficient while remaining about 1× to 2.7× less accurate.As the number of students increases, accuracy improves for both methods, with a stronger effect for FL.Source: D. Scaling Capability · p. 12, Figure 5 · p. 12

E. The impact of segmentation and data distribution

  • Uniform segmentation can leave intervals empty or sparsely populated, so clients may lack some segment labels; density-based segmentation gives each segment the same number of samples but different sizes.The preferred strategy depends on data distribution, and segmentation-induced distribution impairments can make increasing the number of segments degrade accuracy.Source: E. The impact of segmentation and data… · pp. 12–13, Figure 6 · p. 12, Figure 7 · p. 12

F. Impact of the distillation regularizer

  • For UJIIndoorLoc, smaller distillation regularizers yield better accuracy, and the authors recommend λ < 1 for this application.The paper relates this observation to analysis showing student prediction error increases with λ when the teacher model is imperfect.Source: F. Impact of the distillation… · p. 13, Figure 8 · p. 13

V. CONCLUSION

  • The proposed regression FD framework supports indoor IoT localization and reduces communication load by about 98% compared with FL, with scalability suited to large IoT applications.The authors identify segmentation impairments and device heterogeneity in computation and communication capabilities as limitations to address.Source: §V · p. 13

Related papers

Loading 2205.11440v2…