Source-linked AI summary
FedVision: An Online Visual Object Detection Platform Powered by Federated Learning
Yang Liu, Anbu Huang, Yun Luo, He Huang, Youzhi Liu, Yuanyuan Chen, Lican Feng, Tianjian Chen, Han Yu, Qiang Yang
TL;DR
Centralized visual object-detection training is difficult because data sharing raises privacy concerns and transmitting large datasets is costly, while accessible FL tools for non-experts are lacking. FedVision addresses this gap with an end-to-end platform that trains detection models across locally stored datasets. After four months, three corporate customers reported improved operational efficiency, reduced costs, and no need to transmit sensitive data.
Problem
Centralized object-detection training faces privacy, data-sharing, transmission-cost, and update-timing challenges, while easy-to-use FL tools for non-expert computer-vision developers are lacking.
Method
FedVision is an end-to-end machine-learning engineering platform that supports joint object-detection training across clients with locally stored datasets using federated learning.
Results
Three corporate customers used FedVision for smart-city safety-monitoring applications over four months, improving operational efficiency, reducing costs, and eliminating sensitive-data transmission.
Takeaways & Limitations
FedVision demonstrates an industry deployment of federated learning for computer-vision safety applications while protecting data privacy.
Takeaways & Limitations
The current platform supports the FedYOLOv3 algorithm, with broader FL algorithms, explainability, and incentive mechanisms left for subsequent work.
Abstract
from arXiv · showhide
Visual object detection is a computer vision-based artificial intelligence (AI) technique which has many practical applications (e.g., fire hazard monitoring). However, due to privacy concerns and the high cost of transmitting video data, it is highly challenging to build object detection models on centrally stored large training datasets following the current approach. Federated learning (FL) is a promising approach to resolve this challenge. Nevertheless, there currently lacks an easy to use tool to enable computer vision application developers who are not experts in federated learning to conveniently leverage this technology and apply it in their systems. In this paper, we report FedVision - a machine learning engineering platform to support the development of federated learning powered computer vision applications. The platform has been deployed through a collaboration between WeBank and Extreme Vision to help customers develop computer vision-based safety monitoring solutions in smart city applications. Over four months of usage, it has achieved significant efficiency improvement and cost reduction while removing the need to transmit sensitive data for three major corporate customers. To the best of our knowledge, this is the first real application of FL in computer vision-based tasks.
Introduction
Centralized object-detection training requires sharing large datasets, creating privacy, communication, and update-timing challenges. FedVision applies federated learning to let non-experts develop computer-vision applications without transmitting sensitive training data.
- Centralized training requires sharing data across organizations, despite liability concerns and increasingly strict regulations such as GDPR.
- Large datasets make uploading training data costly, while offline retraining delays updated models and correction of inference errors.
- Federated learning trains models on distributed datasets, aggregates model parameters centrally, and keeps the actual training data at local sites.
- FedVision provides an easy-to-use platform for non-experts, supporting end-to-end joint training with locally stored datasets from multiple clients.Its simplified task-creation interface does not require users to understand federated learning.
- After four months, deployment with three corporate customers improved operational efficiency, reduced costs, and eliminated the need to transmit sensitive data.
- The authors identify FedVision as the first industry application of federated learning in computer-vision tasks.
Application Description
FedVision combines crowdsourced local annotation, federated model training, and federated model updates for object-detection applications. Its design uses horizontal federated learning so parties with the same feature space can jointly train models without centralizing their data.
- Workflow: FedVision’s workflow has three steps: crowdsourced image annotation, federated model training, and federated model update.
- Crowdsourced Image Annotation: Users annotate local images by drawing bounding boxes and assigning labels, with annotation files automatically mapped to training directories.
- Crowdsourced Image Annotation: Each Darknet annotation row records a category label, bounding-box center coordinates, width, and height.
- Horizontal Federated Learning: Horizontal federated learning applies when parties share a feature space but have different samples, allowing locally trained models to be encrypted, aggregated, and redistributed.
- Federated Model Training: The training module includes configuration, scheduling, task management, resource exploration, a federated server, and federated clients.
- Federated Model Update: FedVision stores growing collections of federated model-parameter files in Cloud Object Storage and provides model aggregation for the federated detector.
Uses of AI Technology
FedVision’s AI Engine uses federated YOLOv3 for efficient visual object detection and compresses transmitted model parameters through neural-network pruning.
- Federated Object Detection Model Training: FedVision implements FedYOLOv3, a federated version of the one-stage YOLOv3 detector chosen to prioritize efficiency over accuracy.With one round of end-to-end training, it identifies object bounding-box positions and classes in images.
- Federated Object Detection Model Training: YOLOv3 divides each image into an S × S grid, with each grid detecting objects whose centers lie within it.For each grid, the model predicts bounding boxes, confidence scores, and class probabilities.
- Federated Object Detection Model Training: The YOLOv3 loss function combines class prediction, bounding-box coordinate, and confidence-score prediction losses.The platform preconfigures the λcoord and λ¬obj hyper-parameters used by the model.
- Federated Model Training: FedVision trains models through encrypted local parameter uploads, federated averaging at the server, and distribution of updated global weights to participating clients.This round-based workflow lets clients use a model trained with the participating users’ datasets.
- Neural Network Compression: At about 15MB/sec bandwidth, uploading a 230MB model parameter matrix takes more than 20 seconds, illustrating the transmission cost FedVision targets.Figure 8 compares upload times for federated model parameters of different sizes.
- Neural Network Compression: Network pruning reduces federated parameter sizes by removing less useful weights while preserving model performance and speeding transmission.FedVision ranks layers by their contribution value v(j) and uploads only the first n layers, with n configurable by the user.
Application Use and Payoff
FedVision was deployed for three corporate customers across safety monitoring, suspicious transaction monitoring, and photovoltaic-panel safety. Over four months, it improved efficiency, reduced costs, and reduced exposure of sensitive data.
- Customer applications: FedVision supported three large-scale corporate customers in developing computer vision monitoring applications.Applications covered safety hazards in more than 100 factories, suspicious transaction behaviours at ATM equipment, and photovoltaic-panel safety across more than 10,000 square meters.
- Efficiency: More than 20 days were removed from CRC’s model-optimization process, saving labor cost.The original process required five labellers for about two weeks and could take up to 30 days including testing and packaging.
- Data privacy: FedVision reduced GRG Banking’s risk of data exposure by training a suspicious-activity model using local ATM storage and computation.The centralized process required 10,000 photos to be collected and downloaded over two to three days, passing through two to three locations.
- Cost: Less than 1 MB/sec of network bandwidth was required for model updates, compared with at least 50 MB/sec for transmitting image data across 100 video channels.This result came from SPIC’s generator-monitoring system.
- Overall payoff: The reported improvements significantly enhanced customer operations and provided competitive business advantages.The platform’s four-month usage period underlies this reported business outcome.
Application Development and Deployment
FedVision combines privacy-preserving learning with an automated user-facing workflow. It evaluates privacy-preserving alternatives and lets users monitor federated training while the platform handles subsequent processes.
- Design choices: FedVision selected federated learning after considering secure multi-party computation and differential privacy for its AI Engine.The selection considered transmission efficiency, the need for a trusted third party, and the privacy-performance requirements of the application.
- User workflow: After local annotation and federation enrollment, FedVision automatically handles the remaining processes and provides a user interface for monitoring training rounds.The production interface was developed for use in China and is presented in Chinese.
Maintenance
FedVision’s modular architecture separates task and personnel concerns from the AI Engine, allowing operational updates without changing the engine. No AI maintenance task had occurred since deployment in May 2019.
- Modular maintenance: The modular architecture supports adding computer-vision tasks, changing personnel access rights, and changing operating parameters without affecting the AI Engine.The design follows separation of concern around tasks and personnel.
- Maintenance outcome: No AI maintenance task had occurred since deployment in May 2019.
Conclusions and Future Work
FedVision is an end-to-end platform for developing federated-learning computer-vision applications across organizations. Its deployment with three corporate customers improved operational efficiency, protected data privacy, and reduced costs, while future work broadens algorithms, explainability, and incentives.
- Conclusions: FedVision addresses visual object-detection development when image data are owned by different organizations.
- Conclusions: The deployed platform supported three corporate customers developing safety-hazard warning solutions in smart-city applications.
- Conclusions: Over four months, customers improved operational efficiency, achieved data privacy protection, and reduced costs significantly.
- Future work: The platform currently supports FedYOLOv3, with planned work on federated transfer learning, model explainability, and incentive mechanisms.These extensions target more complex learning tasks, user trust, and a sustainable federated-learning ecosystem.