Large Language Models Empowered Autonomous Edge AI for Connected Intelligence

Yifei Shen; Jiawei Shao; Xinjie Zhang; Zehong Lin; Hao Pan; Dongsheng Li; Jun Zhang; Khaled B. Letaief

Source-linked AI summary

Large Language Models Empowered Autonomous Edge AI for Connected Intelligence

Yifei Shen, Jiawei Shao, Xinjie Zhang, Zehong Lin, Hao Pan, Dongsheng Li, Jun Zhang, Khaled B. Letaief

arXiv:2307.02779v3Jul 6, 2023cs.ITcs.LGcs.NIeess.SP

TL;DR

Edge AI still requires human effort to coordinate heterogeneous models and customize privacy-preserving training for users’ diverse needs. The paper proposes a GPT-enabled cloud-edge-client framework that orchestrates existing models and generates federated-learning code. Experiments report accurate request understanding, low-delay execution, and effective creation of high-performance models.

Problem
Existing edge AI systems do not fully automate personalized coordination across sensors and models or privacy-preserving customization, leaving integration and code implementation to human experts.
Method
A cloud-edge-client framework uses GPT for natural-language understanding, planning, model coordination, and code generation alongside task-oriented communication and federated learning.
Results
The proof-of-concept system accurately comprehended user demands, executed AI models with minimal delay, and created high-performance AI models at edge servers.
Takeaways & Limitations
GPT can serve as a controller for autonomous edge AI that organizes existing models and supports privacy-preserving creation of new models.

Abstract

from arXiv · show

The evolution of wireless networks gravitates towards connected intelligence, a concept that envisions seamless interconnectivity among humans, objects, and intelligence in a hyper-connected cyber-physical world. Edge artificial intelligence (Edge AI) is a promising solution to achieve connected intelligence by delivering high-quality, low-latency, and privacy-preserving AI services at the network edge. This article presents a vision of autonomous edge AI systems that automatically organize, adapt, and optimize themselves to meet users' diverse requirements, leveraging the power of large language models (LLMs), i.e., Generative Pretrained Transformer (GPT). By exploiting the powerful abilities of GPT in language understanding, planning, and code generation, as well as incorporating classic wisdom such as task-oriented communication and edge federated learning, we present a versatile framework that efficiently coordinates edge AI models to cater to users' personal demands while automatically generating code to train new models in a privacy-preserving manner. Experimental results demonstrate the system's remarkable ability to accurately comprehend user demands, efficiently execute AI models with minimal cost, and effectively create high-performance AI models at edge servers.

I. INTRODUCTION

The paper argues that edge AI has not yet achieved connected intelligence because coordinating heterogeneous models and customizing privacy-preserving models still depends on human experts. It proposes GPT-empowered autonomous edge AI to understand requests, coordinate existing models, and generate training code.

Motivation: Existing edge AI components are often designed separately, making seamless integration for connected intelligence difficult.The system must coordinate sensors and models from different vendors while serving personalized user demands.
Motivation: Users may need customized AI models, but experts traditionally interpret manuals, integrate sensors, parse requests, and write training code.Privacy must also be preserved during customized model training.
Approach: GPT processes natural-language requests and routes them to suitable edge AI models for personalized services.The framework uses task-oriented inference to support autonomous, low-latency collaboration.
Approach: GPT generates configurations and code from a template to automate privacy-preserving federated learning.The method identifies key configurations to modify before generating detailed configurations and corresponding code.
Contributions: A cloud-edge-client framework combines GPT’s language and code-generation abilities with edge AI methods for lower latency and higher reliability.The proof-of-concept system reports accurate demand comprehension, minimal-delay model execution, and effective creation of high-performance models.

II. AUTONOMOUS EDGE AI: CHALLENGES AND OPPORTUNITIES

The paper identifies user-interface, sensor-model integration, and customized-training challenges in edge AI. Autonomous edge AI addresses them through self-organization and self-improvement, with potential applications including healthcare, driving, and industrial IoT.

Challenges: Future edge deployments will contain many sensors and AI models, creating challenges for users, vendors, and developers.The paper frames these challenges as a transition from conventional edge AI toward autonomous edge AI.
Challenges: Natural-language dispatchers could route user requests to appropriate applications while considering preferences and context.This addresses the need for users to learn each application’s functionality.
Challenges: Coordinating sensors and AI models from multiple providers traditionally requires substantial interdisciplinary human effort.Experts must integrate heterogeneous capabilities and write application-specific code.
Autonomous edge AI: Autonomous edge AI combines self-organization for request-driven orchestration with self-improvement for privacy-preserving model training.Self-organization coordinates existing sensors and models, while self-improvement fine-tunes or creates models through generated code.
Opportunities: The proposed concept could support patient monitoring, autonomous driving, and industrial IoT by integrating or adapting models for application-specific needs.The paper describes real-time vital-sign analysis, coordinated driving models, and generated code for industrial models.

B. GPT as a Potential Solution

The paper presents GPT as a controller for autonomous edge AI because it can understand language, plan tasks, learn from context, and generate code. These capabilities support interpreting user intent, dispatching subtasks, incorporating device knowledge, and adapting models.

GPT capabilities: GPT’s language understanding can interpret user intentions and recognize edge AI model specialties from manuals.This supports matching requests with model capabilities.
GPT capabilities: GPT’s planning ability decomposes ambiguous requests into solvable subtasks and dispatches them to edge AI models.The paper treats planning as a mechanism for coordinating model execution.
GPT capabilities: GPT’s in-context learning lets the system supply device and model information without fine-tuning GPT.The supplied knowledge can describe devices and edge AI models absent from GPT’s training data.
GPT capabilities: GPT’s code generation can refine existing AI models and create new ones for autonomous edge AI.This capability supports automated model customization and training.

III. CLOUD-EDGE-CLIENT HIERARCHICAL FRAMEWORK FOR AUTONOMOUS EDGE AI

The proposed autonomous edge AI framework uses a cloud-edge-client hierarchy. Cloud servers provide substantial resources for GPT, edge servers offer nearby computation, and clients contain users’ devices and IoT sensors.

Hierarchical framework: The hierarchy assigns GPT support to cloud servers, enhanced computation to nearby edge servers, and sensing and user interaction to resource-limited clients.The framework is designed to distribute autonomous edge AI functions across these three layers.

A. Edge AI Model Coordination

The framework uses GPT to dispatch natural-language user requests to suitable edge AI models and applies task-oriented inference to complete them efficiently.

A. Edge AI Model Coordination: Task-oriented inference is adopted to reduce inference latency when executing coordinated edge AI models.
A. Edge AI Model Coordination: GPT dispatches natural-language user requests to specialized edge AI models by understanding intentions and decomposing requests into solvable tasks.The model pool describes each AI model, its associated equipment, and its function.
A. Edge AI Model Coordination: The coordination process selects models from a vendor-provided pool and consolidates their outputs to obtain final results.

2) AI Model Offloading:

The framework uses client-edge co-inference and task-oriented compression to execute models with lower communication overhead while preserving inference performance.

2) AI Model Offloading:: Centralized execution can create excessive communication overhead when radio resources are limited or channels fluctuate dynamically.
2) AI Model Offloading:: Client-edge co-inference partitions a deep neural network between a lightweight device model and a larger edge-server model.The partition point divides the model so sensors perform part of the computation before transmitting intermediate features.
2) AI Model Offloading:: Task-oriented compression filters redundancy and encodes only task-relevant information into bitstreams for server-side inference.
2) AI Model Offloading:: The compression objective combines inference loss with communication cost, reducing communication overhead while preserving inference performance.

B. Automatic Edge Federated Learning

Automatic edge federated learning uses GPT-guided configuration and template-code modification to create customized models while keeping private training data distributed.

B. Automatic Edge Federated Learning: Federated learning keeps training data on individual devices while communicating a shared model across distributed devices to enhance privacy and security.
B. Automatic Edge Federated Learning: A predefined federated-learning template specifies configurable model architectures, optimizers, learning-rate schedules, and data-augmentation methods.
B. Automatic Edge Federated Learning: GPT receives dataset profiles and user-requested model purposes, suggests configurations, and modifies the corresponding template-code sections.
B. Automatic Edge Federated Learning: Repeating the process for a few trials lets GPT use learning curves to refine key configurations, with improvements observed across iterations.The paper gives 3 trials as an example and excludes architecture from later configuration updates.

IV. EXPERIMENTS OF AUTONOMOUS EDGE AI

The experiments evaluate autonomous edge AI capabilities, including model coordination and execution performance.

IV. EXPERIMENTS OF AUTONOMOUS EDGE AI: The evaluation section is designed to demonstrate the capabilities of autonomous edge AI.

A. Edge AI Model Coordination

The framework coordinates edge AI models from natural-language requests and executes selected models across a cloud-edge-client system. Experiments evaluate model selection, communication cost, performance, and latency, showing low-latency execution across diverse requests.

Experimental setting: The evaluation measures model-selection accuracy and system latency in a cloud-edge-client setup using GPT for request understanding and expert models deployed at the edge and client.The test system processes daily photos using ViT for classification and BLIP for image captioning and VQA.
Model coordination: 82 user requests test GPT-3 variants and zero-shot classification, with smaller models and zero-shot classification failing to achieve satisfactory accuracy.The evaluated GPT-3 models contain 350M, 6.7B, or 175B parameters; GPT-4 is excluded because its latency is 5s-20s.
Inference execution: The client-edge co-inference scheme achieves comparable performance to lossless edge-only inference while significantly reducing communication cost.Lossy compression reduces communication overhead but degrades performance, whereas client-only inference can incur higher computation latency on limited devices.
Latency: Cloud-only inference has much higher latency, while client-edge co-inference remains consistently lower than edge-only and client-only inference across communication rates.Edge-only inference is comparable to or lower than client-only inference at 300 KB/s to 500 KB/s but becomes much slower as the rate decreases.
Conclusion: The experiments demonstrate that the framework understands user requests and executes edge AI models for diverse requests with low latency.The latency comparison includes transmission time, computation time, and downloading time.

B. Automatic Federated Learning

The framework uses GPT-4 to schedule code generation for federated learning across edge devices. Iterative trials modify model, augmentation, optimizer, and learning-rate choices to improve training performance while preserving privacy.

Experimental setting: The automatic federated learning experiment uses federated averaging with GPT-4 as scheduler across 10 Jetson Nano clients holding IID CIFAR10 partitions.The setup uses 250 KB/s uplink, 500 KB/s downlink, local epoch 10, and global epoch 30.
User request: Users ask the system to select and adapt a torchvision model for CIFAR10 while configuring the classifier, learning rate, data augmentation, and scheduler.The request specifies training configuration rather than fixed model code.
Iterative improvement: GPT-generated training code iteratively changes the model and optimization strategy, with a learning-rate scheduler producing better performance than the baseline by the third trial.The first MobileNetV2 trial underperforms the ResNet18 baseline, while the second trial oscillates because its learning rate is too large.
Conclusion: The experiment demonstrates that GPT can generate federated-learning training code and iteratively improve performance automatically while preserving users’ privacy.This result supports the paper’s vision of an edge AI system that can improve itself without exposing users’ raw data.

V. CONCLUSIONS AND FUTURE WORKS

The framework demonstrates autonomous edge AI using LLMs to orchestrate existing models and train new ones at the network edge. Future work includes addressing energy efficiency and privacy, extending GPT-based algorithm selection, and exploring other Generative AI models.

The cloud-edge-client framework orchestrates existing AI models and trains new ones at the network edge, with proof-of-concept experiments validating its effectiveness.The system leverages large language models within a hierarchical autonomous edge AI architecture.
The initial framework prioritizes latency, leaving energy efficiency and privacy as important areas for sustainable edge AI improvements.Suggested directions include incorporating energy consumption into each stage and adding privacy techniques such as information bottlenecks and adversarial training.
GPT could be extended beyond AI-model selection to select resource-allocation algorithms tailored to network environments, potentially reducing latency and improving spectral efficiency.
Other Generative AI models could improve task-oriented communication compression and generate federated-learning datasets with better privacy protection.