PaperlayerView exact revision on arXiv

Source-linked AI summary

Sparks of GPTs in Edge Intelligence for Metaverse: Caching and Inference for Mobile AIGC Services

Minrui Xu, Dusit Niyato, Hongliang Zhang, Jiawen Kang, Zehui Xiong, Shiwen Mao, Zhu Han

arXiv:2304.08782v2cs.NI

TL;DR

Serving PFMs at the edge matters for low-latency, privacy-preserving Metaverse AI, but limited resources and in-context workloads complicate caching and inference. The paper develops joint model caching and inference, introduces Age of Context, and proposes least context management. The reported outcome is reduced system cost and improved edge execution through better use of contextual information.

Abstract

from arXiv · show

Aiming at achieving artificial general intelligence (AGI) for Metaverse, pretrained foundation models (PFMs), e.g., generative pretrained transformers (GPTs), can effectively provide various AI services, such as autonomous driving, digital twins, and AI-generated content (AIGC) for extended reality. With the advantages of low latency and privacy-preserving, serving PFMs of mobile AI services in edge intelligence is a viable solution for caching and executing PFMs on edge servers with limited computing resources and GPU memory. However, PFMs typically consist of billions of parameters that are computation and memory-intensive for edge servers during loading and execution. In this article, we investigate edge PFM serving problems for mobile AIGC services of Metaverse. First, we introduce the fundamentals of PFMs and discuss their characteristic fine-tuning and inference methods in edge intelligence. Then, we propose a novel framework of joint model caching and inference for managing models and allocating resources to satisfy users' requests efficiently. Furthermore, considering the in-context learning ability of PFMs, we propose a new metric to evaluate the freshness and relevance between examples in demonstrations and executing tasks, namely the Age of Context (AoC). Finally, we propose a least context algorithm for managing cached models at edge servers by balancing the tradeoff among latency, energy consumption, and accuracy.

I. INTRODUCTION

PFMs support diverse Metaverse AI services, but their scale and edge-resource limits make caching, inference, and resource management difficult. The article addresses these challenges with joint caching and inference, Age of Context, and a least context algorithm.

  • PFMs enable Metaverse services including autonomous driving, digital twins, and AIGC for extended reality while supporting low-latency, personalized, customized, and privacy-preserving edge AI.Source: §I · p. 1
  • Edge servers cannot simultaneously load all PFMs, while cloud offloading adds networking latency, traffic overhead, and privacy risks and cached outputs are inefficient for realtime AI.Source: §I · p. 1
  • Joint PFM caching and inference must balance accuracy, latency, and energy consumption under dynamic requests, interchangeable models, and heterogeneous model configurations.Source: §I · pp. 1–2
  • Continuous in-context learning improves inference without parameter updates but complicates cache replacement and deployment through a tradeoff among latency, resource consumption, and accuracy.Source: §I · p. 2
  • The proposed framework introduces Age of Context and a least context algorithm to manage cached PFMs and reduce total system cost through improved accuracy, lower offloading latency, and efficient resource use.Source: §I · p. 2

II. SERVING PFMS OF SERVICES IN METAVERSE

PFMs use large-scale pretraining to initialize downstream AI services and include language and visual foundation models. These models support varied modalities but can require substantial computation during inference.

A. Fundamentals of Pretrained Foundation Models

1) Language Foundation Models:

  • LFMs process and generate human language and support tasks including sentiment analysis, translation, summarization, question answering, and text generation.Source: 1) Language Foundation Models · p. 2

2) Visual Foundation Models:

  • VFMs process visual information to understand and generate complex images and videos, but their inference is computing-intensive.Source: 2) Visual Foundation Models · p. 2
  • Fig. 2 compares zero-, one-, and few-shot accuracy under different model caching settings [3].Source: Figure 2 · p. 3

3) Multimodal Foundation Models:

MFMs jointly process text, images, and audio, while parameter-efficient fine-tuning adapts PFMs to downstream tasks with fewer tunable parameters and lower resource consumption.

3) Multimodal Foundation Models:

B. Fine-Tuning of Pretrained Foundation Models

C. Inference of Pretrained Foundation Models

PFM inference makes predictions without parameter updates, with zero-, one-, and few-shot contexts trading adaptation benefits against computation and latency. The proposed framework dynamically caches models using runtime and performance characteristics.

C. Inference of Pretrained Foundation Models

  • Few-shot or in-context learning adapts PFMs during inference without changing parameters, improving performance from demonstrations but increasing computation and latency with context size.Source: C. Inference of Pretrained Foundation… · p. 3

III. JOINT MODEL CACHING AND INFERENCE FRAMEWORK

  • The framework jointly caches models and performs inference using caching, computing, and communication resources to satisfy Metaverse service objectives.Source: §III · pp. 3–4

A. Model Caching Configuration

B. Model Caching and Eviction

PFM caching and eviction must account for more complex model structures than traditional content caching. The main choices distinguish reactive versus predictive policies and whole-model versus partial loading under GPU-memory constraints.

  • PFM caching and eviction involve passive or active operations, as well as binary or partial operations.These choices reflect the complexity of managing models rather than conventional cached content.Source: B. Model Caching and Eviction · p. 4
  • Passive caching evicts models only when insufficient GPU memory prevents loading a requested model.Source: B. Model Caching and Eviction · p. 4
  • Active caching proactively loads and evicts models using predicted future demand, improving efficiency but requiring sophisticated prediction and responding less well to sudden demand changes.Source: B. Model Caching and Eviction · p. 4
  • Binary caching loads an entire model before inference, whereas partial caching starts inference with only part of the model and can improve quality as memory becomes available.Partial caching offers a lower inference level initially but is useful when memory resources are limited.Source: B. Model Caching and Eviction · p. 4

C. Collaborative Mobile Edge-Cloud Caching and Inference

Collaborative mobile edge-cloud infrastructures distribute PFM caching and inference across mobile devices, edge servers, and cloud resources. This hierarchy supports local execution when possible and uses edge or cloud alternatives when resources or cached models are insufficient.

C. Collaborative Mobile Edge-Cloud Caching and Inference

1) Mobile Caching and Inference:

2) Edge Caching and Inference:

  • When local resources are insufficient, edge servers can execute requested models, migrate misses to the cloud, or load missing models before execution.Because edge GPUs are limited, edge servers cache only several PFMs; edge execution can reduce response time and cloud load.Source: 2) Edge Caching and Inference · p. 5

3) Cloud Caching and Inference:

Cloud execution supplies broad PFM availability but adds core-network latency, while effective edge caching must reduce misses, manage switching costs, and time decisions dynamically. In-context inference further makes model residency and accumulated examples important to service quality.

3) Cloud Caching and Inference:

D. Model Caching and Eviction Policy

  • Active preloading and dynamic scheduling can reduce model miss rates while improving GPU utilization and request handling.Source: D. Model Caching and Eviction Policy · p. 5
  • Model misses require either cloud offloading or loading missing models, incurring additional latency, energy consumption, and hardware wear-and-tear.Source: D. Model Caching and Eviction Policy · p. 5
  • Cache decisions should be made when models are first loaded and when new requests arrive, enabling adjustments to current conditions and usage patterns.Source: D. Model Caching and Eviction Policy · p. 5
  • PFM cache structures differ from traditional content caches because runtime configurations and computation resources make them dynamic and in-context inference requires models to be preloaded.Additional in-context examples collected during interaction can further improve cached-service performance.Source: D. Model Caching and Eviction Policy · p. 5

IV. POTENTIAL APPLICATIONS AND CHALLENGES

Metaverse applications such as autonomous driving and digital twins rely on computationally intensive PFMs. Their diverse and changing scenarios make model caching, inference scheduling, and quality management important for supported services.

A. Applications

  • Potential PFM-based Metaverse applications include autonomous driving, digital twins, semantic communication, and AIGC for XR.Source: A. Applications · p. 5

1) Autonomous Driving:

  • Autonomous driving uses computationally intensive PFMs for traffic and driving simulation, while active model switching can adapt to changing roads or traffic patterns.Model caching and efficient inference scheduling are needed on resource-limited edge servers.Source: 1) Autonomous Driving · p. 5

2) Digital Twins:

  • Digital twins use PFMs for predictive maintenance, anomaly detection, and optimization across diverse physical-system scenarios.Managing many PFMs while maintaining quality and consistency can support accurate representation and operational decision-making.Source: 2) Digital Twins · p. 5

3) Semantic Communication:

Semantic communication and AIGC for XR require edge servers to coordinate cached PFMs under dynamic service objectives, limited resources, and multi-model workflows.

3) Semantic Communication:

  • Semantic communication can use device-to-device communication without centralized cloud infrastructure, but requires caching algorithms that preserve model quality and consistency under resource limits.Source: 3) Semantic Communication · p. 5

4) AIGC for XR:

  • AIGC for XR integrates multiple PFMs to generate relevant 3D immersive content, requiring context-aware caching and coordination among models.Source: 4) AIGC for XR · p. 5

1) Dynamic User Service Requests and Objectives:

  • Dynamic user requests and objectives require edge servers to manage limited resources, cached-model quality, and joint caching-inference policies using model size, usage frequency, and accuracy.Source: 1) Dynamic User Service Requests and… · pp. 5–6
  • Table I reports detailed PFM parameters and performance using K as the number of examples in context.Source: Table I · p. 6

2) Heterogeneous Model Configuration and Computing Resources:

Heterogeneous PFMs and edge resources make joint caching and inference difficult, while contextual examples connect resource allocation with model accuracy and latency.

2) Heterogeneous Model Configuration and Computing Resources:

  • Joint caching and inference is typically an NP-hard mixed integer programming problem because model architectures, GPU-memory needs, compute requirements, and edge capabilities differ.Source: 2) Heterogeneous Model Configuration… · p. 6

3) Context-aware Caching and Inference Algorithms:

A. Mobile AIGC Service Serving Model

  • A mobile AIGC transportation scenario may require several PFMs to work together, as Stable Diffusion combines a VAE, CLIP ViT-L/14, and U-Net block.Source: A. Mobile AIGC Service Serving Model · p. 6
  • Large language PFMs support in-context learning, and contextual information can improve AIGC quality while increasing inference resource consumption and latency as context grows.Source: A. Mobile AIGC Service Serving Model · pp. 6–7
  • Fig. 4 models few-shot accuracy as a2 = a0 + a1 log2(1 + Kα), with a0 as zero-shot accuracy, a1 as one-shot accuracy, and α as a coefficient.Source: Figure 4 · p. 7

B. Age of Context and Lease Context Algorithm

The paper introduces Age of Context to quantify demonstration freshness and relevance, then uses least-context eviction to improve edge caching outcomes for mobile AIGC services.

B. Age of Context and Lease Context Algorithm

VI. CONCLUSIONS

  • The conclusion reports that least context reduces system costs and improves the execution ratio at edge servers.Source: §VI · p. 7
Loading 2304.08782v2…