An Overview on Generative AI at Scale with Edge-Cloud Computing

• GenAI generates multimodal human-like content, but its large models require centralized cloud infrastructure that can impose high latency, energy use, and cost.Applications include image, audio, text, and 3D generation, while high traffic further increases latency.Source: §1 · p. 1
• Edge-cloud computing combines cloud computation with edge resources and data management to provide greater computational capacity and lower latency through collaboration.Source: §1 · pp. 1–2
• GenAI creates three scalable-system challenges: increased transmission latency from abundant generated data, the need for computation closer to users, and enormous resource demands.The paper also identifies computation and data offloading, low latency, personalization, and privacy as deployment concerns.Source: §1 · p. 2
• The overview contributes a review of GenAI and edge-cloud developments, identifies large-scale deployment challenges, and proposes design considerations for efficiency, latency, personalization, and privacy.Source: §1 · p. 2
• Two large-scale GenAI applications are used as concrete examples for discussing and validating deployment challenges in edge-cloud systems.Source: §1 · p. 2
• The paper proceeds from related-work comparison and technical reviews to application scenarios, deployment challenges, design considerations, future directions, and conclusions.Source: §1 · p. 2

2 COMPARISON WITH RELATED WORK

• The section compares related overviews across GenAI and edge-cloud computing, supported by a figure showing VAE, GAN, and Transformer architectures.Figure 4 identifies the three model categories but does not provide further architectural detail in the supplied passage.Source: §2 · p. 4, Figure 4 · p. 5

2.1 GENERATIVE AI

• This work studies deployment challenges and proposes a practical cloud-edge computing solution, whereas most GenAI overviews focus on algorithmic aspects.It addresses technical challenges, design considerations, exemplary applications, and future technology outlook.Source: §2.1 · p. 4

2.2 EDGE INTELLIGENCE

• Existing edge-intelligence overviews mainly consider discriminative tasks such as surveillance, autonomous vehicles, and IoT applications.The cited examples include binary-decision systems and a roadmap for integrating edge-cloud computing with AI.Source: §2.2 · p. 4
• GenAI introduces machine-created content, large models, power consumption, and low-latency requirements that differ from traditional edge-intelligence settings.The paper identifies [26] as the only prior work addressing GenAI at the edge, while emphasizing a broader scope here.Source: §2.2 · p. 4

3.1 GENERATIVE AI

• GenAI evolved through VAE and GAN methods, the Transformer era, and the large-model era, with Transformers enabling powerful language models.The overview places these eras in 2014–2017, 2018–2019, and 2020–present, respectively.Source: §3.1 · pp. 5–6
• VAEs encode inputs into latent vectors and decode them back into outputs, while GANs train a generator against a discriminator before retaining the generator for inference.VAE noise injection enables multiple samples, and GAN training approaches an equilibrium between generated and real data.Source: §3.1 · pp. 5–6
• Cloud-only GenAI deployment creates high latency and scalability concerns because users transmit requests over long distances while generated data and demand continue growing.These constraints hinder AR, VR, and MR applications and can make popular services costly to operate.Source: §3.1 · p. 6

3.2 EDGE-CLOUD COMPUTING

• Edge-cloud computing combines cloud resources with edge proximity, supporting lower latency, flexible load balancing, and improved preservation of user data and privacy.Cloud, MEC, and edge-cloud computing are presented as three paradigms, while MEC is limited for computationally demanding tasks.Source: §3.2 · pp. 6–7
• Cloud servers suit large, high-level models, whereas edge devices handle lower-level preprocessing, but neither centralized cloud nor edge-only computing alone scales well as AIGC grows.Offloading computation and data across cloud, edge, and user devices is presented as a key edge-cloud concept.Source: §3.2 · p. 7
• Green-learning models offer smaller sizes, lower FLOPs, faster inference, and lower power consumption, making hybrid deep- and green-learning solutions compatible with edge-cloud collaboration.The paper links this collaboration to processing low-level data, aggregating high-level abstractions, and generating creative content.Source: §3.2 · p. 8
• Edge-cloud computing is motivated by the need to scale GenAI beyond expensive clusters such as Meta’s 16,000-GPU system by using expandable, under-utilized resources closer to users.The cited cluster performs five exaflops with 16,000 Nvidia A100 GPUs and 16 Tb/s interconnect bandwidth.Source: §3.2 · pp. 8–9
• The edge-cloud deployment model keeps computationally heavy, offline asynchronous training in the cloud while placing trained models near users for inference and edge-side personalization.Edge servers can fine-tune models, preserve privacy, and interface between cloud and edge services.Source: §3.2 · p. 9

4 TWO EXEMPLARY SERVICES

• The two exemplary services target low-latency GenAI applications with many future users, making scalable edge-cloud deployment important for successful adoption.The examples are the metaverse and Artificial Intelligence of Things (AIoT).Source: §4 · p. 9

4.1 METAVERSE SYSTEM

• In the metaverse design, cloud clusters generate and render scenes from user viewpoints, transmit compressed scenes, and let lightweight user devices decode and display them.Centralized maps are shared among users, while heavy computation remains in powerful cloud infrastructure to reduce transmitted data and latency.Source: §4.1 · p. 9
• Metaverse edge servers distribute geographically relevant content, cache shared scenes, and assist with compression and decompression to reduce latency and improve scene quality.Users in the same location connect to the same edge server, allowing cached scenes to be reused.Source: §4.1 · p. 10

4.2 ARTIFICIAL INTELLIGENCE OF THINGS

• AIoT combines AI with IoT and wireless networking so end devices can process and analyze data and support applications such as voiced assistants.The examples include autonomous driving, smart cities, and smart homes, where fluent speech is generated from text data.Source: §4.2 · p. 10
• AIoT edge-cloud systems can train personalized lightweight models on user devices, aggregate parameters through federated learning, and synchronize updated models through firmware updates.Online optimization supports continual training from streams of device data while preserving user-data privacy.Source: §4.2 · p. 10
• AIoT deployment can distribute large, middle-size, and lightweight models across cloud servers, edge servers, and user devices, using distillation, pruning, and personalization.The supplied passage describes specialization by application and efficient fine-tuning of lightweight device models.Source: §4.2 · p. 10
• The AIoT discussion compares power consumption, carbon emission, and cloud computational cost during training across GenAI modalities.Table 2 is identified as the comparison source, but no numerical values are supplied here.Source: Table 2 · p. 11

5 TECHNICAL CHALLENGES

• The major deployment challenges are model-size growth, power consumption, latency, and infrastructure reliability, motivating resource coordination between edges and the cloud.These challenges jointly define the need for scalable edge-cloud GenAI systems.Source: §5 · p. 11

5.1 GROWTH IN MODEL SIZES

• GenAI model sizes double every 6 months, while CPU and GPU computation power doubles every two years, creating a potential computation-supply gap.The paper calls this trend “Moore’s Law for GenAI” and identifies collaborative cloud-edge training and execution as an urgent issue.Source: §5.1 · p. 11

5.2 POWER CONSUMPTION

• Power consumption, carbon emissions, and computational costs rise with FLOPs, making large-scale GenAI services difficult to scale sustainably.The paper cites GPT-3 requirements of 10^23 FLOPs for one training iteration and 10^15 FLOPs for inference, alongside substantial energy demand per request.Source: §5.2 · pp. 11–12

5.3 LATENCY

• Latency equals uplink transmission time, inference time, and downlink transmission time, with GenAI multimedia outputs further increasing transmission demands.Cloud systems suffer long user-to-cloud transmission times, while edge-cloud coordination can reduce inference and transmission delays through task division and faster edge-cloud links.Source: §5.3 · pp. 12–13

5.4 INFRASTRUCTURE RELIABILITY

• Large GenAI services require costly GPU infrastructures that are difficult to replicate globally and vulnerable to physical and cyberattacks.The paper proposes distributed infrastructure with multiple lightweight cloud servers and many edge servers as a more robust direction.Source: §5.4 · p. 13

6 DESIGN CONSIDERATIONS

• Training should coordinate cloud and edge resources through computation offloading, personalization, privacy preservation, and information recency.The design framework treats training as distinct from deployment and emphasizes collaboration across user devices, edges, and cloud servers.Source: §6 · p. 13, §6.1 · p. 14

6.1 TRAINING

• DNN training can distribute layers across devices, edges, and clouds, or train smaller models in parallel and combine them through federated learning.Hybrid model and data parallelism is also proposed, with gradient transmission reducing the information exchanged in layer-partitioned training.Source: §6.1 · pp. 14–15
• Personalization can fine-tune a cloud-trained foundation model at edge servers using relatively small amounts of user data.The foundation model handles general requests, while edge fine-tuning shifts it toward user-specific behavior.Source: §6.1 · p. 15
• Federated learning shares model parameters rather than personal data, aggregating user models in the cloud and synchronizing the resulting model back to users.This design is intended to preserve privacy while collecting relevant information from users.Source: §6.1 · p. 15
• Online and offline models can maintain information recency by combining cloud-side updates with smaller edge-side models that are periodically synchronized.The cloud model maintains updated information, while the edge model supports low-latency inference.Source: §6.1 · pp. 15–16

6.2 DEPLOYMENT

• Deployment requires lightweight inference models because edge servers and user devices cannot efficiently host large, power-intensive GenAI models.Knowledge distillation, pruning, quantization, and compression are identified as ways to reduce model size and deployment cost.Source: §6.2 · pp. 16–17
• Cross-domain GenAI systems can place user-facing input and output modules at edges while keeping the computation-intensive generation module in the cloud.This decomposition applies to multimodal systems such as CLIP [104] and reduces transmission of generated content.Source: §6.2 · p. 17, Figure 18 · p. 17

7.1 GENERIC VERSUS DOMAIN-SPECIFIC GENERATIVE AI MODELS

• Domain-specific GenAI models may trade breadth for depth to reduce service cost and improve content quality in areas such as healthcare and financial advice.The paper highlights accuracy as a priority for some application domains.Source: §7.1 · p. 17

7.2 DECOMPOSITION OF LARGE LANGUAGE MODELS

• Decomposing large language models into smaller models connected to domain-specific knowledge graphs could reduce complexity and improve transparency, scalability, and personalization.Generic knowledge graphs can remain in the cloud while personalized graphs are stored locally or on user devices.Source: §7.2 · p. 17

7.3 QUALITY AIGC ASSURANCE

• Visual AIGC assurance must address common-sense errors, sensitive or copyrighted content, trademarks, and deepfakes, while automatic detection remains an open problem.A lightweight deepfake detection solution is cited as one response [109].Source: §7.3 · p. 18
• Textual AIGC should be evaluated for grammatical correctness, readability, and factual correctness because GenAI can facilitate large-scale misinformation.Readability includes coherency and conciseness and is harder to assess than grammatical errors.Source: §7.3 · p. 18

7.4 GREEN GENERATIVE AI MODELS

• Green GenAI models aim to reduce carbon footprints through lightweight architectures, lower computational complexity, and logical transparency.The paper identifies green-learning-based models such as NITES [110], TGHop [111], Pager [112], and GENHOP [113] as promising for edge and cloud use.Source: §7.4 · p. 18

8 CONCLUSION

• Scalable GenAI should balance computational resources between edges and cloud servers while considering latency, data privacy, and personalization.The conclusion emphasizes federated learning, with small models trained at edges and large models trained in the cloud.Source: §8 · p. 18
• Most inference tasks can be distributed to edges, while future research should address domain-specific models, large-language-model decomposition, green GenAI, and quality assurance.These directions are presented as responses to the deployment challenges identified throughout the paper.Source: §8 · p. 18