Source-linked AI summary
Generative Artificial Intelligence: A Systematic Review and Applications
Sandeep Singh Sengar, Affan Bin Hasan, Sanjay Kumar, Fiona Carroll
TL;DR
Generative AI’s rapid development has created a need to organize evidence about its techniques, applications, and limitations. This paper conducts a systematic review of recent advances and applications, finding major impact in language generation, image translation, and interdisciplinary tasks while emphasizing responsible development and ethical considerations.
Problem
The review addresses the need to gather and synthesize research on Generative AI techniques, applications, performance, trends, and challenges.
Method
The paper conducts a systematic literature review of recent Generative AI advancements, algorithms, application-specific models, and developments.
Results
The review identifies major Generative AI advances and applications in language generation, image translation, video translation, question answering, code generation, and interdisciplinary tasks.
Takeaways & Limitations
Future Generative AI research is expected to pursue evolving architectures, responsible generation, bias minimization, ethical alignment, and interdisciplinary applications.
Takeaways & Limitations
Generative AI does not supplant humans in software engineering because it struggles with intricate software design principles and identifying code smells.
Abstract
from arXiv · showhide
In recent years, the study of artificial intelligence (AI) has undergone a paradigm shift. This has been propelled by the groundbreaking capabilities of generative models both in supervised and unsupervised learning scenarios. Generative AI has shown state-of-the-art performance in solving perplexing real-world conundrums in fields such as image translation, medical diagnostics, textual imagery fusion, natural language processing, and beyond. This paper documents the systematic review and analysis of recent advancements and techniques in Generative AI with a detailed discussion of their applications including application-specific models. Indeed, the major impact that generative AI has made to date, has been in language generation with the development of large language models, in the field of image translation and several other interdisciplinary applications of generative AI. Moreover, the primary contribution of this paper lies in its coherent synthesis of the latest advancements in these areas, seamlessly weaving together contemporary breakthroughs in the field. Particularly, how it shares an exploration of the future trajectory for generative AI. In conclusion, the paper ends with a discussion of Responsible AI principles, and the necessary ethical considerations for the sustainability and growth of these generative models.
1 Introduction
This section frames Generative AI as a major development in artificial intelligence and explains the systematic review’s scope, selection process, and organization. The review synthesizes techniques, applications, trends, challenges, and future directions.
- Generative AI creates text, images, and other media by learning patterns and structures from training data.
- The review gathers, evaluates, and synthesizes research on Generative AI techniques, applications, and model performance.
- The review focuses primarily on significant developments in Generative AI between 2018 and 2023.
- Its selection criteria emphasize English-language peer-reviewed conference and journal studies reporting significant advancements or innovative applications.
- The paper discusses historical development, real-world uses, early architectures, recent applications, challenges, opportunities, and future directions.
2 What is Generative Artificial Intelligence?
This section defines Generative AI and introduces GANs, Transformers, VAEs, and diffusion models as major generative architectures. It also describes GAN operation and the early training difficulties that motivated later advances.
- Generative AI systems generate novel text, images, or other media by learning patterns and structures from training data.
- GANs use a generator and discriminator in an adversarial process to produce synthetic data that can resemble real data.The generator creates samples from noise, while the discriminator distinguishes generated from genuine examples and feeds signals back into training.
- Early GAN-based models faced training divergence and model collapse, limiting stability, output quality, and diversity.Training divergence prevents stable equilibrium, while model collapse produces limited and repetitive outputs.
- Transformers use self-attention and multi-head attention mechanisms for sequence-to-sequence tasks, including machine translation and language generation.
- VAEs combine an encoder and decoder with variation in a latent space to generate new synthetic data samples.
- Diffusion models iteratively transform simple data distributions into complex, high-dimensional ones.
3 Evolution of Generative AI Models: A Look at Earlier Variants
This section traces efforts to extend and stabilize generative models, covering convergence analysis, architectural variants, information-based objectives, sequence generation, Transformers, VAEs, and diffusion models.
- GAN convergence analysis examines Jacobian eigenvalues to study local stability during training.
- Eigenvalues near the imaginary axis were observed in practical scenarios, but their prevalence and causal role in training instability remain unresolved.
- Under absolute continuity and sufficiently small learning rates, GANs exhibit local convergence according to the cited analysis.
- Wasserstein GAN addresses vanishing gradients associated with Jensen-Shannon divergence becoming constant when real and generated distributions overlap insufficiently.
- BiGANs extend GANs with an encoder-like decoder Q to map real data into latent space for feature learning.
- InfoGAN maximizes mutual information between latent codes and generated outputs, while AC-GAN incorporates class information into synthesis and discrimination.
- SeqGAN treats the generator as a reinforcement-learning policy for generating discrete token sequences with discriminator-based policy gradients.
- Transformers, VAEs, and diffusion models broaden generative modeling through sequence modeling, probabilistic latent representations, and distribution transformation.
4 Advancements in Generative AI and Their Diverse Applications
The review surveys generative AI advances across image translation, video generation, natural language processing, and other applications, emphasizing application-specific models and reported performance gains. It also connects these developments to unresolved multilingual capabilities and educational opportunities.
- Image Translation: MMTrans produces translated T2 MRI images closest to ground truth, outperforming Pix2Pix, CycleGAN, and RegGAN qualitatively.
- Image Translation: UVCGAN improves unpaired image translation while preserving image regeneration, whereas ACL-GAN and Council-GAN overemphasize or alter non-essential features.
- Video Synthesis and Generation: Depth-aware GAN outperforms state-of-the-art models for talking-head face generation on VoxCeleb and ClebV datasets.
- Video Synthesis and Generation: DaGAN++ improves expression-related facial motion capture, particularly around the eyes and mouth, through precise facial geometry estimation.
- Video Synthesis and Generation: StyleTalker generates high-quality audio-driven talking-head videos while preserving the target identity.
- Video Synthesis and Generation: 86.1% top-1 accuracy is achieved by fine-tuned MViTv2-L on Kinetics-400, while training from scratch yields 81.0% for MViTv2-S and 82.9% for MViTv2-B.
- Natural Language Processing: Generative AI applications in natural language processing include multilingual evaluation, malware classification, and extractive text summarization.
- Natural Language Processing: BERT-SVM and BERT-kNN achieve 96% accuracy for malware classification, while Word2Vec-based methods remain below 90%.
5 Challenges and opportunities of Generative AI
Generative AI offers opportunities for collaboration, creative expression, content generation, and education, while raising ethical, security, fairness, privacy, and interpretability challenges.
- Challenges and proposed solutions: GenAI can be misused to create deepfakes for identity theft or misinformation, motivating ethical governance structures, guidelines, and regulations.Security measures and research into adversarial robustness are also proposed to address vulnerabilities in generative models.
- Challenges and proposed solutions: Generative models may amplify training-data biases, producing discriminatory or unfair outputs unless bias mitigation and inclusive datasets are implemented.The proposed response includes methods to detect and mitigate bias in training data.
- Challenges and proposed solutions: Large training datasets may cause generative models to remember sensitive information, creating privacy risks addressed through data-protection regulations and privacy-preserving approaches.Interpretability is another challenge because generative algorithms often operate as black boxes.
- Opportunities: Generative AI supports human-AI collaboration, creative expression, content generation, and education and training across numerous industries.Applications include design, problem-solving, creativity, music, generative art, literature, image synthesis, text generation, and video creation.
6 Conclusion and Future Direction
The paper synthesizes recent generative AI algorithms, methodologies, and application-specific advances, highlighting major activity in NLP and video translation. It projects continued architectural, interdisciplinary, and responsible-AI development across domains.
- Recent advancements: The review synthesizes key generative AI algorithms, including diffusion models, transformers, GANs, VAEs, and application-specific advancements.These methods represent the paper’s systematic overview of recent developments.
- Application advances: Generative AI’s current impact is especially prominent in NLP and video translation, supporting question answering, code generation, language translation, and image transformation.The paper presents these as state-of-the-art methodologies addressing human-centric and interdisciplinary challenges.
- Application advances: Recent generative AI techniques have produced cutting-edge advances across the application areas reviewed by the paper.
- Future directions: Future research is expected to evolve AI architectures, strengthen responsible generation, reduce bias, and align systems with changing ethical standards.The paper also anticipates interdisciplinary applications in healthcare, climate science, and education.
- Future directions: NLP, multilingual translation, code generation, and multimedia processing are expected to remain important areas as human–AI collaboration deepens.The paper emphasizes question answering and expanded content creation, enhancement, and interpretation.
Statements and Declarations
The paper reports partial financial support from the Wales Innovation Network and Global Wales Small Grant Fund and declares no conflicts of interest. It is a systematic review based on existing literature rather than a specific dataset.
- Funding: The research received partial financial support from the Wales Innovation Network and Global Wales Small Grant Fund.The grant number is GW-230433/414 (1.3.1b).
- Competing interests: All authors declare that they have no conflict of interest.
- Review basis: This systematic review does not use a specific dataset, instead collating and evaluating existing literature to survey generative AI.