PaperlayerView exact revision on arXiv

Source-linked AI summary

Reliable and Responsible Foundation Models: A Comprehensive Survey

Xinyu Yang, Junlin Han, Rishi Bommasani, Jinqi Luo, Wenjie Qu, Wangchunshu Zhou, Adel Bibi, Xiyao Wang, Jaehong Yoon, Elias Stengel-Eskin, Shengbang Tong, Lingfeng Shen, Rafael Rafailov, Runjia Li, Zhaoyang Wang, Yiyang Zhou, Chenhang Cui, Yu Wang, Wenhao Zheng, Huichi Zhou, Jindong Gu, Zhaorun Chen, Peng Xia, Tony Lee, Thomas Zollo, Vikash Sehwag, Jixuan Leng, Jiuhai Chen, Yuxin Wen, Huan Zhang, Zhun Deng, Linjun Zhang, Pavel Izmailov, Pang Wei Koh, Yulia Tsvetkov, Andrew Wilson, Jiaheng Zhang, James Zou, Cihang Xie, Hao Wang, Philip Torr, Julian McAuley, David Alvarez-Melis, Florian Tramèr, Kaidi Xu, Suman Jana, Chris Callison-Burch, Rene Vidal, Filippos Kokkinos, Mohit Bansal, Beidi Chen, Huaxiu Yao

arXiv:2602.08145v1cs.LGcs.AIcs.CLcs.CVcs.CY

TL;DR

Research areas relevant to foundation-model reliability and responsibility are interconnected, yet their intersections need greater attention. This survey reviews the research landscape and its future directions, concluding that a holistic approach to those intersections is important for reliable and responsible development.

  • Problem

    The survey identifies a need for greater attention to intersections among interconnected areas of foundation-model reliability and responsibility research.

    Source: §12 · p. 87

  • Method

    The survey examines technical and societal considerations across foundation-model reliability and responsibility, highlighting intersections and potential future research directions.

    Source: §12 · p. 87

  • Results

    The survey concludes that the covered research areas are interconnected and influence one another, and highlights challenges and future directions at their intersections.

    Source: §12 · p. 87

  • Takeaways & Limitations

    The survey emphasizes a holistic approach to interconnected challenges rather than addressing them in isolation to support overall reliability and responsible development.

    Source: §12 · p. 87

    Building privacy attacks that work under weaker assumptions and in more general scenarios remains an open challenge.

    Source: §6.4.1 · p. 42

Abstract

from arXiv · show

Foundation models, including Large Language Models (LLMs), Multimodal Large Language Models (MLLMs), Image Generative Models (i.e, Text-to-Image Models and Image-Editing Models), and Video Generative Models, have become essential tools with broad applications across various domains such as law, medicine, education, finance, science, and beyond. As these models see increasing real-world deployment, ensuring their reliability and responsibility has become critical for academia, industry, and government. This survey addresses the reliable and responsible development of foundation models. We explore critical issues, including bias and fairness, security and privacy, uncertainty, explainability, and distribution shift. Our research also covers model limitations, such as hallucinations, as well as methods like alignment and Artificial Intelligence-Generated Content (AIGC) detection. For each area, we review the current state of the field and outline concrete future research directions. Additionally, we discuss the intersections between these areas, highlighting their connections and shared challenges. We hope our survey fosters the development of foundation models that are not only powerful but also ethical, trustworthy, reliable, and socially responsible.

1 Introduction

This survey examines how to develop foundation models that are both reliable and responsible, unifying nine critical dimensions across four model classes. It highlights connections among these dimensions and focuses on models used as intended by their developers.

1 Introduction

  • Reliability means accurate, consistent, robust performance under challenging conditions, while responsibility means alignment with ethical principles and societal values, including fairness, privacy, security, and transparency.The survey aims to synthesize technical challenges and solutions for satisfying both criteria.Source: §1 · p. 4
  • The survey’s central contribution is a cross-cutting analysis connecting nine reliability and responsibility dimensions across four foundation-model classes and their interactions.It emphasizes interconnections and trade-offs that may be overlooked in surveys focused on individual issues.Source: Figure 1 · p. 2, §1 · p. 4
  • The survey focuses on intrinsic model properties when foundation models operate as intended, complementing work on deliberate malicious misuse such as disinformation and cyberattacks.Source: §1 · p. 4
  • The review covers bias and fairness, alignment, security, privacy, hallucination, uncertainty, distribution shift, explainability, and AIGC detection.It discusses how biases arise and are evaluated or mitigated, security threats and safeguards, privacy in data collection, hallucinations, uncertainty quantification and mitigation, robustness to shifts, explainability, and AIGC detection methods and assumptions.Source: §1 · p. 4
  • The review covers literature publicly available through May 2025 and is intended to inform researchers, practitioners, and policymakers working across real-world settings.Source: §1 · p. 4

2 Types of Foundation Models

The survey groups foundation models into four types: language models, multimodal-to-text models, image generative models, and video generative models. They differ in their input and output modalities and in the architectures used to process or generate content.

LLMs

  • LLMs understand, generate, and manipulate language, using encoder-only, decoder-only, or encoder-decoder architectures for tasks including classification, content creation, sequence-to-sequence tasks, summarization, translation, and dialogue.Transformer architectures are the main implementation, while State-Space Models and Linear RNNs are emerging alternatives [323] [1059].Source: §2 · p. 9

MLLMs

  • MLLMs process multiple input modalities to generate text, typically integrating multimodal features in a shared latent space, as in CLIP and LLaVA [570].Although the general mapping allows text, image, video, or audio inputs, this survey primarily focuses on text-and-image inputs mapped to text; applications include robotics, healthcare, and augmented reality.Source: §2 · p. 9

Image generative models

  • Image generative models map multimodal inputs—most commonly textual instructions—to images, using diffusion models, GANs, or autoregressive Transformers.Their architectures often combine a Transformer text encoder with a generative backbone; this survey focuses primarily on diffusion models, which it describes as dominant in image generation. Applications include digital art, product design, and education [843].Source: §2 · p. 9

Video generative models

  • Video generative models map multimodal inputs to videos, often extending diffusion models to capture temporal relations for coherent generation, with autoregressive approaches as an alternative.Applications include realistic physical-world simulations and high-quality human-centric interactions.Source: §2 · p. 9

3 Bias and Fairness

Foundation models inherit biases from large-scale training data, which can propagate downstream, yet their sources and impacts remain poorly understood. The section surveys how bias and fairness are defined, measured, and mitigated across language and multimodal models.

3 Bias and Fairness

  • Training-data biases can propagate into downstream applications [324] [97], but their sources and impacts across data, models, and applications remain poorly understood [281].The survey calls for more work to measure and mitigate bias to advance fairness and equity in AI systems [100].Source: §3 · p. 10
  • The section covers fairness definitions, consequences, and LLM criteria, bias measurement and mitigation, and bias in MLLMs and image generative models.Figure 3 presents a taxonomy of bias and fairness in foundation models; Figure 5 shows the organization of measurement and mitigation methods.Source: §3 · p. 10, Figure 3 · p. 10

3.1 Definitions

This section defines social bias and fairness in LLMs, framing them as reliability challenges tied to social groups and connecting them to security and content detection. It describes how biased data can shape model behavior and how fairness can be assessed across groups.

Definitions

  • Social biases associate demographic groups with stereotypical, uncharacteristic, or overgeneralized traits, and the section situates bias and fairness within foundation-model reliability alongside security and content detection.Prior work addresses these concepts across social domains and downstream tasks.Source: §3.1 · p. 11
  • Biases are systematic deviations in LLM responses, representations, or reasoning that reflect disparities, stereotypes, or inaccuracies in training data and may misrepresent social or cultural norms.They can arise from imbalanced or biased distributions across domains and genres.Source: §3.1 · pp. 11–12
  • Bias can appear across generation, translation, retrieval, question answering, inference, and classification, including gendered job choices, stereotyped answers, and dialects mislabeled as aggressive.The examples include translation systems defaulting to masculine job terms and searches for successful leaders overlooking women.Source: §3.1 · p. 12
  • Fairness concerns disparities in task-specific performance across social groups and can be assessed by comparing scalar scores on group-related prompts, though other definitions and metrics are also reasonable.The measurement function maps a model response to a scalar score; the section presents this as one possible fairness definition.Source: §3.1 · p. 12
  • Fairness and non-toxicity are important in deployed LLMs for customer service, decision support, and social services; proposed principles include invariance, equal associations, and replicated distributions.Gallegos et al. [281] discuss principles intended to support consensus across stakeholders and avoid disproportionate impacts on communities.Source: §3.1 · pp. 12–13

3.2 Methods for Bias Evaluation

LLM bias evaluation is organized around generated text, feature embeddings, and token-selection probabilities, using distributional, classifier-based, word-level, association, and likelihood-based measures.

Methods based on Generated Text

  • Prompted-text evaluations use benchmarks such as Dhamala et al. [224] and Gehman et al. [297], with distribution metrics testing group invariance, word co-occurrence, and demographic representation against data distributions.Measures include Social Group Substitutions, word co-occurrence scores [102], and demographic representation [545], sometimes compared with a baseline distribution using distance metrics.Source: §3.2 · pp. 13–14
  • Classifier-based evaluation measures toxicity probabilities, including expected maximum toxicity, Toxicity Probability, and Toxicity Fraction, and classifies group regard, sentiment, respect, or style variation.Regard classifiers are trained on human-annotated responses, while style-based evaluation measures variance in classifier outputs across groups.Source: §3.2 · p. 14
  • Word-level metrics compare generated words with a predefined harmful-word list or assign them precomputed bias scores [224].Source: §3.2 · p. 14

Methods based on Feature Embedding

  • Feature-embedding methods assess bias through distances between neutral and identity-related words; WEAT [117] tests concept associations, while sentence-level SEAT [637] adds contextualized templates and extends bias dimensions.WEAT resembles the Implicit Association Test; SEAT uses semantically bleached and unbleached templates.Source: §3.2 · pp. 14–15

Methods based on Token Selection Probability

  • Token-probability methods use masked templates to compare group-conditioned predictions; Webster et al. [976] normalize predicted probabilities by model priors to isolate bias attributable to the group token.Webster et al. also average divergent predictions across templates, while Kurita et al. [482] use a similar approach for neutral attributes such as occupations.Source: §3.2 · p. 15
  • PLL scores sentence-word likelihoods, supporting CrowS-Pairs stereotype comparisons [683], CAT and iCAT option scoring [679], AUL, and LMB’s perplexity comparison of biased and counterfactual statements [67].AUL extends CrowS-Pair Score and CAT by considering multiple correct predictions and avoiding word-masking selection bias.Source: §3.2 · p. 15

3.3 Methods for Bias Mitigation

The survey organizes LLM bias mitigation into training-process methods and inference-time post-processing, covering data augmentation, alignment, and detection and correction of biased outputs. It notes that post-processing may miss biases, introduce erroneous information, or increase latency.

3.3 Methods for Bias Mitigation

  • LLM bias mitigation methods broadly use training-process interventions or post-processing techniques.The survey divides training-process methods into data augmentation and alignment with instruction tuning.Source: §3.3 · p. 15
  • Data augmentation improves training-data quality, diversity, and balance, including through counterfactual replacements, mixup, iterative dataset generation, and filtering.Counterfactual Data Augmentation replaces protected-attribute words [732] [976], while interpolation blends augmented instances with originals; cited examples include mixup methods. Wang et al. [962] generate and filter diverse instruction data, while other filtering methods remove low-quality data or retain diverse, underrepresented data [101] [877].Source: §3.3 · pp. 15–16
  • Instruction-tuning alignment addresses biases that can arise from training data, training procedures, or human design choices that do not inherently prioritize human values.The survey points to alignment techniques and training algorithms for harmonizing LLMs with human preferences in Section 4.Source: §3.3 · pp. 15–16
  • Post-processing detects and corrects biased or context-misaligned text using retrieval or word-level checks, including retrieval during each Chain of Thought generation stage [433].Other approaches invoke external knowledge bases [29] or detect counterfactual or context-misaligned responses [165]; related interventions include cultural-specific instruction data [511], safety fine-tuning [761], and multi-objective probability alignment.Source: §3.3 · p. 16
  • Post-processing may leave biases uncorrected when knowledge bases lack relevant information, introduce erroneous information, and significantly increase latency.These drawbacks qualify the effectiveness of post-processing methods.Source: §3.3 · p. 16

3.4 Bias and Fairness in MLLMs

Fairness in MLLMs centers on aligning responses with multimodal inputs, while research has focused mainly on image hallucination. Broader biases from training data and dataset imbalance remain underexplored.

3.4 Bias and Fairness in MLLMs

  • MLLM fairness emphasizes responses that faithfully align with image or audio inputs; most research focuses on image hallucinations that introduce entities, quantities, or logical information absent from images [944] [943] [532] [566] [568].The section points to Section 7 for a detailed analysis of hallucinations in MLLMs.Source: §3.4 · pp. 16–17
  • Limited image-text data for specific groups, biased information, and imbalanced distributions can lead MLLMs to encode stereotypes and respond with bias to particular image-text pairs; counterfactual-image studies demonstrate bias encoding under imbalanced training data [1020].The section says bias and fairness beyond hallucinations remain largely unexplored and important for equitable foundation models.Source: §3.4 · p. 17
  • BEND-VLM [301] tailors debiasing of MLLM embeddings to each test-time input to mitigate inference-time bias while avoiding catastrophic forgetting from fine-tuning.Source: §3.4 · p. 17

3.5 Bias and Fairness in Image Generative Models

Image generative models can diverge from real-world distributions and amplify social biases. Mitigation research includes widely adopted prompt-based interventions and fairness-aware sampling methods.

3.5 Bias and Fairness in Image Generative Models

  • Text-guided diffusion models can amplify real-world biases and underrepresent marginalized identities; Figure 6 shows DALL·E’s spurious gender-profession correlation.Studies report biases involving professions, ethnicities, and social classes [1156] [182] [89]; underrepresentation of marginalized identities is highlighted by Luccioni et al..Source: §3.5 · p. 17, Figure 6 · p. 17
  • Prompt-based methods seek balanced gender representation through pronoun injection, soft-token optimization, text-embedding adjustments, or interpretable latent-concept manipulation.These prompt-based regulations are described as the most widely adopted strategy, but keyword-based approaches may disproportionately affect marginalized groups, a concern that could also apply at the prompt level [234].Source: §3.5 · p. 17
  • Sampling methods address bias by balancing generated classes with few-shot conditional diffusion or reducing amplified training-data biases through fairness-aware sampling.D2C uses few-shot conditional diffusion to generate unconditional diffusion and balance class counts; Fair Sampling introduces a fairness-aware sampling technique [183].Source: §3.5 · pp. 17–18

3.6 Current Limitations and Future Directions

Bias and fairness evaluation in foundation models remains limited by challenges in measuring bias over generation, interpreting fairness across contexts, and balancing fairness with factual accuracy and utility. Future work spans training, post-processing, auditing, and targeted interventions.

3.6.1 Limitations and Open Challenges of Bias and Fairness

  • Token- and paragraph-level metrics miss how bias can accumulate and propagate across autoregressive generation.Each token depends on previously generated tokens, so evaluation needs to capture bias over the generation process.Source: §3.6.1 · p. 18
  • Bias in current-event and social content can be subtle, contextual, and entangled with historical, cultural, and social influences.Fairness definitions also vary across social, cultural, and historical perspectives, creating tension between unbiased outputs and accurate representation of news and events [232].Source: §3.6.1 · p. 18
  • Fairness assessments must preserve accurate reporting of inequalities while spanning diverse social norms, and societal bias remains difficult to mitigate in web-scale data.Data resampling [364] can struggle to mitigate societal biases, and addressing them in web-scale datasets remains open.Source: §3.6.1 · p. 18

3.6.2 Future Directions

  • Future work should reduce bias throughout model development and generation, while ensuring security measures and AIGC detectors do not introduce unfair effects.Directions include unbiased tokenization and embeddings [721], constrained RLHF, DPO [956] [1200], revised LoRA, fairness-aware security training, post-processing autoregressive outputs, and auditing AIGC detectors for demographic disparities.Source: §3.6.2 · pp. 18–19
  • Explainability and modality-specific interventions could enable more targeted bias reduction, including locating and editing stereotype-encoding circuits and addressing disparate impacts across MLLM modalities.Mechanistic interpretability is proposed for identifying and editing model circuits; modalities in MLLMs may require distinct interventions [997].Source: §3.6.2 · p. 19
  • Future research must develop effective trade-offs between fairness and utility because pursuing absolute fairness can conflict with practical utility and performance.The survey identifies balancing fairness and utility as a significant research challenge.Source: §3.6.2 · p. 19

4 Alignment

This section surveys alignment as a means of steering LLMs and MLLMs toward human values for safer deployment, covering SFT, RLHF, and prompt engineering. It also highlights unresolved challenges in feedback reliability, direct-alignment robustness, interpretability, and keeping future superintelligent systems aligned with human values.

4.1 Supervised Fine-Tuning

  • SFT tunes pretrained LLMs to mimic desired ground-truth responses and can stabilize alignment, but its effectiveness and safety depend on high-quality, diverse, carefully vetted training data.LIMA reports improvements after fine-tuning LLaMA-65B on 1,000 prompt-response pairs without reinforcement learning or explicit preference modeling; Self-Instruct uses seed instructions to generate training data, while Qi et al. [729] show that strategically crafted examples can undermine safety protocols.Source: §4.1 · pp. 20, 22

4.2 Reinforcement Learning from Human Feedback

  • RLHF initializes from SFT, trains a reward model on pairwise preferences, and optimizes the policy with PPO and KL regularization; alternatives explore finer-grained rewards and AI-generated feedback.Reward models can be biased, preference dimensions can conflict, and values vary across populations, making a single universal alignment target ill-posed; process-reward models and verification methods aim to improve reward reliability and curb reward hacking. RLAIF [58] uses constitutional principles and AI feedback to reduce reliance on human labels.Source: §4.2 · pp. 23–24
  • DPO [744], KTO, and other alternatives simplify preference optimization or reduce resource demands, but methods trade off complexity, data requirements, and robustness.DPO removes the separate reward function, while GRPO [209] removes the critic and value estimation; direct methods may overfit preference data, whereas PPO-based RLHF supports online exploration and can be more robust with high-quality verified rewards.Source: §4.2 · pp. 25–26

4.3 Prompt Engineering

  • Prompt engineering steers responses through tunable continuous prompts, interpretable discrete prompts, reasoning structures such as CoT, and iterative prompt optimization.CoT elicits intermediate reasoning, extended by tree- and graph-structured approaches; prompt formats can be brittle to minor changes, and generated reasoning chains may be post-hoc rationalizations rather than faithful computational traces.Source: §4.3 · pp. 26–28
  • Prompt engineering primarily improves task performance, but can indirectly support alignment by making LLM outputs more interpretable and better suited to human expectations.Source: §4.3 · p. 28

4.4 Alignment for MLLMs

  • MLLM alignment uses visual instruction tuning to connect frozen LLMs and image encoders, supporting multimodal task performance and alignment with human preferences.Vision-language instruction samples and linear projections support visual understanding; subsequent work extends MLLMs to synthesis [891] [1032] [1184] and interactive agents [57] [1199] [1033] [807].Source: §4.4 · pp. 28–29
  • MLLM alignment work improves instruction data and feedback, using data-quality filtering, critique and refinement, synthetic preference contrasts, and self-rewarding methods.InstructionGPT-4 selects training examples; DRESS uses natural-language critique and refinement, POVID [1200] contrasts hallucinatory and truthful answers without human feedback, and STIC [212], SIMA [956], CSR, and AnyPrefer explore self-rewarding without extra models or data.Source: §4.4 · p. 29

4.5 Current Limitations and Future Directions

  • RLHF faces tractability problems in feedback quality, poisoning, and bias, alongside deeper limits in modeling diverse values and preventing reward hacking or sycophancy.Casper et al. [128] distinguish these tractability and generality challenges; rule-based RL successes in reasoning [209] [21] are difficult to extend to broader domains.Source: §4.5 · p. 29
  • Direct alignment methods such as DPO simplify RLHF and reduce training costs, yet can overfit and prove nonrobust out of distribution, while scaling may require more SFT data or larger models.Iterative training [1111] [968] and online alignment aim to enrich training data, but remain early-stage; small models may learn simple preference-data features such as length correlation [745].Source: §4.5 · pp. 29–30
  • Superalignment seeks to keep superintelligent AI aligned through scalable oversight and weak-to-strong generalization, but superficial alignment can conceal values failures and backdoor vulnerabilities.Potential approaches include ensembles of weak models for domain-specific oversight [500] [108], model merging for generalization [753], and combining the two strategies; the challenge was introduced by OpenAI [115].Source: §4.5 · pp. 29–30

5 Security

Foundation models are vulnerable to attacks during training and inference, and the survey reviews attacks and defenses across model families within a broader reliability and responsibility framework.

5.1.1 Attack in LLMs

  • LLM attacks include jailbreaks that bypass safeguards, prompt injections that override instructions, and poisoned data or rewards that implant trigger-activated behavior.Jailbreaks may exploit conflicts between helpfulness and safety; prompt injections can arrive through external sources, while backdoors can also be introduced through RAG databases.Source: §5.1.1 · pp. 31–33

5.1.2 Defense in LLMs

  • LLM defenses use prompt preprocessing and detection, safer decoding and alignment, and provable techniques, but provable guarantees can increase overhead and reduce utility.Methods include aggregating outputs, removing malicious activation concepts, and perturbing prompts or erasing tokens before safety checks.Source: §5.1.2 · p. 33

5.2.1 Attack in MLLMs

  • MLLM attacks exploit both visual inputs and text prompts, enabling harmful outputs, privacy leakage, and backdoors triggered by modified training examples.Visual attacks include optimized adversarial images, while prompt attacks can exploit leaked system prompts or multi-hop inference of protected image attributes.Source: §5.2.1 · pp. 34–35

5.2.2 Defense in MLLMs

  • MLLM defenses address visual and language perturbations through adversarial training or pretrained-model supervision, and mitigate backdoors through detection, removal, or prevention.Some approaches disrupt poisoned image-caption pairings or search for minimal image triggers and malicious target texts.Source: §5.2.2 · p. 35

5.3.1 Attack in Image Generative Models

  • Image generative models face jailbreaks that evade safety filters, backdoors that produce triggered targets, and adversarial generation methods that attack visual models.Harmless-looking prompts can yield NSFW outputs, while attack methods also manipulate latent diffusion processes to create adversarial examples.Source: §5.3.1 · pp. 35–36

5.3.2 Defense in Image Generative Models

  • Image generative model defenses span dataset curation, trigger detection, fine-tuning, and post-generation moderation, but fine-tuning can lack scalability and degrade benign output quality.Moderation avoids changing training but depends on labeled data and may generalize poorly to unseen inappropriate content or attacks.Source: §5.3.2 · pp. 36–37

5.4.1 Limitations and Open Challenges of Attacks

  • Attack research remains costly, jailbreaks often produce only simple harmful text, and security under multi-agent, tool-use, and RAG paradigms remains an open challenge.White-box optimization is computationally intensive, while black-box optimization can incur high token costs.Source: §5.4.1 · p. 37

5.4.2 Limitations and Open Challenges of Defenses

  • Defense challenges include lacking formal guarantees, weaker multimodal effectiveness, over-safety, and unresolved trade-offs between adversarial safety and user experience.External safety systems may require extensive adversarial training or remain vulnerable to evasion while adding inference overhead; on-the-fly defenses that preserve output quality remain an open challenge.Source: §5.4.2 · pp. 37–38

6 Privacy

Foundation models can memorize or reveal sensitive training data, while privacy protections face regulatory constraints and trade-offs in accuracy, utility, and computational cost. The section surveys threats and defenses across LLMs, MLLMs, and image-generation models, alongside open challenges in attacks and privacy-preserving methods.

6.1.1 Privacy threats in LLMs

  • LLM memorization creates privacy risks spanning membership inference during training and deployment, extraction of sensitive data, and exposure of confidential prompts or decoding hyperparameters.Reported extraction targets include personally identifiable information and code-model data; some attacks also exploit fine-tuning or in-context learning.Source: §6.1.1 · pp. 39–40

6.1.2 Privacy-preserving techniques in LLMs

  • LLM defenses include DP-SGD noise during optimization and targeted measures to scrub PII, remove sensitive weights, filter outputs, or reduce memorization.The privacy-utility balance is controlled by (ϵ, δ); other methods include synthetic-text generation, token-loss exclusion, and embedding noise.Source: §6.1.2 · pp. 40–41

6.2 Privacy in MLLMs

  • MLLMs face membership inference, model stealing, and extraction of personal information from images, while training-data protections leave input-image leakage an open problem.Proposed defenses include multimodal unlearning, federated learning with differential privacy, and a differentially private CLIP variant that maintains accuracy across vision-language tasks.Source: §6.2 · p. 41

6.3 Privacy in Image Gneration Models

  • Diffusion models can expose over a thousand training examples, and personalized fine-tuning can reproduce realistic people or artists’ styles, raising privacy, authenticity, and copyright concerns.Differentially private diffusion methods include DP-SGD and efficient attention-only fine-tuning; protective image perturbations can reduce output quality or alter semantics, and GrIDPure can remove them while preserving image quality.Source: §6.3 · pp. 41–42

6.4.1 Limitations and Open Challenges of Privacy Attacks

  • Privacy attacks remain limited by member–non-member ambiguity, low extraction coverage, and strong assumptions; weaker-assumption attacks and contextualized privacy are open challenges.Context across multi-turn chatbot conversations may reveal personal attributes even when no individual turn contains private information.Source: §6.4.1 · p. 42

6.4.2 Limitations and Open Challenges of Privacy Preserving techniques

  • Differential privacy’s worst-case bounds can diverge from practical attack conditions, while DP fine-tuning and data sanitization can degrade task performance, leaving privacy–utility balance unresolved.The survey identifies strong privacy-preserving techniques that balance privacy and performance as an open challenge.Source: §6.4.2 · p. 42

7 Hallucination

This section surveys hallucination in foundation models, defining it as ungrounded or unverifiable output and covering its forms, causes, detection, measurement, and mitigation across LLMs, MLLMs, and image-generation models. It also reviews inference-time methods for reducing hallucinations in MLLMs and identifies unresolved challenges in defining, detecting, and mitigating hallucinations across foundation models and modalities.

7 Hallucination

  • Hallucinations are ungrounded or unverifiable outputs that can conflict with input or real-world facts, affecting text, images, video, and audio.They may appear plausible and be expressed confidently, undermining applications across domains.Source: §7 · p. 43

7.1.1 Hallucinations in LLMs

  • LLM hallucinations divide into factuality errors—world-knowledge contradictions or unverifiable fabrications—and faithfulness conflicts with instructions, context, or earlier generated text.Faithfulness failures include input confliction, context fabrication, and inconsistency with preceding generation.Source: §7.1.1 · pp. 43–44

7.1.2 Hallucinations in MLLMs

  • MLLM hallucinations include intrinsic or extrinsic factuality errors and faithfulness failures involving nonexistent objects or unsupported visual relationships, attributes, or quantities.Intrinsic errors may reflect out-of-distribution knowledge or limited visual ability, while extrinsic errors arise from other factors despite correct image description.Source: §7.1.2 · pp. 44–45

7.2 Sources of Hallucinations

  • Hallucinations arise from flawed or outdated data, knowledge boundaries, model architecture and training dynamics, decoding randomness, imperfect representations, and autoregressive error propagation.Token-by-token errors can compound with sequence length and produce inconsistencies with earlier output.Source: §7.2 · pp. 45–46

7.3.1 Detecting Language Hallucinations

  • Detection commonly decomposes outputs into facts, retrieves evidence, and scores veracity; LLM methods also use internal signals, uncertainty, and response consistency.Retrieval-based methods extract claims and verify them against external sources, sometimes distinguishing false claims from claims with not-enough-information evidence.Source: §7.3 · p. 46, §7.3.1 · pp. 46–48

7.3.2 Measuring Multimodal Hallucinations

  • For MLLMs, object-focused metrics and polling-based queries assess object hallucinations, while image-generation benchmarks check whether prompted concepts appear or are spuriously added.POPE uses yes-or-no object-existence queries [532]; other evaluations extend to visual question answering, image-sequence reasoning, and object attributes and relations, while RAG does not guarantee elimination of hallucinations [1013].Source: §7.3.2 · p. 48, §7.3.3 · p. 48

7.4.1 Reducing Hallucinations in LLMs

  • LLM mitigation combines data cleaning and refinement with model-based strategies such as retrieval grounding, self-contradiction removal, fine-tuning, and alignment.The review identifies training-data memorization and parametric-knowledge bias as contributors; model-based mitigation remains at an early academic stage.Source: §7.4.1 · pp. 48–49

7.4.2 Reducing Hallucinations in MLLMs

  • MLLM mitigation includes robust visual instruction tuning and fine-grained reward modeling, while enabling models to refuse questions beyond their capabilities can reduce hallucinations.The LRV-Instruction framework uses positive and negative instructions [567], and M-HalDetect supports detection and prevention of object and logical hallucinations.Source: §7.4.2 · pp. 49–50
  • Post-processing approaches correct MLLM outputs through methods including HMM- and Viterbi-based caption refinement and truthful-guided inference-time intervention.Hoxha et al. use probabilistic language structure to correct hallucinations and improve caption coherence; Duan et al. [247] learn a truthful decoding direction with an auxiliary model.Source: §7.4.2 · p. 50
  • LLM-assisted correction refines annotations, diagnoses, and captions, while LLM-free approaches include LURE and a switching value that modifies word embeddings.LURE targets co-occurrence, uncertainty, and object position as factors contributing to object hallucination; Zhou et al. describe both GPT-based correction and LURE.Source: §7.4.2 · p. 50
  • VISTA reinforces visual information in activation space using early-layer activations and reports consistent improvements across models and decoding strategies.Other decoding methods contrast outputs from different visual inputs, penalize over-trust, or compare logits across layers to reduce hallucinations.Source: §7.4.2 · p. 50

7.5 Current Limitations and Future Directions

  • Hallucination research remains difficult to formulate comprehensively, and retrieval-based validation depends on unverified knowledge bases and verifiers that may themselves hallucinate.The relationship between autoregressive hidden states and hallucination remains unclear, and research on validating image, video, and audio synthesis is insufficient.Source: §7.5 · p. 51
  • Future work should verify retrieval knowledge bases, reduce hallucinations in validation models, investigate autoregressive mechanisms, and extend research to image, video, and audio generation.Proposed directions include independent-model cross-verification, mechanistic and representational latent editing, and calibrated refusals integrated with retrieval and multi-agent pipelines and extended beyond text [1042] [1212].Source: §7.5 · p. 51

8 Uncertainty

This section examines sources of foundation-model uncertainty and methods to estimate, calibrate, select, and control it, while identifying limits in current LLM uncertainty quantification and calling for evaluation across development and deployment. It notes that evidence remains limited for complex, open-ended generation.

8.1.3 Aleatoric vs. Epistemic Uncertainty

  • Aleatoric uncertainty reflects irreducible randomness in data, while epistemic uncertainty reflects knowledge gaps that can sometimes be reduced with additional training data or informative examples.Model choices also contribute: architecture may mismatch the data-generating process, common objectives can encourage overconfidence, and fine-tuning may dilute generalist capabilities.Source: §8 · p. 52, §8.1.3 · pp. 53–54, §8.1.2 · p. 53

8.1.1 Data

  • Natural-language uncertainty comes from ambiguous, open-ended, or differently phrased inputs, as well as training data that may be limited, ambiguous, conflicting, or outdated.At inference, missing context and input errors can add further uncertainty.Source: §8.1.1 · pp. 52–53

8.2.1 Estimating Uncertainty

  • Predictive entropy is a common uncertainty measure, but generation makes it harder to estimate because it requires accounting for distributions over possible sequences.Approaches sample multiple outputs for self-consistency, group semantically equivalent sequences, or estimate epistemic uncertainty through model disagreement and direct prediction.Source: §8.2.1 · pp. 54–55

8.2.2 Calibration

  • Calibration asks whether confidence reflects answer correctness; neural models often remain overconfident, and generated or open-ended responses make calibration more difficult.Post-hoc recalibration can refine confidence estimates, while generation-focused approaches produce sequence-level confidence scores or use self-evaluation and self-consistency.Source: §8.2.2 · pp. 55–56

8.2.3 Verbalized Uncertainty

  • >80% variation in accuracy was associated with changes in epistemic markers in prompts, while studies also find verbalized uncertainty can be overconfident and difficult to optimize consistently.Models can express confidence numerically or linguistically, but such expressions may mimic training-data language rather than reflect underlying uncertainty.Source: §8.2.3 · p. 57

8.2.4 Addressing Uncertain Examples

  • Selection methods identify difficult inputs so models can abstain, defer answers, or request more information, supporting safer responses under uncertainty.Interactive approaches detect ambiguity and ask clarifying questions; datasets and benchmarks have been developed to evaluate ambiguity handling.Source: §8.2.4 · pp. 57–58

8.2.5 Distribution-free Uncertainty Quantification

  • Distribution-free uncertainty methods use calibrated prediction sets or risk bounds, while Prompt Risk Control selects prompts using upper bounds on risk measures.These methods extend from classification toward generation, with applications including stopping rules, candidate-generation sets, conformal abstention, and factuality control.Source: §8.2.5 · pp. 58–59

8.3 Current Limitations and Future Directions

  • Uncertainty research remains limited because studies often use short-answer tasks with a single correct response, leaving its relevance to long-form, open-ended generation unclear.The survey notes some progress but identifies complex sequential settings such as chat and customer care as unresolved evaluation contexts.Source: §8.3 · p. 59
  • Uncertainty methods can cost 2 to 20 times—or more—than standard inference, while verbal confidence may track accuracy through spurious training features.Verbalization can add inference costs, and consistency-based methods may not suit open-ended tasks or long generations where a single answer is needed.Source: §8.3 · p. 59
  • It remains unclear whether advanced zero-shot LLM uncertainty methods robustly outperform sequence entropy, which may be unavailable through black-box APIs.Token probabilities may reflect the training cross-entropy objective and data plausibility rather than task-specific uncertainty; RLHF can make accurate uncertainty estimation more difficult.Source: §8.3 · p. 59
  • Future work should study uncertainty across the model development and deployment pipeline, including how interventions and measurements interact to affect downstream outcomes.It should also examine how training-data selection, mixing, and filtering affect users’ ability to estimate downstream confidence through token probabilities or verbalizations.Source: §8.3 · p. 59
  • New architectures and pretraining recipes should be benchmarked for calibration as well as accuracy, while tracking that supervised and RL fine-tuning can worsen uncertainty quantification.Source: §8.3 · p. 59
  • For deployed models, future methods could select system prompts and few-shot exemplars to control uncertainty in the wild using rigorous statistical approaches such as DFUQ.Source: §8.3 · p. 59

9 Distribution Shift

This section examines how distribution shifts between training and test data can undermine foundation-model reliability, distinguishing covariate, label, and concept shifts and reviewing OOD detection, generalization, and domain adaptation approaches. It also covers domain-adaptation methods, trade-offs and challenges in model editing, and limitations and future directions for improving robustness, adaptability, and data efficiency.

9.2 Out-of-Distribution Detection

  • LLMs can detect OOD inputs without fine-tuning, and ID fine-tuning can improve detection; visual methods use LLM-generated knowledge and peer classes to identify unfamiliar samples.Liu et al. report that generative fine-tuning generalizes better and a simple cosine-distance detector is effective. OOD detection can also flag inputs for scrutiny and human oversight in dynamic, high-stakes settings.Source: §9.2 · pp. 61–62

9.3 Out-of-Distribution Generalization

  • Simple EDA often degrades LLM performance; adversarial perturbations improve robustness, but Gaussian noise and complex questions harm MLLMs, while label smoothing did not improve LLM generalization.Adversarial training uses min-max optimization to find worst-case input perturbations and train smoother decision boundaries; Gaussian noise was the most detrimental tested image augmentation, and multi-connective questions also reduced performance.Source: §9.3.1 · p. 62, §9.3.2 · p. 63, §9.3.3 · p. 63
  • Invariant learning seeks environment-stable representations; Focal Loss emphasizes hard examples, while ensembling showed limited gains in Yuan et al. but newer algorithms improved downstream performance.Focal, Dice, and Mixup losses use different inductive biases to form more generalizable decision boundaries; Jiang et al. [421] and Wan et al. [930] report improvements from advanced ensemble algorithms.Source: §9.3.4 · p. 63, §9.3.5 · pp. 63–64

9.4.1 In-context Learning

  • ICL adapts without parameter updates and performs better than fine-tuning in low-resource OOD scenarios, but MLLM ICL is vulnerable to domain, label, and spurious-correlation shifts.BOSS finds fine-tuning advantageous when sufficient training data is available; Reizinger et al. describe approximate non-identifiability in ICL and its implications for OOD generalization.Source: §9.4.1 · p. 64

9.4.2 Retrieval-augmented Generation

  • RAG supplements queries with retrieved domain knowledge, while larger and more diverse retrieval databases cover more distributions and multimodal retrieval methods address cross-modal knowledge shifts.MASSIVEDS contains 1.4 trillion tokens; RAFT trains LLMs to ignore distractor documents, and domain-specific RAG methods have shown effectiveness across real-world tasks.Source: §9.4.2 · pp. 64–65

9.4.3 Fine-Tuning with New Knowledge

  • Fine-tuning on full data favors ID performance, while ICL can favor OOD instances; RLHF generally improves unseen-input generalization, and TAIA uses fine-tuned attention parameters alone at inference.RLHF is especially helpful under significant train-test shifts; the cited work also finds parameter-efficient tuning can retain broader pretrained capabilities and explores activation steering for domain-relevant behavioral changes.Source: §9.4.3 · pp. 65–66

9.4.4 Test-time Training

  • Test-time training creates a self-supervised learning task per test sample and updates parameters before prediction, aligning training and test distributions to improve performance under shifts.Proposed TTT layers optimize hidden states, while customizable large-chunk updates support long-sequence modeling across diverse modalities.Source: §9.4.4 · p. 66

9.4.5 Model Editing

  • Model editing aims to correct specific errors while preserving reliability, generality across equivalent contexts, and locality for unrelated queries; methods use external memories or classifier-controlled interventions.Memory-based approaches store or retrieve edits, while classifier-based methods decide whether to intervene; the latter can depend heavily on classifier quality and require many unrelated training samples.Source: §9.4.5 · pp. 66–68
  • Current MLLM-editing methods can edit textual components but fall short on vision modules, and MMEdit evaluates Reliability, Locality, and Generality across VQA and image captioning.The benchmark addresses Editing VQA and Editing Image Captioning; the survey calls for efficient, accurate cross-modal editing and evaluation of larger MLLMs.Source: §9.4.5 · p. 68
  • Model-editing paradigms trade off flexibility, locality, scalability, and unintended interference, while multimodal editing can amplify their limitations.Hypernetwork methods generate updates from edit descriptions but require pretraining and may generalize poorly beyond synthetic edits; knowledge-based methods localize and update parameters associated with facts but can affect unrelated knowledge. ROME uses a rank-one update, while MEMIT edits multiple facts across layers.Source: §9.4.5 · pp. 68–69

9.5 Current Limitations and Future Directions

  • Distribution shifts can reduce foundation-model performance, while limits in OOD detection, generalization, and adaptation undermine reliability in changing real-world environments.OOD detection methods often face scalability problems; domain adaptation and adversarial training can require resource-intensive retraining or fine-tuning, and robustness methods may depend on large, high-quality datasets. Multimodal processing remains difficult, and continual adaptation can cause horizontal or vertical catastrophic forgetting [818].Source: §9.5 · p. 69
  • Future work should develop lightweight OOD-detection and generalization methods that identify and mitigate shifts in large-scale settings with low resource requirements.The aim is to make these methods practical to integrate into foundation-model systems.Source: §9.5 · p. 69
  • Continual or lifelong learning should adapt to new distributions without extensive retraining while preserving prior knowledge against horizontal and vertical forgetting [1058] [818].Data-efficient transfer learning or diverse synthetic data could also help address data scarcity and improve generalization.Source: §9.5 · p. 70
  • Unified multimodal frameworks using cross-modal learning and multimodal embeddings could improve handling of diverse data types and performance in complex scenarios [1003] [1155] [1103].For editing, memory- and classifier-based methods may generalize more easily across modalities but need modality-aware retrieval or classification, while parameter-based methods need cross-modal localization; hybrid approaches could combine precision with flexibility and safety.Source: §9.4.5 · p. 69, §9.5 · p. 70

10 Explainability

This section surveys explainability methods for foundation models, including approaches to attribute LLM predictions, uncover stored knowledge and sample roles, evaluate explanations and their applications, and explain MLLMs. It also considers explainability’s potential to support fairer, safer, and more up-to-date models, alongside challenges in faithfulness, cross-modal transfer, validity, robustness, and fit with practitioners’ needs.

10.1.1 Perturbing the Input for Explanation

  • Perturbation methods explain predictions by masking or editing inputs to identify influential tokens, construct contrastive explanations, and generate counterfactuals.Polyjuice is a general-purpose generator of diverse, realistic counterfactuals.Source: §10.1.1 · pp. 71–72

10.1.2 Gradient-based Explanation

  • Gradient-based methods assign token relevance, while attention analyses expose information flow; combining gradients with attention generally performs better than attention alone [260] [216] [68].Sequential Integrated Gradients [260] attributes along GPT-2’s generation path, and propagates importance through autoregressive decoding steps to scale to long generations.Source: §10.1.3 · p. 72, §10.1.2 · p. 72

10.2 Exploring the Knowledge in LLMs

  • Knowledge-oriented explanations probe representations, concepts, and internal mechanisms, but linguistic information can be entangled and its use during inference remains incompletely understood [636].Probes find syntactic information more prominent in lower layers and semantic information in higher layers; mechanistic interpretability examines attention and FFN circuits as contextual or parametric knowledge stores.Source: §10.2.1 · p. 73, §10.2.2 · p. 74, §10.2.3 · p. 74

10.3 Discovering the Roles of Samples in Training, Fine-tuning, and Few-shot Learning

  • Sample-role research examines individual training-example influence, pretraining versus instruction-tuning, and how demonstrations and model scale shape in-context learning [986].LIMA reports near-GPT-4 performance in a controlled human study after training Llama-65B on 1,000 curated examples.Source: §10.3.1 · p. 75, §10.3.2 · p. 75, §10.3.3 · p. 75

10.4 Evaluation of Explainability

  • Explainability evaluation distinguishes human-perceived plausibility from faithfulness to model behavior, but faithfulness metrics can disagree and do not readily assess natural-language explanations [131] [50] [146].Plausibility measures persuasiveness and understandability; faithfulness measures alignment with model behavior. Proposed alternatives include counterfactual tests, input reconstruction, and REV’s measurement of label-relevant information beyond the input.Source: §10.4 · p. 76, §10.4.2 · p. 76, §10.4.1 · p. 76

10.5 Applications of Explainability

  • Explanations can expose shortcut reliance and support model improvement, but chain-of-thought rationales may be systematically unfaithful and misleading [244] [145] [689].Applications include explanation-guided prompting, intermediate computation steps, supervised attention, and rationale regularization for out-of-distribution generalization.Source: §10.5.1 · p. 77, §10.5.2 · p. 77

10.6 Explainability of MLLMs

  • MLLM explainability mainly studies CLIP-based image-text alignment, while explanations for LLM-based image and video understanding models remain underexplored [1071] [1209] [570].Agarwal finds explanations for zero-shot CLIP classifiers more faithful than those for fine-tuned versions.Source: §10.6 · pp. 77–78

10.7 Current Limitations and Future Directions

  • Future work should improve raw-feature faithfulness, clarify how knowledge is stored and triggered, and develop robust transferable cross-modal explanations with standardized benchmarks [131] [415].Gradient attributions can have superficial faithfulness and low robustness to perturbations, while the credibility of attention-based explanations remains contested.Source: §10.7.1 · p. 78, §10.7.2 · p. 78, §10.7.3 · p. 78

10.7.4 Reliability and Responsibility of Foundation Models from the Explainability Perspective

  • Sanity checks found some saliency methods independent of model and data [8], while small input perturbations can substantially change explanations [24].These findings question the validity and robustness of interpretability methods.Source: §10.7.4 · p. 79
  • Identifying training-data biases and applying de-biasing strategies could yield more equitable models [534].Source: §10.7.4 · p. 79
  • Understanding how models store knowledge can support model editing and harmful-information unlearning, helping produce safer, up-to-date foundation models [947] [1132].Source: §10.7.4 · p. 79
  • Mismatches between interpretability-tool capabilities and data scientists’ needs can limit their effectiveness, motivating tools designed around practical user requirements.Source: §10.7.4 · p. 79

11 AIGC Detection

AIGC detection classifies content as human- or AI-generated and encompasses zero-shot, watermark-based, and neural-network methods. The survey also highlights challenges involving detector fairness, watermark robustness, and image-origin attribution.

11.1 The AIGC Detection Problem

  • AIGC detection is a binary classification task, and its detectors are grouped into zero-shot, watermark-based, and learnable methods.The positive class includes content generated wholly or partially by AI; the negative class is human-created content.Source: §11.1 · p. 80

11.2 Zero-shot Detectors

  • Zero-shot detectors use intrinsic content cues without additional training, but statistical methods can require generator probabilities and intuitive cues are harder to rely on as models improve.Statistical detectors use measures such as perplexity and token probabilities; these approaches can falter as generated content becomes more diverse and higher quality.Source: §11.2 · pp. 80–81, §11.2.1 · p. 82, §11.2.2 · p. 83

11.2.3 Pre-trained LLMs

  • Pre-trained LLMs often perform poorly at zero-shot detection, while in-context-learning results are mixed and retrieval-based detection raises privacy concerns.Some ICL results outperform traditional zero-shot and RoBERTa-based detectors, but another study finds no significant improvement with GPT-3.5-turbo; retrieval can resist paraphrasing but requires storing generated text.Source: §11.2.3 · p. 83

11.3.1 Training-free Watermarking

  • Training-free watermarks encode detectable signals through content transformations or generation-time token choices, but may be conspicuous, removable, or damaging to content quality.Watermarks can indicate AI origin or carry additional information; desirable properties include easy verification, imperceptibility, minimal quality impact, and robustness to attacks.Source: §11.3 · pp. 83–84, §11.3.1 · p. 84

11.3.2 Learnable Watermarking

  • Learnable watermarking embeds keyed payloads during generation and uses learned encoders or logits to balance semantic consistency, detectability, and robustness.Unlike post-hoc classifiers, these methods assume an intentionally embedded signal; publicly verifiable schemes aim to avoid exposing watermark-generation keys.Source: §11.3.2 · p. 85

11.4 Neural Network Detectors

  • Neural detectors train post-hoc classifiers that can work with unknown generators, but data scarcity, paraphrasing, and training-data overfitting undermine generalization.Approaches include fine-tuning pretrained LLMs and classifying hand-crafted features without test-time access to token logits; performance can decline across domains or for outputs from different models.Source: §11.4 · pp. 85–86

11.5.1 Fairness of AIGC Detection

  • Detection and learnable watermarking require fairness evaluation, while watermarks must resist both removal or post-processing and adversarial attacks.Perplexity-based detectors can misclassify non-native writers, and watermark models trained on native English may misclassify non-native text; semantic watermarks can resist some transformations but remain vulnerable to adversarial attacks.Source: §11.5.1 · p. 86, §11.5.2 · p. 86

11.5.3 Origin Attribution of Generated Images

  • Improved visual generators make identifying the source model of an image an important challenge, especially when model access is restricted and only few source images are available.Source: §11.5.3 · p. 86

12 Intersection and Conclusion

The survey finds progress in reliable and responsible foundation models but emphasizes that unresolved challenges remain as their societal impact grows. Because the areas are interconnected, it advocates addressing them holistically rather than in isolation.

12.1 Summary of Recent Community Efforts

  • Community surveys cover complementary reliability and responsibility perspectives, advancing shared terminology, taxonomies, and evaluation practices while underscoring the need for both method innovation and standardized assessment.Examples include reviews of hallucination and grounding, cross-modal syntheses, safety at scale [620], trustworthiness benchmarking [1157], and explainability.Source: §12.1 · p. 87, Table 3 · p. 87

12.2 Bias, Fairness, and Security

  • Adversarial training can improve security yet worsen fairness, while data poisoning can introduce or exacerbate bias, creating a dual challenge for model robustness and equitable performance.Robust models may disproportionately degrade performance for underrepresented groups; hate-speech systems have misclassified non-offensive African American English as hate speech.Source: §12.2 · pp. 87–88

12.3 Bias, Fairness, and AI-generated Content Detection

  • Biased training data can skew generated images, and AIGC detectors or inconsistently applied watermarks can unfairly flag or suppress content associated with minority groups.Zero-shot or neural detectors may mark minority-created content as AI-generated, while watermarks concentrated on content associated with particular ethnic or gender identities can make that content easier to block.Source: §12.3 · p. 88

12.4 Security and Privacy

  • Protecting private information is both a security and legal challenge: unlearning, prompts, and RLHF can be circumvented, and deleted information may remain recoverable under repeated queries.The GDPR’s Right to be Forgotten requires users to be able to delete their private information [1132].Source: §12.4 · p. 88

12.5 Security and AI-generated Content

  • AIGC detectors remain vulnerable to adversarial attacks [777] [26], while malicious AIGC also threatens public institutions and individuals, requiring safeguards at larger scale.Reported risks include political and health misinformation [231], mental-health harms from hyper-personalization, and personalized fraud; proposed safeguards include public education and strategies against misuse.Source: §12.5 · pp. 88–89

12.6 Uncertainty and Alignment

  • Alignment methods can train models to express uncertainty or abstain, but future work must address unknown unknowns and improve models’ ability to resolve uncertainty.Approaches use internal-state control codes, a smaller model to predict a larger model’s uncertainty [908], or speaker-listener supervision [853].Source: §12.6 · p. 89

12.7 Hallucination, Uncertainty, Distribution Shift, and Alignment

  • Out-of-distribution inputs can trigger overconfident errors and hallucinations, while shifting multimodal data can cause forgetting and disrupt image-text alignment, motivating memory retention and continual learning.Unfamiliar contexts can make prior knowledge inapplicable; changing language or visual context can weaken links between textual descriptions and visual cues.Source: §12.7 · p. 89

Related papers

Loading 2602.08145v1…