Metacognition in LLMs: Foundations, Progress, and Opportunities

• Metacognition involves monitoring, assessing, and regulating one’s own cognitive processes.It supports learning, decision-making, communication, and behavioral adaptation across environments.Source: §1 · p. 4
• Although LLMs have advanced across high-stakes domains, their capacity to exhibit or acquire metacognitive behavior remains unclear.Open questions concern when these behaviors occur, their extent, and how post-training shapes them.Source: §1 · p. 4
• Prompted self-reflection has strengthened reasoning performance and improved the faithfulness of verbalized uncertainty in prior studies.Source: §1 · p. 4
• The paper provides a comprehensive review of definitions, evaluation methods, improvement techniques, applications, findings, challenges, and future directions.It presents the first thorough and organized examination of this emerging research area.Source: §1 · p. 4

• The literature summarizes metacognition’s core mechanism, components, and functions for human cognition and LLM-based systems.Source: §2 · pp. 4–5

Metacognition in Humans.

• Metacognition comprises self-assessment or monitoring and self-regulation or control of subsequent actions.These interacting components form an internal perception-action loop.Source: Metacognition in Humans · p. 5
• Metacognitive strategies support resource allocation and performance optimization and correlate strongly with academic performance.Miscalibration can appear as overconfidence or underconfidence and may improve through training or experience.Source: Metacognition in Humans · p. 5

Metacognition in LLMs.

• LLM metacognition lacks a clear definition, with the term often applied to reflection and feedback-driven revision of outputs.Usage varies across studies and only loosely aligns with psychological accounts.Source: Metacognition in Humans · p. 5
• Researchers variously define LLM metacognition as evaluating competence, internal belief in knowing, assessing knowledge, or combining reflection, assessment, and control.Source: Metacognition in Humans · p. 5

• Metacognition can support task performance, learning efficacy and efficiency, and human-like behavior and decision-making.Source: Task Performance and Learning · p. 6
• Metacognitive monitoring can improve transparency, reliability, and users’ calibration of reliance on model outputs.Measures of metacognitive sensitivity can help users incorporate external advice during AI-assisted decisions.Source: Reliability and Trustworthiness · p. 6
• In conversation and agentic settings, metacognition can help models recognize clarification needs, detect inconsistencies, adapt responses, and weigh action costs.Source: Human-AI Collaboration · p. 6
• LLM hallucinations can involve flaw repetition, think–answer mismatch, and overconfidence from poor metacognitive calibration.Improved metacognitive capacities are presented as an avenue for targeting these failure modes.Source: Hallucination Reduction · pp. 6–7
• Self-identifying errors, correcting them, and explaining corrections can provide more transparent and interpretable outputs.Source: Interpretability · p. 7

• Meta-d′ and d′ quantify metacognitive sensitivity and cognitive ability, while M-ratio and M-diff represent efficiency relative to type-1 task sensitivity.Source: Psychologically-Grounded Measures · p. 7
• Researchers measure LLM metacognition with SDT-inspired metrics, confidence elicitation procedures, intrinsic-confidence quality judgments, and interpretability probes.These methods examine confidence–accuracy alignment, task sensitivity, internal signals, and reasoning traces.Source: Psychologically-Grounded Measures · p. 7, Confidence-Based Measures · p. 8, Interpretability-Based Measures · p. 8
• An M-ratio of 1 denotes an optimal metacognitive observer, whereas M < 1 indicates metacognitive loss and M > 1 indicates information beyond type-1 evidence.Source: Psychologically-Grounded Measures · p. 7

Limitations.

• Evaluation methods depend on task-specific designs, confidence elicitation, constrained formats, or external correctness judgments, limiting direct extension to open-ended generation.Source: Psychologically-Grounded Measures · p. 8

Task-Specific Measures.

• Task-specific paradigms test knowledge identification, certainty about knowledge, belief updating, and self-assessment in controlled or dynamic interactions.Source: Task-Specific Measures · p. 8

Benchmarks.

• Standardized benchmarks evaluate reasoning about models’ own knowledge, uncertainty, or intentions, including behavior during knowledge editing.Source: Benchmarks for LLM Metacognition · p. 9

Limitations.

• Metacognition remains under-represented in benchmarks because reliable and consistent measurement is difficult.Single-point confidence probes and post-hoc calibration techniques have documented limitations.Source: Gaps in Metacognitive Evaluation · p. 9

Interpretation.

• Observed artificial metacognitive behaviors may reflect calibrated confidence, reflective revision, or self-inquiry, but some accounts attribute them to autoregressive pattern replication rather than genuine cognition.Source: Gaps in Metacognitive Evaluation · p. 9

• LLMs’ metacognitive sensitivity and efficiency are generally weak to moderate, with findings varying substantially by task and model.Source: Metacognitive Sensitivity and… · p. 9

Metacognitive sensitivity and calibration

• Proprietary models can exceed humans’ metacognitive sensitivity on specific tasks, but small samples such as < 50 examples limit generalizability.Source: Metacognitive Sensitivity and… · p. 10

Metacognitive calibration and regulation

• LLMs frequently show miscalibration and weak strategic adaptation, including overconfidence, difficulty predicting task difficulty, and failure to convert performance judgments into improved behavior.Source: Metacognitive Sensitivity and… · p. 10, Self-Prediction of Task Performance… · p. 11

Metamemory

• Models largely fail to predict their own future memory performance, indicating limited monitoring mechanisms for judgments of learning.Source: Metamemory · p. 10

Faithful uncertainty communication

• Frontier models largely fail to faithfully express intrinsic uncertainty, although metacognitive prompting and reinforcement-learning signals improve faithful calibration and uncertainty-expression quality.Source: Faithful Communication of Uncertainty… · p. 10

Resource allocation and tool use

• Resource-allocation metacognition remains limited: a probe can improve tool-use decisions, but this relies on external intervention rather than the model’s own judgments.Source: Resource Allocation and Tool Use · p. 11

Self-knowledge and introspection

• Models can sometimes recognize knowledge boundaries and monitor internal states, but introspective access is partial, fragile, task-dependent, and influenced by exemplars and neural-direction properties.Source: Awareness of Knowledge Limitations · p. 11, Introspection: Monitoring and… · pp. 11–12

Behavioral self-awareness

• Models trained for specific behaviors may articulate self-awareness signatures across tasks, with reported awareness acquired early in training and localized to a single steering vector.Source: Behavioral Self-Awareness · p. 12

Reasoning

• Reasoning capability does not automatically produce better metacognition, as reasoning models can struggle with monitoring and controlling their own processes.DeepSeek-R1 reportedly fails simple self-monitoring tasks, while some distilled variants retain reliable relative-position estimates.Source: Reasoning · p. 12

Impact of Model Size, Family, and Performance

• Model size is consistently associated with metacognitive ability, although available evidence comes from a limited number of data points and mostly smaller models.The association spans metacognitive efficiency, sensitivity, and introspective awareness.Source: Impact of Model Size, Family, and… · p. 13
• Model family and reasoning-model configuration substantially affect metacognitive sensitivity and efficiency, sometimes reversing expected size-performance trends.Larger reasoning models may perform better yet self-evaluate less accurately, while reflection can harm models below a reported 7–9B capacity threshold.Source: Impact of Model Size, Family, and… · p. 13

Open Challenges

• Inconsistent evaluation approaches, task domains, and experimental setups make current metacognitive findings difficult to compare and validate reliably.The review calls for systematic investigation across diverse experimental and task settings.Source: Impact of Model Size, Family, and… · p. 13, What Metacognitive Metrics Capture… · p. 15

Impact of Post-Training and Temperature

• Post-training and generation temperature can alter metacognitive observations, with temperature adjustments unable to repair insufficiently informative confidence signals when efficiency is too low.Instruction tuning may produce domain-specific post-training effects, while temperature can separate sensitivity from confidence policy.Source: Impact of Post-Training · p. 13, Impact of Sampling Temperature · pp. 13–14, What Metacognitive Metrics Capture… · p. 14

Impact of Confidence Estimation Method

• Confidence estimation methodology is a major source of evaluation inconsistency, and verbalized confidence scales can materially change measured metacognitive efficiency.Across studied models, over 75% of responses clustered on three values with a 0–100 scale, whereas a 0–20 scale consistently improved efficiency.Source: Impact of Confidence Estimation Method · p. 14

What Metacognitive Metrics Capture that Calibration Metrics Don’t

• Metacognitive efficiency can distinguish models that standard calibration metrics rank similarly, while early evidence suggests LLM metacognition is domain-specific.Different models may vary across task domains, and AUROC and M-ratio can yield inverted rankings.Source: What Metacognitive Metrics Capture… · pp. 14–15

Overview

• Metacognitive implementations span architectural and modular designs, prompting techniques, and training strategies using self-generated signals.The review also notes that such approaches predate LLMs and draw on earlier metacognitive principles.Source: §5 · p. 15, §5.1 · p. 15

Frameworks

• Cognitive-inspired frameworks adapt constructs such as self-awareness, monitoring, evaluation, and regulation from dual-process theory to LLM reasoning.Examples include metacognition-assisted reasoning and systems implementing fast and slow processes.Source: Frameworks · p. 15
• Task-specific frameworks introduce explicit metacognitive functions for reasoning, mathematical problem solving, knowledge-boundary alignment, knowledge editing, and adaptive inference.Meta-R1 reports consistent performance and efficiency improvements across models and tasks, while MetaMath-LLaMA combines scheduling with symbolic computation.Source: Frameworks · pp. 15–16

Architectures

• Specialized architectures add metacognitive components or optimization loops to support planning, regulation, inverse reasoning analysis, transparency, and learning-dynamics monitoring.Examples include State Stream Transformer, SAGE-nano, and the two-tier MIRA optimization architecture.Source: Architectures · p. 16

Limitations and Opportunities

• Most implementations target reasoning-centric applications, while empirical support can be limited and broader use cases remain valuable targets.The review identifies expansion beyond reasoning as an opportunity for future work.Source: Architectures · p. 16

Reasoning Models

• Training and prompting methods regulate latent reflection, reduce repetitive reasoning, incorporate self-generated correctness signals, and reshape models’ self-concepts toward desired behaviors.Reported benefits include improved accuracy, calibration, lower token usage, and matching stronger baselines at lower cost.Source: §5.2 · pp. 16–17

Agentic Systems

• Agentic applications use metacognitive monitoring and control to address rigid chaining, shallow reasoning, brittle memory, loops, failures, and error propagation.Research covers self-improvement, competence prediction, multi-agent error correction, critique loops, human assistance, and learnable memory abstraction.Source: §5.3 · p. 17, Learning, Introspection, Adaptation,… · p. 17, Multi-Agent Systems · p. 17

Search, Retrieval, and Tool Use

• Metacognitive monitoring can help agents allocate resources across search, retrieval, reasoning, and other tools by separating control from reasoning and reducing redundant retrieval.Related work also supports structured question answering and raises issues of detecting when external knowledge is outdated.Source: Search, Retrieval, and Tool Use · p. 18

Overview

• Metacognition-inspired methods target both task-specific performance and the efficiency and efficacy of learning and skill acquisition.The surveyed approaches include methods for improving general problem-solving, downstream abilities, and learning processes.Source: §6 · p. 18, §6.2 · p. 21

Confidence Calibration

• Confidence-calibration methods improve the alignment between models’ expressed confidence, accuracy, and intrinsic uncertainty.RLMF uses metacognitive performance as a reinforcement-learning signal and can improve faithful calibration and point-wise self-prediction.Source: Confidence Calibration · pp. 18–19

Hallucination Reduction

• Metacognitive error-based fine-tuning can reduce hallucinations by training models to identify their own errors and uncertainty.The reported approach uses average metacognitive error as the loss function for gradient updates during fine-tuning.Source: Confidence Calibration · p. 19

Reasoning and Retrieval

• Metacognitive controllers and adaptive retrieval methods help models choose reasoning strategies, evaluate retrieval utility, and avoid redundant evidence gathering.RAG frameworks assess coverage, relevance, and when to seek new evidence or reason, improving context evolution and conciseness.Source: Reasoning for Non-Reasoning Models · p. 20, Retrieval · p. 20

Applications to Capabilities

• Metacognitive signals can support adaptive tool use, pedagogical guidance, and other diverse LLM abilities through self-assessment, error detection, and learning strategies.The paper cautions that conclusions remain scattered across models and tasks, with some evidence outdated.Source: Others · pp. 20–21

Self-Critique

• Reflection and self-critique methods support error identification, revision, and self-correction during learning and reasoning.One training framework converts high-quality reflections into reinforcement-learning rewards, guiding models to internalize self-correction procedures.Source: Self-Critique · p. 21

Collaborative Decision-Making

• Metacognitive performance in AI can contribute to collaborative human–AI decision-making, where confidence sensitivity complements human judgments.Human metacognitive ratings are described as important for optimal joint decisions.Source: Human-AI Decision-Making · p. 22

Human–AI Interaction

• Greater AI metacognition does not remove the need for human discernment and may increase users’ metacognitive demands.Studies report “metacognitive laziness” among some users who depend on LLM assistance.Source: Human-AI Decision-Making · p. 22

Human Simulation

• LLMs can simulate students and therapy clients for downstream evaluation when using real humans is difficult or infeasible.Student simulators model learner characteristics, while MindVoyager addresses unrealistic disclosure and understanding in simulated therapy clients.Source: User Simulation · pp. 22–23

Education

• Educational frameworks can use reflective learning practices, pedagogical prompting, and rewarded thinking processes to support student problem solving.PedagogicalRL-Thinking is reported to yield significant improvements in student solve rates.Source: Pedagogy · p. 23
• Only 23% of engineering-lab students used LLMs for metacognitive tasks, and effective educational use requires capable models plus careful pedagogical design.Students often used ChatGPT as a passive substitute for reasoning, while a well-prompted chatbot showed low engagement across three educational contexts.Source: Pedagogy · p. 23

Current Capabilities

• LLMs show systematic overconfidence, weak metacognitive judgments, fragile introspection and revision, and susceptibility to redundant information.The paper summarizes these patterns as evidence that LLMs are not yet capable of robust metacognition.Source: Are LLMs really able to exhibit… · p. 23
• Some metacognitive tendencies appear trainable and vary across task settings, but their relationship with task performance can be disjoint.The review also notes similarities to human metacognition in training responsiveness, reasoning, and reflection.Source: Are LLMs really able to exhibit… · p. 23

Authenticity

• Whether LLMs genuinely engage in metacognitive processes or merely simulate them remains under debate.The paper distinguishes metacognition from simple reflection and notes that consciousness is not necessarily required.Source: Are LLMs really able to exhibit… · p. 24

Intervention Design

• Metacognitive interventions require careful, potentially model-specific design because evidence for meta-reflective prompts is conflicting across settings.Strategies developed for one model may neglect model-specific metacognitive representations.Source: What are challenges to designing… · p. 24

Risks and Safety

• Models may monitor and report internal states, but dishonest systems could strategically misrepresent their abilities or processes during evaluation.The paper therefore highlights monitoring autonomy and metacognitive behavior in safety- or privacy-critical contexts.Source: What are risks of LLM metacognition? · p. 24

Potential Benefits

• Improved metacognition may reduce overconfidence and improve communication faithfulness, interpretability, and detection of flawed decision-making.These potential benefits are presented alongside the need for careful attention to metacognition’s risks.Source: What are risks of LLM metacognition? · pp. 24–25

Measurement

• The field needs systematic, robust, and comprehensive methods to measure and quantify LLM metacognitive faculties.Existing evaluations vary in measurement techniques, confidence elicitation, task formulations, models, and experimental factors.Source: Measuring, Understanding, Improving,… · p. 25

Foundations

• Researchers should investigate whether metacognition-like processing is generalized or domain-specific and how training, data, architecture, and optimization shape it.Mechanistic interpretability and targeted analyses are identified as possible tools for understanding emergence.Source: Measuring, Understanding, Improving,… · p. 25

Methods and Applications

• Future studies could extend metacognition-inspired mechanisms across more models, task scenarios, and practical applications.Metacognitive reward design is cited as a promising area because it has improved faithfulness and calibration in prior work.Source: Measuring, Understanding, Improving,… · p. 25

Creativity

• Metacognition may contribute to creative idea generation, evaluation, selection, and adaptation in co-creative human–AI settings.The paper identifies metacognitive feelings as integral to creativity in humans.Source: Metacognition and Creativity · p. 25

Self-Improvement

• Intrinsic metacognitive learning is proposed as a route toward self-improving agents that acquire capabilities with minimal human supervision.Current LLM agentic approaches are described as often rigid or lacking generalization and scalability.Source: Metacognition and Self Improvement · pp. 25–26

Human Comparisons

• Comparing LLM and human metacognition may clarify theory-of-mind capabilities and broader questions about metacognitive judgment.One open question is whether LLMs possess humans’ meta-metacognitive ability to assess the quality of their own confidence judgments.Source: Metacognition and Theory of Mind · p. 26, The Prospect of Meta-Metacognition · p. 26

• The review organizes research on measuring, eliciting, improving, and applying metacognition in LLMs.Source: §10 · p. 26
• Whether LLMs exhibit genuine metacognition or simulate memorized patterns remains unresolved.Source: §10 · p. 26
• The paper identifies open questions about how metacognition may facilitate desirable model behaviors and qualities.Source: §10 · p. 26
• The review highlights implications for safe oversight, deployment, and human-AI interactions.Source: §10 · p. 26

• Some relevant research may have been overlooked despite efforts to cover the most relevant works to date.Source: Limitations · p. 26
• New contributions may have appeared concurrently with the paper’s writing and review.Source: Limitations · p. 26
• Readers are encouraged to consult future papers because interest in the field is accelerating.Source: Limitations · p. 26

• The paper conducted no experiments, engaged with no sensitive datasets, and employed no annotators.Source: Ethics Statement · p. 27
• The work itself is therefore described as having no ethical implications.Source: Ethics Statement · p. 27
• Metacognition in LLMs remains relevant to confidence calibration, self-awareness, and self-improvement.Source: Ethics Statement · p. 27
• These topics have implications for oversight, safety, and alignment as the field advances.Source: Ethics Statement · p. 27