Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs

1 Introduction

• CoT breaks complex problems into intermediate subtasks, which can improve reasoning performance and make LLM reasoning easier to interpret.The introduction describes these benefits across a wide range of reasoning tasks.Source: §1 · p. 1
• CoX methods extend beyond reasoning thoughts to components such as feedback, instructions, and histories, and address tasks including multimodal interaction, hallucination reduction, and planning.Examples include Chain-of-Feedback, Chain-of-Instructions, and Chain-of-Histories.Source: §1 · p. 1
• The survey fills a collective-review gap by organizing CoX methods according to their chain components and the tasks they address.It also discusses findings, implications, and potential future directions.Source: §1 · p. 1

2 What is Chain-of-X?

• CoT prompting uses an input, intermediate thoughts, and an output, with reasoning steps leading to the final answer.The paper notes that CoT is most effective on tasks requiring complex reasoning.Source: §2 · p. 2
• Chain-of-X generalizes CoT to diverse tasks beyond reasoning, with nodes that can include intermediates, augmentation, feedback, or models.These node types are illustrated in Figure 1.Source: §2 · p. 2, Figure 1 · p. 2
• A CoX chain has the format <input, X1, . . . , Xn, output>; its components contribute compositionally to a solution or iteratively refine outputs.The paper says this format can extend beyond prompting to other algorithmic frameworks or structures.Source: §2 · p. 2
• CoX methods span tasks including multimodal interaction, factuality and safety, reasoning, instruction following, agents, and evaluation tools.The paper gives Chain-of-Verification as an example for revising responses through verification questions.Source: §2 · p. 2

3 Chain-of-X Nodes

• The section classifies CoX by node type, with intermediates, augmentation, feedback, and models as the principal categories.The survey also depicts representative methods within its node-and-task taxonomy.Source: §3 · p. 2, Figure 2 · p. 3

3.1 Chain-of-Intermediates

• Chain-of-intermediates generalizes reasoning steps into subtasks or accumulated information, including decomposition for reasoning and other tasks.Examples include event extraction for summarization and question-specific table operations; visual methods also focus on relevant regions or compose scene-graph knowledge.Source: §3.1 · pp. 2–3

3.2 Chain-of-Augmentation

• Chain-of-augmentation methods add knowledge or instructions to chains; Chain-of-Instructions uses previous-step outputs as instructions and reports improved complex problem-solving outcomes.Retrieval methods can be affected throughout the chain by a misleading sub-question; Chain-of-Query can revisit retrievals and adjust reasoning direction after verification.Source: §3.2 · pp. 4–5

3.3 Chain-of-Feedback

• Chain-of-feedback interlaces external or self-generated feedback with generation to refine outputs, including human preference feedback and iterative self-review.Self-Refine uses the same LLM for initial responses, feedback, and revision; Chain-of-Verification instead plans and answers verification questions.Source: §3.3 · pp. 5–6

3.4 Chain-of-Models

• Chain-of-models assigns sequential work to specialized LLMs or networks, while multi-model dialogue methods critique and refine contributions before consensus.The survey describes these approaches for intricate operations-research problems and broader problems divided across domains.Source: §3.4 · p. 6

4.1 Multi-Modal Interaction

• CoX methods address multimodal interaction across text-image, text-table, text-code, and text-speech tasks.Examples include visual reasoning chains, evolving tables, compiler-feedback code repair, and separating and reassembling speech components.Source: §4.1 · pp. 6–7

4.2 Factuality & Safety

• For factuality, methods either verify and refine initial answers or retrieve domain knowledge; alignment methods use preference sequences and red-teaming conversations.Chain-of-Hindsight is reported to outperform RLHF on alignment; Chain-of-Utterance exposes safety issues.Source: §4.2 · p. 7

4.3 Multi-Step Reasoning

• CoX is applied to multi-step reasoning in rule-based, database, legal, user-behavior, graph, summarization, and translation tasks.The survey attributes these applications to CoX’s sequential structure for breaking complex tasks into connected steps.Source: §4.3 · p. 7

4.4 Instruction Following

• Instruction-following methods prompt or tune with instruction sequences, while Chain-of-LoRA routes instruction types to specialized LoRA networks.The latter is presented as an alternative to the cost of training one LLM to follow different instructions.Source: §4.4 · pp. 7–8

4.5 LLMs as Agents

• Agent-focused CoX methods guide planned actions and augment agents with history, model feedback, human-scene interaction, tools, or multi-agent discussion.The survey presents these approaches as supporting autonomous and collaborative agent capabilities.Source: §4.5 · p. 8

4.6 Evaluation Tools

• CoX methods can evaluate LLMs by exposing safety issues, backdoor vulnerabilities, and response-quality declines after repeated non-informative prompts.The examples include harmful-model interactions, unintended malicious outputs under backdoor attack, and repeated “make another attempt” prompts.Source: §4.6 · p. 8

Causal Analysis on Intermediates

• Causal analysis is needed to understand how intermediate reasoning steps affect final results, since models may skip rational steps and gains can occur with invalid rationales.These observations motivate examining the causal role of intermediate steps rather than focusing only on task-specific generated results.Source: §5 · p. 8

Reducing Inference Cost

• CoX chains can require costly, time-consuming sequential inference, motivating shorter chains or executing intermediate nodes in parallel or jointly.The survey presents these as future research directions.Source: §5 · p. 8

Knowledge Distillation

• Whether intermediate nodes from broader CoX methods are as useful for student learning as CoT rationales remains open.The survey notes that prior studies show students can learn from LLM-generated CoT rationales.Source: §5 · p. 8

End-to-End Fine-tuning

• Without enforced self-correction, generation errors may accumulate along CoX chains because the paradigm is not end-to-end.The survey proposes exploring CoX fine-tuning that penalizes errors in final outputs.Source: §5 · p. 8

• The survey examines Chain-of-X methods as extensions built upon Chain-of-Thought.Source: §6 · p. 8
• It categorizes CoX methods by nodes and tasks to provide a comprehensive overview.Source: §6 · p. 8
• The overview highlights CoX’s potential to enhance LLM capabilities and opens avenues for future research.Source: §6 · p. 8

• The survey’s collected works may not include every relevant study.The authors say they made their best efforts to include studies leveraging CoX, whether or not explicitly named as such.Source: Limitations · p. 9
• The rapidly growing number of works makes omissions possible.Source: Limitations · p. 9
• The authors welcome community suggestions and plan to continue updating the collection.Source: Limitations · p. 9

A Taxonomies of Nodes and Tasks

• Figure 4 presents the complete version of Figure 2, which categorizes Chain-of-X by nodes and tasks.The two taxonomies provide the organizing dimensions for the figure.Source: A Taxonomies of Nodes and Tasks · p. 14, Figure 4 · p. 15