CWM: Controllable White-Box Meta-Prompting for Adaptive Retrieval-Augmented Generation and Reasoning Ability

1 Introduction

• Existing adaptive RAG methods often depend on task-specific classifiers or databases, use multiple inference calls, and are evaluated mainly on knowledge-intensive tasks.The resulting impact on general prompt processing, including non-knowledge-intensive tasks, remains insufficiently analyzed.Source: §1 · pp. 1–2
• CWM uses query Self-Perplexity as a pre-generation signal to select a retrieval-decision module within meta-prompting, without external decision modules or repeated inference.The approach applies to both RAG and reasoning tasks.Source: §1 · p. 2
• CWM achieves state-of-the-art performance on adaptive RAG benchmarks and consistently outperforms Self-Discover on reasoning benchmarks.The approach also enables fine-grained retrieval control through hyperparameter settings for different models and tasks.Source: §1 · p. 2
• The paper introduces a controllable framework using white-box signals for task- and model-specific prompt processing, alongside a unified evaluation of adaptive RAG and reasoning tasks.The evaluation is intended to support a more general analysis of prompt-processing strategies.Source: §1 · p. 2

2 Related Work

• Adaptive RAG determines whether to retrieve documents based on query complexity, with prior approaches using complexity judgments, classifiers, or consistency-based detection.Adaptive-RAG classifies queries as straightforward, simple, or complex; Rowen uses a consistency-based detection module.Source: §2 · p. 2
• CWM’s overview depicts query self-perplexity informing a retrieval decision in an atomic reasoning module before response generation conditioned on the meta-prompt and context.Source: Figure 2 · p. 3
• SEAKR uses uncertainty in a generated rationale to decide whether external knowledge retrieval is required.Its method generates a brief rationale for each query.Source: §2 · p. 3
• Meta-prompting approaches such as Self-Discover construct task-specific reasoning prompts from atomic modules, but have primarily been studied for reasoning tasks.Whether LLM meta-prompting is effective for adaptive RAG had not been systematically explored.Source: §2 · p. 3

3 Method

• CWM adds a Retrieval decision prompt to Self-Discover and uses query Self-PPL as a pre-generation signal to guide retrieval selection within meta-prompting.Source: §3 · p. 3

3.1 Stage 1: Compute Self-Perplexity of the Input Query

• Higher Self-PPL marks greater uncertainty and difficulty; a thresholded mapping produces signed α, with easy queries suppressing and difficult queries emphasizing retrieval modules.Source: §3.1 · p. 4

3.2 Stage 2: Meta-Prompt Generation

• During meta-prompt generation, CWM locates the predefined retrieval-keyword span and transforms its embeddings in place, preserving sequence length.Source: §3.2 · p. 4
• A directional perturbation complements multiplicative scaling, and gdelta is applied during decoding as an additive bias to retrieval-keyword embeddings.Source: §3.2 · p. 4
• Table 1 reports CWM’s best and mean performance across hyperparameter settings and RAG O, the percentage of instances judged to require retrieval.Source: Table 1 · p. 5

4.1 Benchmarks & Models

• Evaluation covers HotpotQA, MuSiQue, StrategyQA, selected BBH subtasks, T4D, and MATH500 on GPT-oss-20b, Qwen3-14b, and Llama3.1-8b.One hundred randomly sampled instances were used per dataset.Source: §4.1 · p. 5

4.2 Baselines

• Adaptive RAG comparisons include Adaptive-RAG, Rowen-CL, SEAKR, a Self-Discover-based retrieval baseline, and gold-context upper and lower bounds.Source: §4.2 · p. 5

4.3 Implementation Details

• Outputs use greedy decoding with a 1024-token limit; when retrieval is required, baselines and CWM receive the same gold context for answer generation.Experiments used four NVIDIA RTX A5000 GPUs.Source: §4.3 · pp. 5–6

4.1 Benchmarks & Models

• Figure 3 measures Self-PPL across the three RAG and three reasoning benchmarks for GPT-oss-20b, Qwen3-14b, and Llama-3.1-8b.Source: Figure 3 · p. 6

5.1 Results for Adaptive RAG task

• Across all adaptive RAG experiments, CWM achieves the best performance, outperforming Self-Discover and using significantly lower computational cost than classifier- or multi-sampling-based methods.Source: §5.1 · p. 6
• CWM stays above the w/o GC lower bound in all cases, surpasses w/ GC for two model–benchmark pairs, and has the highest RAG O ratios on MuSiQue.The w/ GC cases are HotpotQA with Qwen3-14b and MuSiQue with Llama-3.1-8b.Source: §5.1 · p. 6

5.2 Empirical Validation of Self-PPL

• Increasing delta raises retrieval usage for GPT-oss-20b and Llama3.1-8b, while Qwen3-14b consistently uses retrieval across cases.Source: Figure 4 · p. 7, §5.2 · p. 7
• RAG benchmarks consistently show higher Self-PPL than reasoning benchmarks, while question-group patterns vary between HotpotQA and MuSiQue.In HotpotQA, Bridge and Comparison questions show no consistent association with high or low Self-PPL; in MuSiQue, high-hop questions more often have high Self-PPL, but many low-hop questions do too.Source: §5.2 · p. 7
• CWM achieves the best performance across methods on samples where retrieval is not triggered.Source: §5.2 · p. 7, Table 3 · p. 8

5.4 Analysis for Efficiency

• Adaptive-RAG’s Straightforward path has the lowest inference time and fewest calls, whereas Rowen-CL’s repeated reasoning calls increase latency and limit real-time suitability.Adaptive-RAG’s reported test-time figures exclude silver-label inference and classifier training (+α); Self-Discover and CWM use three inference calls for prompt selection, meta-prompt generation, and the final answer.Source: §5.4 · pp. 8–9

5.3 Results for Reasoning task

• CWM consistently outperforms Self-Discover across reasoning models and benchmarks.The comparison is reported in Table 4; M_AVG denotes the average number of selected atomic prompt modules.Source: §5.3 · p. 7, Table 4 · p. 8

5.4 Analysis for Efficiency

• CWM computes only input-query perplexity with minimal overhead, whereas SEAKR’s hidden-state sampling incurs substantially higher inference time.CWM achieves efficient performance without increasing cost compared with the black-box Self-Discover method.Source: §5.4 · p. 9, Table 5 · p. 9

• CWM performs strongly on reasoning and adaptive RAG tasks with recent open-source LLMs while reducing computational cost compared with methods relying on external components or multi-sampling.Source: §6 · p. 9
• CWM controls retrieval decisions using internal model signals derived from the input query rather than signals obtained during generation.Source: §6 · p. 9
• Atomic reasoning prompts also support more general prompt selection beyond retrieval.Source: §6 · p. 9

• The experiments add one Retrieval prompt to 39 existing modules but do not test Self-Perplexity control of the other 39 prompts.Source: Limitations · p. 9
• CWM requires token-level likelihoods and embedding-level representations, limiting direct use with API-only models; it remains applicable to self-hosted open-weight LLMs with accessible internal signals.Source: Limitations · p. 9
• Self-Perplexity can be low for recent-information queries that need retrieval or high for ambiguous phrasing that does not need retrieval.Source: Limitations · p. 9

• The experimental atomic reasoning library contains 40 high-level cognitive heuristic modules.Source: Table 6 · p. 14
• The retrieval prompt is Module 1 in the library.Source: Table 6 · p. 14
• Modules 2–40 are adapted from Fernando et al. (2024).Source: Table 6 · p. 14

A.1 Meta-Prompting Details

• Self-Discover and CWM select modules from the atomic library to construct a meta-prompt, which is then used to generate the final answer.Source: §A.1 · p. 15

A.2 Method Details

• CWM injects a perplexity-derived control signal into retrieval-related keyword embeddings using clamped functions of the control coefficient and a directional term.Source: §A.2 · p. 15

A.2.1 Normalization Layers and Embedding Scaling

• With LayerNorm, residual connections and affine parameters allow pre-norm embedding scaling to affect downstream activations, whereas RMSNorm exactly cancels scalar scaling.Source: §A.2.1 · p. 15

A.2.2 Orthogonal Delta Injection

• For RMSNorm backbones, CWM adds an approximately orthogonal directional perturbation that survives normalization and changes the embedding direction.Source: §A.2.2 · p. 15

A.2.3 GPT-oss-20b: Inference and Decoding Constraints

• For GPT-oss-20b, the authors use eager/math attention, set reasoning verbosity low, and apply protocol-specific stopping and final-channel extraction.Source: §A.2.3 · p. 16

A.2.4 Backbone-Specific Summary

• The control coefficient cap is 5.0; RMSNorm backbones rely primarily on the directional term, with negative directional caps varied across 0.5, 1.0, and 1.5.Source: §A.2.4 · p. 16

A.3 baseline Details

• Adaptive-RAG uses a fixed T5-Large query-complexity classifier and BM25 retrieval, while SEAKR estimates uncertainty using 20 internal sampling passes.Source: §A.3 · p. 16

B Further Validation of Self-PPL

• Rowen-CL uses six cross-language question variants per query to aggregate consistency scores for retrieval decisions, and the appendix notes further Self-Perplexity validation analyses.Source: §A.3 · pp. 16–17, B Further Validation of Self-PPL · p. 17

B.1 Dataset-level Separation

• Self-PPL is consistently higher on knowledge-intensive RAG benchmarks than on reasoning benchmarks across all models.The authors interpret this separation as indicating that Self-PPL reflects knowledge insufficiency rather than reasoning complexity.Source: §B.1 · p. 17, Table 7 · p. 17

B.2 Correlation with Retrieval Gain

• Self-PPL has consistent positive correlations with retrieval gain across models and datasets, associating higher scores with larger empirical retrieval gains.The reported average correlation is the mean across three datasets.Source: §B.2 · p. 17, Table 8 · p. 17

B.3 Retrieval Gain by Self-PPL Groups

• Across all models, the High Self-PPL group has substantially larger average retrieval gains than the Low Self-PPL group.Queries are split at the within-dataset median Self-PPL; the reported Gain Difference is the High-group gain minus the Low-group gain.Source: §B.3 · p. 17, Table 9 · p. 17

B.4 Comparison with Entropy

• Self-PPL shows stronger correlation with retrieval gain and lower variance across inputs than token-level entropy.The comparison treats correlation as alignment with empirical retrieval usefulness and variance as signal stability; the authors selected Self-PPL for more robust scaling across diverse inputs and models.Source: §B.4 · pp. 17–18, Table 10 · p. 18

C Evaluation under Noisy Retrieval

• Across noisy BM25 and dense retrieval settings, CWM improves over both retrieval baselines, with results between the gold-context upper bound and no-context lower bound.The authors report that CWM captures most retrieval gains while remaining effective when retrieved documents may be incomplete or irrelevant.Source: C Evaluation under Noisy Retrieval · p. 18, Table 11 · p. 18

D Comparison with Threshold-based White-box Prompting

• CWM consistently outperforms Self-PPL threshold prompting across datasets and models, while the threshold baseline improves over no-context results.The comparison uses a fixed threshold based on each dataset’s average Self-PPL; CWM’s continuous modulation is reported as more effective.Source: D Comparison with Threshold-based… · pp. 18–19, Table 12 · p. 18, Table 13 · p. 19

E Full Results

• CWM results are reported for adaptive RAG and reasoning tasks using GPT-oss-20b, Qwen3-14b, and Llama3.1-8b.The tables vary directional delta values and negative caps; the captions identify separate results for each task and model.Source: E Full Results · p. 19, Table 15 · p. 20, Table 18 · p. 23
• Optimal performance may differ by model and benchmark, so the authors use model-specific hyperparameter configurations and describe tuning as straightforward.The stated rationale is that input perplexity scales differ across models, making model-specific configurations natural and necessary.Source: E Full Results · p. 19