MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization

• Different LLMs produce widely varying answers from the same task-specific prompts, motivating model-specific prompt adaptation.Source: §1 · p. 1, Figure 1 · p. 1
• Prior prompt optimization emphasizes task-specific adaptation, while model-specific prompt adaptation had not been studied in NLP.Source: §1 · p. 1
• MAPO optimizes original prompts for each LLM by creating candidate prompts, building a warm-up dataset, and combining SFT with reinforcement learning.Source: §1 · p. 2
• The paper quantitatively shows that different prompts should be adapted to different LLMs to improve performance across NLP downstream tasks.Source: §1 · p. 2
• MAPO exhibits greater robustness and generalization while achieving superior performance across varied downstream tasks.Source: §1 · p. 2

• Performance distributions differ significantly across BLOOM-7B, GPT-J-6B, and LLaMA-7B on QA, classification, and generation tasks using the same prompts.The study evaluates nine P3 datasets with F1, accuracy, and ROUGE-L for QA, classification, and generation, respectively.Source: §2 · p. 2

3.1 Warm-up Dataset Establishment

• MAPO generates candidate prompts, identifies the optimal candidate using task metrics or stronger-LLM outputs, and forms prompt pairs for training.It generates 1,000 candidates per prompt and uses F1, accuracy, or ROUGE-L according to task type.Source: §3.1 · pp. 2–3

3.2 Prompt Optimizer Construction

• The optimizer refines an original prompt P into a model-tailored prompt Po by changing its wording or structure.Source: §3.2 · p. 3

3.2.1 Supervised Fine-tuning

• SFT trains the LLM across downstream tasks to generate responses aligned with model preferences, with task instructions distinguishing generation, QA, and classification.Source: §3.2.1 · p. 3

3.2.2 Building Reward Model

• The reward model ranks original and candidate prompts using task metrics, then learns scalar scores that imitate the LLM’s preferences.Source: §3.2.2 · pp. 3–4

3.2.3 Reinforcement Learning

• RL optimizes prompts using reward feedback, combining PPO and RRMF while minimizing KL divergence from the SFT model.RRMF adapts ranking-based learning toward model feedback, and KL regularization limits divergence from the SFT prompt distribution.Source: §3.2.3 · pp. 4–5
• The RL policy uses original prompts as environments, generated prompts as actions, and reward-model scores to update an actor against a frozen SFT benchmark.Generalization maintenance samples 10% of GLUE and SuperGLUE data so performance is retained beyond specific downstream tasks.Source: §3.2.3 · pp. 4–5

4.2 Main Results

• MAPO improves performance across LLMs and tasks, including BLOOM gains of 20.5% on CloseQA and 30.9% on Explan, GPT-J gains of 21.4% and 20.6%, and LLaMA’s 22.8% gain on News.These improvements are reported relative to SFT and are statistically significant at p<0.01.Source: §4.2 · p. 5, Table 2 · p. 6
• MAPO generally outperforms prompt-optimization baselines, while RLprompt exceeds SFT and MAPO further improves performance except in a few cases.Most other baselines do not exceed SFT, and some do not outperform the original LLMs.Source: §4.2 · p. 6, Table 3 · p. 6, Table 4 · p. 7
• MAPO transfers across domains: every evaluated LLM improves with optimized prompts, with BLOOM showing the largest increase.The domain-transfer evaluation uses zero-shot inference without training on the target datasets.Source: §4.2 · p. 6, Table 5 · p. 8, Table 6 · p. 8

4.3 Ablation Study

• RL adds value beyond SFT across warm-up dataset sizes, although its improvement margin declines as the SFT dataset grows, supporting low-resource use.Increasing the warm-up dataset typically improves performance, while reduced data causes only minimal degradation and 10% few-shot experiments remain promising.Source: §4.3 · pp. 6–8

4.4 Case Study and Error Analysis

• Prompt optimization substantially changes wording: only about 10% of prompt pairs remain unchanged, with an average normalized edit distance of 0.67.Examples include voice changes, synonym substitutions, word-order rearrangements, and added topic details while preserving core information.Source: §4.4 · p. 8, Table 7 · p. 9
• MAPO exhibits model-specific prompt preferences, but optimized prompts can contain incomplete sentences, improper prepositions, missing parts, or ambiguous meanings.Exploratory word distributions differ across BLOOM, GPT-J, and LLaMA, although the paper states that stronger conclusions require more experiments.Source: §4.4 · p. 8, §4.5 · pp. 8–9

5 Related Work

• Existing approaches retrieve, generate, edit, or continuously optimize prompts, using methods such as greedy search, latent-space generation, feedback, reinforcement learning, and gradient-based optimization.These methods include APE, SharpT, APO, RLprompt, and continuous prompt optimization.Source: §5 · p. 9
• Most prior approaches optimize prompts for tasks rather than individual language models, whereas MAPO targets model-specific prompt optimization for NLP.The paper positions this distinction as its central difference from earlier work.Source: §5 · p. 9

6 Conclusions

• MAPO adapts prompts to different LLMs and improves their capabilities across various downstream tasks.The authors report this conclusion from extensive experiments and plan finer-grained adaptation for evolving real-world data and broader linguistic contexts.Source: §6 · p. 9

Limitations

• MAPO’s gains can be constrained when the warm-up dataset is limited or fails to cover the specific task.The limitation concerns both warm-up dataset availability and quality.Source: Limitations · p. 10
• MAPO requires extensive supervised fine-tuning and reinforcement learning, making training computationally expensive and potentially limiting scalability.The authors identify large-scale tasks and datasets as settings where this constraint may matter most.Source: Limitations · p. 10

A Candidate Prompts

• For nine P3 datasets, the study provides one original prompt and three GPT-3.5-generated candidate prompts per dataset.These candidates support the prompt-selection and optimization process.Source: A Candidate Prompts · p. 13

C Computational Cost

• MAPO training combines candidate generation, prompt search, SFT, reward-model construction, and RL, with total time complexity O(N × M) + O(E × B × T) + O(N × M × log M) + O(E′ × B′ × T).Inference instead uses one feed-forward operation with O(T) time and effectively O(1) extra space.Source: C Computational Cost · p. 13
• A complete MAPO training run takes about 20 hours for LLaMA-7B, 14 hours for BLOOM-7B, and 26 hours for GPT-J-6B on four A100 80GB GPUs.The reported totals combine SFT and RL durations for each model.Source: C Computational Cost · p. 13

D Baselines

• The study compares MAPO with SOTA prompt-optimization baselines, including GrIPS, RLprompt, TEMPERA, and AMA, using phrase-level editing, reinforcement learning, test-time editing, or recursive reformatting.Source: D Baselines · pp. 13–15
• The baseline study uses few-shot datasets with 16 samples per class, separate validation samples, random in-context exemplars, four seeds, and RoBERTa-large.Baselines include fine-tuning, continuous and discrete prompt methods, manual and in-context prompting, Instructions, GrIPS, RLprompt, TEMPERA, and AMA.Source: D Baselines · pp. 13, 15

Warm-up datasets

• Nine warm-up datasets cover question answering, classification, and generation tasks.The datasets are AdverQA, OpenQA, CloseQA, News, Movie, QASC, Topics, Summary, and Explan.Source: E Datasets · p. 15

Validation datasets

• Validation uses SST-2, Yelp Polarity, MR, CR, RTE, QNLI, SNLI, MNLI, and MRPC across several NLP tasks.The evaluation includes sentiment analysis, text classification, natural language inference, and paraphrase identification.Source: E Datasets · p. 15

Few-shot evaluation

• Table 3 uses Roberta-large for all experiments under a few-shot setup aligned with TEMPERA.The training data aligns with the TEMPERA setup.Source: E Datasets · p. 16

Zero-shot evaluation

• Tables 5 and 6 evaluate zero-shot inference without training on their datasets to assess MAPO’s domain-transfer ability.Further improvement on these datasets would require additional labeled-data training.Source: E Datasets · p. 16

Reward model

• The reward model improves consistently during MAPO training and is used to discriminate the quality of optimized prompts.Its increasing performance is interpreted as adaptation to downstream tasks.Source: F Additional Experiments · p. 16

Original capabilities

• MAPO preserves original capabilities, but improvements on Tables 5 and 6 are not significant with the limited GLUE and SuperGLUE pre-training scope.The authors suggest broader pre-training datasets as future work.Source: F Additional Experiments · p. 16

Generalization control

• A control using MAPO prompts with the original Roberta-Large model shows no significant decline, with a t-test greater than 0.05.The authors use this test to address possible effects of reusing Table 3 data for generalization.Source: F Additional Experiments · p. 16

SST-2

• In SST-2, BLOOM and LLaMA emphasize positive sentiment, whereas GPT-J highlights artistic ability more neutrally.The examples use “terrific flair,” “remarkable skill,” and “tremendous artistry” to illustrate the differences.Source: G Additional Cases · p. 16

Yelp and MR

• In Yelp and MR, BLOOM and LLaMA add more evaluative language, while GPT-J and the original prompts remain comparatively neutral.Yelp examples balance food quality and service, whereas MR examples emphasize positive qualities of a venture.Source: G Additional Cases · p. 16

CR

• In CR, BLOOM, GPT-J, and LLaMA all express possible confusion about the phone’s space-key position, with BLOOM and LLaMA implying a deviation from the norm.The difference is primarily in the strength of the implied comparison with common phone layouts.Source: G Additional Cases · p. 16

RTE

• In RTE, BLOOM and LLaMA add emotional wording, while GPT-J and the original prompt provide more neutral descriptions.The examples contrast phrases such as “somber site” and “distressing sight” with neutral descriptions.Source: G Additional Cases · pp. 16–17

QNLI and SNLI

• In QNLI and SNLI, BLOOM, GPT-J, and LLaMA generally preserve meaning while varying wording or adding contextual details.QNLI changes are mainly stylistic, whereas SNLI adds details about slipping, sliding, celebration, and wetness.Source: G Additional Cases · p. 17

MNLI and MRPC

• In MNLI and MRPC, the models usually retain portions of the original wording while changing emphasis in selected sentences.MNLI changes emphasize interest rates or savings, while MRPC changes emphasize a 15 percent revenue drop or remain neutral.Source: G Additional Cases · p. 17

Word-distribution analysis

• QA and classification show minimal changes when the instruction is removed, but optimized word distributions differ across BLOOM, GPT-J, and LLaMA.The models favor distinct lexical patterns, including temporal, semantic, institutional, and geographic terms.Source: H Additional Exploratory Analysis · pp. 17–18

RL components

• Table 10 evaluates PPO and RRMF separately as components of MAPO’s reinforcement-learning process.Its stated purpose is to demonstrate the roles of both components in enhancing MAPO performance.Source: Figure 6 · p. 18

Warm-up proportion

• Table 11 compares different warm-up-dataset proportions across QA, classification, and generation for three LLMs.It reports performance declines and data reductions using both absolute and percentage measures.Source: Table 11 · p. 18

Model-specific patterns

• The differing word distributions suggest that each LLM has distinct preferences and semantic emphases during prompt optimization.The authors identify these patterns as evidence of model-specific optimization behavior.Source: H Additional Exploratory Analysis · p. 18

Baseline comparison

• Table 12 compares few-shot MAPO with fine-tuning, continuous-prompt, and discrete-prompt baselines under multiple temperature settings.MAPO and MAPO-0 distinguish temperature ranges [0,0.5] and 0, respectively.Source: Table 12 · p. 19

Frozen-LLM improvements

• Table 14 reports absolute and relative improvements of MAPO-optimized prompts over a frozen LLM for classification and generation tasks.The table includes several MAPO temperature settings and averages enhancements across three LLMs.Source: Table 14 · p. 20

Domain transfer

• Tables 15 and 16 evaluate zero-shot domain transfer with original and MAPO prompts for BLOOM, GPT-J, and LLaMA.They report absolute and relative improvements, with Table 16 using a frozen LLaMA for inference.Source: Table 15 · p. 24, Table 16 · p. 24

Training ablations

• Tables 17 and 18 explore SFT without reinforcement learning and performance across training epochs, including declines at longer training durations.Table 17 labels SFT as using SFT-optimized prompts without RL, while Table 18 reports results through 50 epochs.Source: Table 17 · p. 25, Table 18 · p. 26