Representation Alignment as a Bottleneck in LLM-Based Retrosynthesis Planning

1 Introduction

• All evaluated models achieve 0% planning success in direct SMILES-to-PDDL generation, while structured intermediate representations yield high planning success for most strong models.The authors attribute this contrast to representation misalignment rather than insufficient model capacity.Source: §1 · pp. 1–2
• The framework decomposes retrosynthesis planning into molecule mapping, reaction mapping, and PDDL generation, evaluating each stage to locate failures.Each stage incrementally transforms unstructured inputs into structured symbolic representations.Source: §1 · pp. 1–2
• The paper frames the discrepancy between strong subtask performance and complete end-to-end failure as representation misalignment rather than insufficient model capability.The authors argue that planning performance depends more on representation alignment than on the underlying model itself.Source: §1 · p. 2
• Intermediate symbolic representations are presented as important for designing future LLM-based planning systems.The staged framework and corresponding benchmarks support systematic analysis of failures across representation levels.Source: §1 · p. 2

2.1 LLMs for Planning

• Prior LLM planning studies generate action sequences or use code, programs, and external planners, but rarely analyze transitions from unstructured inputs to executable PDDL plans.The authors position their work as centering analysis on representation transitions.Source: §2.1 · p. 3

2.2 LLMs in Chemistry and Retrosynthesis

• LLM chemistry studies largely emphasize reaction prediction and pathway generation, while formalizing outputs into planner-executable symbolic representations has received relatively little attention.This work treats retrosynthesis as executable symbolic planning and analyzes where the process fails.Source: §2.2 · p. 3

2.3 Intermediate Representations and Modular Reasoning

• Intermediate representations and modular reasoning have been studied mainly for language reasoning and code, with limited investigation of chemical-to-planning representation alignment.The paper introduces molecule mapping, reaction mapping, and PDDL generation as a staged framework and quantitatively analyzes intermediate symbolic abstraction.Source: §2.3 · p. 3

3.1 Overview

• The framework sequences molecule mapping, reaction mapping, PDDL generation, and planning execution to identify failures between representation levels.The stages progressively structure chemical inputs and translate them into a planning language before checking for a valid pathway.Source: §3.1 · p. 4

3.2 Molecule Mapping

• Molecule mapping converts symbolic molecule identifiers into corresponding SMILES strings, evaluated by exact match and missing or invalid outputs.This stage measures how reliably symbolic references are grounded in molecular structures.Source: §3.2 · p. 4

3.3 Reaction Mapping

• Reaction mapping organizes molecule identifiers into reactant and product sets, represented as tuples (R, P) and evaluated by reaction-level exact match.The resulting reaction-level structure serves as an intermediate representation for PDDL conversion.Source: §3.3 · p. 4

3.4 PDDL Generation

• PDDL generation converts structured reactions into domain actions and problem definitions, evaluated for syntax, completeness, and semantic consistency.Each action's preconditions and effects symbolically define a reaction's structure.Source: §3.4 · p. 4

3.5 Planning Execution

• Planning execution uses Fast Downward to test generated PDDL, measuring solve rate and whether the resulting pathway matches ground truth.This checks whether outputs produce executable plans, not merely syntactically plausible representations.Source: §3.5 · p. 5

3.6 Evaluation Protocol

• Evaluation compares direct SMILES-to-PDDL generation, independent stages with oracle intermediates, and a non-oracle chained setting to locate bottlenecks.The study aims to identify failure stages, not to propose a fully automated pipeline.Source: §3.6 · p. 5

4.1 Construction Overview

• RetroPlan-Bench reconstructs 368 multi-step READRetro synthesis pathways as aligned molecule-, reaction-, and planning-level representations.The benchmark supports quantitative analysis from unstructured chemical information to executable symbolic plans.Source: §4.1 · pp. 5–6

4.2 Symbolic Transformation Pipeline

• The transformation pipeline replaces molecules with symbolic identifiers, structures reactions as reactant–product pairs, and filters out pathways not fully represented by the reaction set.Coverage filtering is intended to make planning failures reflect representation transformation rather than missing data.Source: §4.2 · p. 6

4.3 Multi-scale Planning Regimes

• Planning sets use 100, 200, 300, and 400 unique reactions, while PDDL-generation domains extend to 1000 actions.Pathway sets are balanced by path length and branching structure; larger domains test long-context handling and structural consistency.Source: §4.3 · p. 6

4.4 Evaluation Protocol

• The evaluation measures molecule mapping accuracy, reaction consistency, PDDL validity and grounding, and planning success.This decomposed evaluation is intended to identify alignment failures not visible from end-to-end performance alone.Source: §4.4 · p. 6

5.1 Strong Performance on Individual Sub-tasks

• Strong models achieve near-perfect individual mapping and PDDL-generation performance, while Qwen-family models and ChemLLM lose coverage and validity as tasks progress.Domain structural completeness breaks down for weaker models as reaction counts increase.Source: §5.1 · pp. 6–7, Table 1 · p. 7

5.2 Failure of End-to-End Generation

• 0 solve rate and path accuracy across all models in direct SMILES-to-PDDL generation, with no valid synthesis pathway produced.This contrasts with strong individual-subtask results and staged recovery for some models.Source: §5.2 · pp. 7–8

5.3 Effect of Intermediate Representations

• GPT-5.2, DeepSeek V3.2, and Gemini 3.1 reach a 1.0 solve rate with structured intermediates; GPT-5.2 and DeepSeek V3.2 retain 0.986 path accuracy in chained evaluation.Smaller models named in the study fail to produce executable plans even with structured inputs.Source: §5.3 · p. 8

5.4 Scaling Behavior and Failure Modes

• GPT-5.2 and DeepSeek V3.2 preserve domain coverage and syntactic validity across scales, but their action-grounding accuracy declines as reactions increase.Gemini models show scale-specific drops and recovery, while Qwen models degrade rapidly, sometimes failing to produce valid structures.Source: Figure 3 · p. 7, §5.4 · p. 8

5.5 Planning Performance

• With structured intermediates, GPT-5.2, DeepSeek V3.2, and Gemini 3.1 reach a 1.0 solve rate, whereas Qwen-family models, Gemini 2.5 Flash, and ChemLLM score 0 on solve rate and path accuracy.ChemLLM’s failures are dominated by malformed structured outputs; all models fail to produce valid plans end-to-end.Source: §5.5 · p. 8

6.1 Representation Alignment vs. Reasoning Capability

• Strong performance on individual operations alongside end-to-end failure supports characterizing the central difficulty as cross-representation composition, not reasoning capability.Planning performance recovers when intermediate representations are provided.Source: §6.1 · p. 9

6.2 Why End-to-End Transformation Fails

• SMILES-to-PDDL transformation requires restructuring across representation spaces, while implicitly handled intermediates risk information distortion and loss.The study also observes semantic grounding loss or structural collapse at some scales.Source: §6.2 · p. 9

6.3 The Role of Intermediate Symbolic Representations

• Separating molecule and reaction mapping lets models satisfy structural constraints stage by stage and improves overall planning performance.The authors argue explicit decomposition is needed to externalize coherent representations consistently.Source: §6.3 · p. 9

6.4 Implications for LLM-based Planning Systems

• The authors argue that staged designs centered on intermediate symbolic representations can be a key determinant of performance, beyond model scale or general reasoning ability.They caution that single-step generation of executable plans from unstructured inputs may face fundamental limitations.Source: §6.4 · p. 9

6.5 Limitations and Future Directions

• The evidence is limited to retrosynthesis, and the chained non-oracle evaluation does not establish domain-general robustness.Whether the observed phenomenon generalizes to other planning problems remains open.Source: §6.5 · p. 9

7 Conclusion

• The paper reframes LLM retrosynthesis failure as a representation-alignment problem and analyzes performance across staged subtasks.It contrasts strong planning with structured intermediates against failure in the end-to-end setting.Source: §7 · p. 10
• All models fail in the end-to-end setting, while structured intermediate representations support strong planning performance.The conclusion identifies intermediate representations as important for generating executable symbolic plans.Source: §7 · p. 10
• The authors conclude that representation-centric structural design can play a crucial role in future LLM-based planning research.Source: §7 · p. 10

A Failure Cases in Direct SMILES-to-PDDL Generation

• Direct SMILES-to-PDDL outputs fail to preserve raw molecular strings as planner-compatible symbols, with errors ranging from malformed syntax to semantic corruption.Across models, failures include reinterpreting SMILES as searchable text, string literals, or encoded forms.Source: A Failure Cases in Direct… · p. 14, Figure 4 · p. 15

A.1 Failure Under Explicit Output Constraints

• Explicit output constraints do not prevent malformed outputs or loss of semantic grounding, so prompt ambiguity alone does not account for the failures.Models were instructed to preserve SMILES exactly or use deterministic PDDL-safe symbols.Source: §A.1 · pp. 15–16

A.2 Failure Persists Under Sanitized Symbol Mapping

• Sanitized symbols do not eliminate failure: models truncate identifiers, collapse distinct molecules, or produce incomplete domains, indicating persistent long-range symbolic-consistency problems.The errors extend beyond special-character handling to symbol reuse and action-level grounding.Source: §A.2 · p. 16, Figure 5 · p. 17

A.3 Failure Modes of ChemLLM in Structured Symbolic Generation

• ChemLLM fails across the benchmark stages, primarily through unstable symbolic serialization rather than subtle chemistry-specific reasoning errors.Its generations often fail to preserve exact identifiers, satisfy rigid schemas, or maintain executable formal structure.Source: §A.3 · p. 16

• The appendix covers RetroPlan-Bench subtask datasets and performance metrics across scales of 100–1000 reactions.Source: B Detailed Experimental Results · p. 17
• ChemLLM fails through malformed JSON, exact-copy errors, non-PDDL outputs, and truncated generations.Source: Figure 6 · p. 18
• These failures often occur before downstream planning can be meaningfully evaluated.Source: Figure 6 · p. 18

B.1 Molecule and Reaction Mapping Integrity

• Stronger models maintain near-perfect molecule-identifier match rates, while SMILES exact-match rates for other models decline as molecule counts increase.Source: §B.1 · p. 18, Table 2 · p. 19
• Most models robustly preserve reaction-ID relationships, making chemical-string preservation the more prominent mapping bottleneck.Source: §B.1 · p. 18, Table 3 · p. 19

B.2 PDDL Generation Validity and Completeness

• GPT-5.2, DeepSeek V3.2, and Gemini models maintain high PDDL syntactic validity across most scales, while Qwen models break down above 500 reactions.Source: §B.2 · p. 19
• Action-grounding accuracy decreases with scale, particularly for Gemini 3.1 at 600 reactions.Source: §B.2 · p. 19
• GPT-5.2 and DeepSeek V3.2 achieve perfect coverage of generated problem instances.Source: §B.2 · p. 19

B.3 Retrosynthesis Planning Performance

• Gemini 2.5 Flash fails to solve staged problems despite high subtask accuracy, indicating that cumulative errors can prevent executability.Source: §B.3 · p. 19
• GPT-5.2, DeepSeek V3.2, and Gemini 3.1 achieve a 100% solve rate in staged planning, with high path accuracy.Source: §B.3 · p. 19, Table 8 · p. 21
• All models achieve a 0% success rate when directly generating PDDL from SMILES.Source: §B.3 · p. 19, Table 9 · p. 21

A Prompt Examples

• The molecule-preprocessing prompt assigns sequential identifiers to SMILES while preserving strings and enforcing JSON; the reaction prompt preserves IDs and ordering in a fixed-key JSON array.Source: Figure 7 · p. 21, Figure 8 · p. 22
• The problem prompt specifies an initial molecule and shared goals, while the domain prompt converts ordered reactions into STRIPS-style actions that preserve identifiers and order.Source: Figure 9 · p. 23, Figure 10 · p. 24