Search-Guided Reasoning Trace Curation for Verilog Generation

Motivation

• Long reasoning traces can add token overhead and contain repetitive, inconsistent, or weakly relevant content, motivating more effective reasoning supervision.Source: §1 · p. 2
• Existing pipelines generally select complete reasoning responses by final Verilog correctness, leaving intermediate decisions weakly controlled and typically expressed as free-form text despite RTL's explicit design decisions.The introduction identifies interfaces, timing, state, behavior, and signal updates as relevant design decisions.Source: §1 · p. 2

Approach

• MCTS-SFT searches sequences of intermediate design states and uses terminal testbench feedback to value them, while Plan-MCTS searches structured hardware decisions rather than accumulated free-form text.The framework also studies retaining the searched plan in the downstream SFT response.Source: §1 · p. 2

Results

• For fixed Plan-MCTS trajectories and Verilog outputs, retaining the explicit plan raises pass@10 from 52.56% to 57.7%; the larger study also improves correctness while reducing generation length by approximately 50% on VerilogEval-v2.The larger study compares MCTS-curated SFT with full-response CoT distillation.Source: §1 · p. 2

2.1 TRAINING LLMS FOR VERILOG GENERATION

• Verilog training has progressed from curated corpora and instruction tuning to functional-feedback post-training, including testbench-based rewards (Wang et al., 2025a).Other reward formulations include hierarchical and entropy-aware objectives (Chen et al., 2025; Shi et al., 2025).Source: §2.1 · pp. 2–3

2.2 REASONING LLMS FOR VERILOG GENERATION

• Reasoning-based Verilog methods improve functional correctness but can produce long, inefficient traces; StepPRM-RTL adds a learned process reward model to guide MCTS exploration (Vijayaraghavan et al., 2026).Source: §2.2 · p. 3
• MCTS-SFT instead uses executable feedback from complete rollouts as search value, avoiding a separate process reward model while retaining hardware-specific states for SFT.Source: §2.2 · p. 3

2.3 INFERENCE-LEVEL VERILOG REASONING OPTIMIZATION

• Inference-time methods use structured prompting, candidate selection, or refinement to improve RTL candidates, whereas MCTS-SFT searches offline to curate training trajectories.Source: §2.3 · p. 3

3 FRAMEWORK

• MCTS-SFT explores intermediate states for each specification, rolls selected states into complete reasoning traces and Verilog, evaluates the code, and converts curated trajectories into student-model SFT examples.Source: §3 · pp. 3–4

3.1 VERIFIER-GUIDED REASONING-TRACE CURATION

• The search state pairs the specification with its current reasoning representation; Thought-MCTS uses a natural-language prefix, while Plan-MCTS uses a structured hardware plan.Source: §3.1 · p. 4
• Because an intermediate decision cannot be verified alone, a rollout materializes it into a complete trace and RTL implementation that is compiled and checked against a testbench.Non-compilable designs are penalized, and functional correctness is measured by the fraction of passing tests.Source: §3.1 · p. 4
• MCTS selects and expands states, rolls out implementations, and backpropagates terminal verifier rewards so repeated simulations guide exploration toward decisions associated with stronger functional outcomes.The action-value estimate and visitation counts support selection, with c_uct controlling exploration.Source: §3.1 · pp. 4–5

3.2 PLAN-MCTS: STRUCTURED HARDWARE DESIGN STATES

• Plan-MCTS updates a seven-field plan—interface, clock reset, state elements, combinational logic, FSM, edge cases, and implementation constraints—so branches represent alternative hardware decisions.The plan serves both as the search state and as an explicit record of decisions that can be shown to the student.Source: §3.2 · p. 5

3.3 PLAN-PRESERVING SUPERVISED FINE-TUNING

• Plan-preserving SFT serializes the searched plan together with rollout reasoning and final Verilog; experiments separate which states search discovers from how the SFT response represents them.Source: §3.3 · pp. 5–6

4.1 EXPERIMENTAL SETUP

• The controlled study compares five curation strategies, separating structured representation, tree exploration, and verifier-guided selection under matched conditions.For SFT-target analysis, three targets use identical Plan-MCTS trajectories, implementations, rewards, and dataset sizes.Source: §4.1 · p. 6
• The 6k-trace study compares CoT, Thought-MCTS, and Plan-MCTS across Verilog benchmarks, using mixed teachers and rollout-only Plan-MCTS supervision.The study is intended for pipeline-level comparisons; component-level conclusions come from the unified-teacher 1k-trace study.Source: §4.1 · p. 6
• The evaluation measures pass@k correctness and response length; functional trace yield counts trajectories whose terminal implementations pass all associated testbench cases.The controlled study uses VerilogEval-v1-Human with n = 10 generations; the extended study adds VerilogEval-v2 tasks with n = 20 and 172 multi-module CVDP tasks.Source: §4.1 · p. 7

5.1 RQ1: HOW DO STRUCTURED PLANNING AND VERIFIER-GUIDED SEARCH AFFECT FUNCTIONAL TRACE DISCOVERY?

• 84.2% is Plan-MCTS's functional trace yield, compared with 81.9% for one-shot Plan-CoT and 82.0% for unguided Plan-RandomTree.Thought-MCTS reaches 83.6%, while independent CoT sampling reaches 73.1%.Source: §5.1 · p. 7

5.2 RQ2: HOW DOES INTERMEDIATE REPRESENTATION AFFECT DOWNSTREAM SFT?

• 5.14 points is the pass@10 improvement from rollout-only to explicit-plan Plan-MCTS supervision, with identical trajectories and terminal programs.Explicit-plan supervision also beats the natural-language rendered plan at pass@1/5/10; explicit-plan Plan-MCTS has the strongest pass@5 and pass@10 among the reported controls.Source: §5.2 · pp. 7–8, Table 2 · p. 8
• Thought-MCTS produces longer, more repetitive traces than plan-based methods, while Plan-MCTS target variants have identical terminal implementations and verifier rewards but different downstream performance.Summary-MCTS remains close to Thought-MCTS, so the reported benefit is not explained simply by compressing free-form reasoning.Source: §5.2 · p. 8, Table 3 · p. 8

5.3 RQ3: DOES SEARCH-CURATED SUPERVISION IMPROVE DOWNSTREAM CORRECTNESS AND EFFICIENCY AT LARGER SCALE?

• 2.6/3.2/3.2 points on VerilogEval-v1-Human and 1.8/2.6/3.7 points on VerilogEval-v2 Code Completion are Plan-MCTS pass@1/5/10 gains over full-response CoT distillation; VerilogEval-v2 responses are about 50% shorter.The VerilogEval-v2 reductions are 50.5% and 49.2% on its two prompt sets while maintaining or improving functional accuracy.Source: §5.3 · pp. 8–9, Table 4 · p. 9
• 12.21% is Plan-MCTS's overall success rate on 172 CVDP-002/003 tasks, versus 11.63% for Thought-MCTS and 9.30% for CoT-SFT; the authors treat this as preliminary transfer evidence.Absolute success rates remain low, so the evaluation is not comprehensive evidence of multi-module RTL capability.Source: §5.3 · p. 9, Table 5 · p. 9

6 CONCLUSION

• MCTS-SFT treats reasoning as intermediate design states and propagates terminal testbench feedback through MCTS to value earlier decisions.It curates reasoning traces for supervised fine-tuning of Verilog generation models.Source: §6 · p. 9
• The strongest controlled performance combines structured hardware planning, verifier-guided search, and explicit preservation of the searched plan during SFT.Structured planning is independently valuable, and explicit plans outperform omitting the plan or rendering it as less-structured prose when trajectories and Verilog outputs are held fixed.Source: §6 · p. 9
• In the larger 6k-trace evaluation, Plan-MCTS improves Verilog correctness over full-response CoT distillation while reducing VerilogEval-v2 generation length by approximately 50%.The conclusion emphasizes that supervision depends on which intermediate design states are discovered and how they are represented during training.Source: §6 · p. 10

AI USE STATEMENT

• LLMs generated, expanded, and curated the reasoning trajectories and Verilog implementations used in the experiments; their manuscript-preparation use was limited to formatting and editorial refinement.The authors state that they determined the scientific claims, experimental design, analyses, figures, tables, and conclusions.Source: AI USE STATEMENT · p. 10

• The experiments use existing research datasets, benchmarks, and model-generated training data, with no human subjects or private personal information.Source: ETHICS STATEMENT · p. 10
• Generated RTL may contain functional or implementation errors if used without appropriate verification.Source: ETHICS STATEMENT · p. 10
• Executable testing during data curation does not guarantee correctness or suitability for safety- or security-critical hardware deployment.Source: ETHICS STATEMENT · p. 10

• Section 3 defines MCTS-SFT and Plan-MCTS, while Appendix A.2 adds algorithm and state-representation details.Source: REPRODUCIBILITY STATEMENT · p. 10
• The experimental setup specifies controlled and extended-scale protocols, with Appendix A.1 reporting available curation, training, model, evaluation configurations, and hyperparameters.Source: REPRODUCIBILITY STATEMENT · p. 10
• Appendices include additional ablations and generalization experiments, and the paper documents teacher, search, and SFT-target distinctions for consistent reconstruction of controlled comparisons.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1.1 REASONING-TRACE CURATION

• The extended study generates 6,000 traces per full strategy from 600 specifications, using DeepSeek-R1 for rollouts and DeepSeek-V3 for intermediate Plan-MCTS states.The controlled study uses 1,000 examples per strategy and DeepSeek-R1 for every LLM call.Source: Table 6 · p. 13, §A.1.1 · p. 13

A.1.2 SUPERVISED FINE-TUNING

• SFT uses a shared optimization configuration, while downstream evaluation uses VerilogEval-v1-Human and VerilogEval-v2 tasks in the extended study and VerilogEval-v1-Human in the controlled study.The controlled study uses Qwen2.5-Coder-7B-Instruct; extended evaluation samples each benchmark problem 20 times.Source: §A.1.2 · pp. 13–14, §A.1.3 · p. 14

A.2.1 MCTS SEARCH PROCEDURE

• MCTS performs selection, expansion, rollout, and backpropagation; terminal verifier rewards update values along the visited state-action edges.Expansion proposes candidate actions, which are free-form reasoning steps in Thought-MCTS or structured plan updates in Plan-MCTS.Source: §A.2.1 · pp. 14–15

A.2.2 THOUGHT-MCTS: FREE-FORM REASONING SEARCH

• Thought-MCTS carries accumulated natural-language reasoning through search, whereas Plan-MCTS refines partial hardware plans and supplies them as context for rollout into reasoning and Verilog.Plan-MCTS may refine the same design aspect over multiple steps.Source: §A.2.2 · pp. 15–16, §A.2.3 · p. 16

A.3.1 COMPRESSION VERSUS HARDWARE-SPECIFIC STRUCTURE

• Summarization raises Thought-MCTS pass@1/pass@10 by 1.86/0.64 points, while explicit-plan supervision exceeds Summary-MCTS by 2.27/4.37/5.78 points at pass@1/5/10.The authors conclude that compression alone does not explain the benefit of structured hardware representation.Source: Table 10 · p. 16, §A.3.1 · p. 16
• Explicit structured-plan targets improve pass@1/5/10 by 1.5/3.2/2.6 points over rendered natural-language plans from the same trajectories and Verilog.Source: Table 10 · p. 16, §A.3.1 · p. 16

A.4.1 GENERALIZATION TO A SECOND TEACHER MODEL

• With GPT-5.4-mini as teacher, Plan-MCTS is slightly stronger at pass@1 and pass@5, while Thought-MCTS is slightly stronger at pass@10; differences do not establish a consistent ranking.Source: Table 11 · p. 17, §A.4.1 · p. 17

A.4.2 SCALING MCTS-CURATED SUPERVISION

• 49.0 pass@1 at 60k traces is 23.4 points above the base model and 10.8 points above the 10k-trace model.Pass@5 and pass@10 also improve overall, although intermediate results are not strictly monotonic.Source: Table 12 · p. 17, §A.4.2 · p. 17