QiMeng-CodeV-R1: Reasoning-Enhanced Verilog Generation

1 Introduction

• RLVR for Verilog generation is limited by difficult hardware verification, scarce high-quality NL–code pairs, and prohibitive training cost.Hardware verification must handle sequential-circuit state spaces, reset behavior, and clock synchronization; existing NL–code datasets are often low quality.Source: §1 · p. 2
• The framework develops a rule-based testbench that checks generated Verilog against golden references and detects more errors than an LLM-generated counterpart.It achieves 96.1% fewer false negatives and detects 62.5% more injected sequential-circuit errors in fuzzing tests.Source: §1 · p. 2
• Round-trip synthesis pairs Verilog with LLM-generated NL, regenerates code from the NL, and retains only equivalence-checked examples for RLVR.The procedure is theoretically supported under strong models and ideal verification by a high probability of producing sufficiently high-quality NL–code pairs.Source: §1 · p. 2
• The two-stage pipeline distills reasoning through SFT before applying adaptive DAPO-based RLVR, which dynamically adjusts sampling to reduce unnecessary verification overhead.The adaptive mechanism achieves a reported 1.25x acceleration.Source: §1 · p. 2
• 68.8% / 72.9% / 68.0% pass@1 on VerilogEval v2, RTLLM v1.1, and RTLLM v2, respectively, is achieved by CodeV-R1-7B, which surpasses 671B DeepSeek-R1 by 8.1% and 3.3% on the two RTLLM versions.The model is trained with around 2,656 A100-GPU-hours.Source: §1 · p. 3

2.3.1 High-quality Data Curation

• Round-trip equivalence checking compares regenerated code with original snippets and retains only validated pairs, yielding data intended to be solvable, challenging, and error-free.Five generated variants are used to exclude questions already mastered by the distilled model, producing 3.1K RL examples.Source: §2 · p. 3, §2.3.1 · pp. 4–5

2.1 Automated Testbench Generation Framework for Verilog Code

• The automated testbench extracts circuit structure, simulates generated and reference Verilog under randomized conditions, and computes output-equivalence error rates.It handles clock and reset polarity, synchrony, and multiple reset combinations; typical sequential circuits receive 2MN assessments.Source: §2.1 · pp. 3–4

2.2 CodeV-R1-7B-Distill: Supervised Distillation for Verilog Data

• The distillation phase creates NL–thought–code triples, filters examples that baseline models can already solve, checks synthesizability and contamination, and retains 87K samples for SFT.DeepSeek-V3 produces NL descriptions, DeepSeek-R1 generates reasoning and code, and Yosys checks synthesizability.Source: §2.2 · p. 4

2.3.1 High-quality Data Curation

• NLCDE defines code–NL semantic or functional equivalence, and the round-trip theorem states that certainty-preserving transformations imply individually equivalent forward and backward mappings.The paper motivates this criterion as a way to limit information-loss errors during code-to-NL-to-code conversion.Source: §2.3.1 · p. 5

2.3.2 Adaptive DAPO Algorithm

• Adaptive DAPO changes the generation batch size using an estimated valid-sample ratio to avoid both costly resampling and wasted excess samples.Its reward is 1 only for correctly formatted responses functionally equivalent to the golden code, and 0 otherwise.Source: §2.3.2 · pp. 5–6

3.2 Main Results

• 18.8% higher pass@1 than CraftRTL-DS-6.7B is achieved on RTLLM v1.1, while the model remains smaller than CraftRTL-SC2-15B on VerilogEval v1-Human.Source: §3.2 · pp. 6–7
• More than 10% pass@1 improvement on RTLLM follows reinforcement learning over CodeV-R1-7B-Distill, indicating the RL stage substantially improves Verilog generation.Source: §3.2 · p. 7

3.3.1 Test-Time Scaling

• 7.1% to 72.9% accuracy growth occurs as the response budget increases from 4096 to 16384 tokens, versus 29.0% to 64.1% for DeepSeek-R1, with higher accuracy per FLOP for CodeV-R1-7B.Source: §3.3.1 · p. 8, Figure 2 · p. 8

3.3.2 Equivalence Checking and Difficulty Filtering Improves RL Training

• The 3.1K equivalence-checked, difficulty-filtered dataset produces steadier response-length growth and higher rewards than pseudo-golden or unfiltered alternatives during RL.Source: §3.3.2 · pp. 8–9, Figure 3 · p. 8

3.3.3 Acceleration via Adaptive DAPO

• 1.25× final speedup is obtained with adaptive DAPO, including 1.44× after step 150 versus 1.04× before step 150 by dynamically increasing generation batches.Source: §3.3.3 · p. 9, Figure 4 · p. 9

3.3.4 Testbench Performance Evaluation

• The generated testbench misclassifies only 0.3% of golden-versus-golden cases and detects 65% of injected sequential-circuit errors, outperforming the DeepSeek-V3 testbench.Source: §3.3.4 · p. 9

4.1 Large Language Models for Reasoning

• Reasoning-model research combines large-scale RL and distillation, but prior domains commonly benefit from easily verifiable numerical outputs and rich open-source datasets.Source: §4.1 · pp. 9–10

4.2 Large Language Models for Verilog Code Generation

• Most Verilog data-construction methods perform syntax checks without strict correctness evaluation, while CraftRTL provides functional correspondence only for Karnaugh maps and finite-state machines.Source: §4.2 · p. 10
• Existing testbench-generation approaches face unverified validation frameworks and costly multi-stage correction, undermining reliable and efficient RL reward computation.Source: §4.2 · p. 10
• CodeV-R1 instead uses auto-generated Verilog equivalence checking to support both data curation and reinforcement learning.Source: §4.2 · p. 10

• CodeV-R1-7B achieves strong Verilog-generation performance on VerilogEval and RTLLM, matching or surpassing DeepSeek-R1.The paper attributes this outcome to automated testbench generation and the two-stage training paradigm.Source: §5 · p. 10
• The framework combines reasoning-data distillation with reinforcement learning on data curated by automated testbench generation.Source: §5 · p. 10
• CodeV-R1-7B is presented as the first Verilog-domain LLM with strong reasoning capability and consistent test-time scaling behavior.Source: §5 · p. 10
• Analytical experiments highlight the framework’s impact on data quality and Verilog code-generation capabilities through reasoning.Source: §5 · p. 10

• The abstract and introduction accurately reflect the paper’s contributions, including equivalence checking, dataset filtering, and adaptive DAPO.Source: §1 · p. 15
• The paper discusses data-synthesis cost and potential low-quality data as limitations.Source: §1 · p. 15

4. Experimental result reproducibility

• The paper describes its dataset creation, RL algorithm, evaluation metrics, models, and experimental setup sufficiently to support reproducibility.Source: §1 · p. 16

5. Open access to data and code

• The paper reports releasing its complete model, code, and dataset through Hugging Face and GitHub.Source: §1 · p. 17

6. Experimental setting/details

• Key SFT, RL, and testing parameters are described in Section 3.1, with full settings provided in Appendix B.Source: §1 · p. 17

7. Experiment statistical significance

• The reported percentages and raw counts lack error bars, confidence intervals, and formal significance tests because repeated RL runs were considered prohibitively expensive.Source: §1 · p. 17

10. Broader impacts

• The paper reports addressing both potential positive and negative societal impacts.Source: §10 · p. 18
• The training data comes from open-source GitHub code and open-source language models, which the checklist characterizes as posing no sensitive-content leakage risk.Source: §10 · p. 19
• The paper credits the datasets and models it uses and reports compliance with their respective open-source licensing terms.Source: §10 · p. 19
• The CodeV-R1 framework and CodeV-R1-7B model are documented primarily through the paper, with practical release documentation typically expected alongside them.Source: §10 · p. 20
• The work does not involve crowdsourcing, human subjects, or institutional-review-board approval.Source: §10 · pp. 20–21
• The paper identifies LLMs as a fundamental component of its method and describes their use across the introduction, methods, and experiments.Source: §10 · p. 21

A.1 Proof and Further Explanation of Theorem 2.1

• Under the NLCDE assumption, the proof establishes that round-trip equivalence implies both EY,X and EX,Y′ almost surely.The result connects functional equivalence after the round trip with correctness of summarization and code generation.Source: §A.1 · p. 22
• NLCDE is necessary because conditional entropy alone does not exclude consistently mismatched code–language pairs.The appendix illustrates this with a multiplier-to-adder mismatch example.Source: §A.1 · p. 22
• The theorem treats code snippets and natural-language descriptions implementing the same function as identical and assumes deterministic functional mappings.These idealized conditions model functional consistency rather than textual uniformity.Source: §A.1 · p. 22

A.2 Algorithm Description of Adaptive DAPO

• Adaptive DAPO dynamically adjusts generation batches using step-level and inner-step-level ratios to obtain enough valid samples for each training update.The algorithm processes epochs over the dataset, updates counters, and then performs DAPO with the required valid samples.Source: §A.2 · pp. 22–23

B Parameter Setting

• The SFT and RL configurations are documented in separate parameter tables, with testing using a 16384-token context and temperature 1.0.Top-p is 1.0 for VerilogEval and 0.95 for RTLLM.Source: B Parameter Setting · p. 23, Table 3 · p. 23, Table 4 · p. 24
• RL generation uses train_batch_size and ppo_mini_batch_size equivalents, with 128 generated samples per step, 64-sample training minibatches, and 16 rollouts.This configuration produces two updates per RL step.Source: B Parameter Setting · p. 23, Table 4 · p. 24
• DAPO adds high- and low-clip ratios, an overlong penalty, and response-length limits, with P_length(y) added to the binary reward.The overlong penalty is defined for response length y and corresponds to DAPO’s L_max and L_cache settings.Source: B Parameter Setting · p. 23

C.1 Benchmark Comparison

• The RL dataset aligns more closely with RTLLM than VerilogEval in embedding distributions, while this comparison reflects centroid similarity rather than data homogenization or overfitting.Figure 5 compares NL problem distributions on the left and code solution distributions on the right; Table 5 reports centroid distance and similarity.Source: §C.1 · pp. 24–25, Figure 5 · p. 24, Table 5 · p. 24
• VerilogEval contains more table- and graph-based problems, where the model underperforms, motivating additional KMap, FSM, and waveform examples in training.The proposed improvement direction is to add high-quality RL data for these problem types.Source: Table 6 · p. 25, §C.1 · p. 25
• RTLLM is generally more complex than VerilogEval by average code-line and token counts, according to the benchmark comparison.The comparison uses code lengths with comments and blank lines removed.Source: Table 7 · p. 25, §C.1 · p. 25

C.2 Additional Benchmark Statistics

• CodeV-R1-7B reduces total VerilogEval compiler errors by 48% versus CodeV-R1-7B-Distill, while runtime errors remain higher than DeepSeek-R1.Syntax errors fall 65% from 107 to 38, and wire-declaration issues fall 57% from 108 to 46.Source: Table 8 · p. 25, §C.2 · p. 25
• CodeV-R1-7B improves pass@1 over its distilled model across all RTLLM v2 module categories, with especially large gains in arithmetic and miscellaneous modules.Arithmetic improves from 69.47% to 83.68%, while miscellaneous improves from 46.94% to 57.78%; both exceed DeepSeek-R1 in these categories.Source: Table 9 · p. 26, §C.2 · p. 26

C.3 Training Dataset Statistics

• The 87K SFT and 3.1K RL datasets have comparable category distributions, while RL prompts are shorter and less variable on average.Average prompt length is 377.81 tokens for SFT versus 336.67 for RL, with standard deviations of 161.30 and 153.88.Source: Figure 6 · p. 26, §C.3 · pp. 26–27, Figure 7 · p. 26
• After reinforcement learning, responses shift toward longer and more symmetric distributions, while incorrect samples remain especially long and often exceed the context threshold.The maximum combined prompt-response context is 16384 tokens, and the authors attribute some dataset-specific overlong behavior to the overlong penalty.Source: Figure 8 · p. 27, §C.3 · p. 27

C.4 Agent Ability Analysis

• The experiment supports CodeV-R1’s potential for integration into agent workflows, while broader end-to-end agent capabilities remain a future direction.MAGE uses golden testbenches, whereas CodeV-R1 operates without them, and MAGE performance depends on prompt engineering and underlying model capability.Source: §C.4 · pp. 27–28
• In the MAGE framework, CodeV-R1 achieves 94.87% Pass@1, compared with Claude-3.5-Sonnet’s 95.51%, and substantially outperforms other baseline models.Source: §C.4 · pp. 27–28

D Case Study

• CodeV-R1-7B correctly implemented the Karnaugh-map function, whereas DeepSeek-R1 produced an incorrect logical function after using the mismatched declaration input [3:0] x.CodeV-R1-7B instead selected input [4:1] x through reflection, avoiding index-mapping ambiguity and producing clearer code.Source: D Case Study · pp. 28, 30
• The dataset includes a low-quality example whose empty-module code is inconsistent with a summarized 32-bit unsigned-divider specification.The summary describes asynchronous clear, clocked operation, 32-bit divisor and dividend inputs, 32-bit quotient and remainder outputs, an 11-stage pipeline, and positive remainder handling.Source: D Case Study · pp. 30–31

E Prompts

• The prompt demonstrations instruct an LLM to generate descriptions and problems whose solutions exactly reproduce provided Verilog snippets, covering ringer control, byte-enabled registers, counters, LFSRs, and vector-neighbor operations.The examples pair code with natural-language descriptions and interface-level specifications, including a cellphone ringer/motor controller, a 16-bit byte-enabled register, a decade counter, a 5-bit Galois LFSR, and 100-bit vector relationships.Source: E Prompts · pp. 32–33
• The example specification defines a 100-bit input and four outputs that test neighboring-bit conjunction, disjunction, difference, and circular comparison relationships.The outputs have widths 99, 99, 100, and 100, respectively, as specified in the signal table.Source: E Prompts · p. 34
• Five demonstrations precede a red-highlighted Verilog snippet, prompting DeepSeek-V3 to generate a matching problem in a process modeled on CodeV’s multi-level summarization [48].Each demonstration pairs a code-snippet description with its corresponding generated problem.Source: E Prompts · p. 34
• The training and testing system prompt requires reasoning in <think>...</think> tags and the final Verilog solution in <answer>...</answer> tags.The final code is further enclosed in a ```verilog code block within the answer tags.Source: E Prompts · p. 34

• CodeV-R1-7B surpasses DeepSeek-R1-671B on both RTLLM v1.1 and RTLLM v2.Source: F Broader Impacts · p. 35
• Generated hardware code may functionally misalign with user intentions, contain security vulnerabilities, or be misused for harmful designs.Source: F Broader Impacts · p. 35
• The authors recommend rigorous verification, security audits, access controls, usage monitoring, and human oversight for generated code.Source: F Broader Impacts · p. 35

• The automated testbench improves code–NL semantic consistency only probabilistically, so synthetic SFT and RL data may still contain low-quality examples.Source: G Limitations and Future Work · p. 35
• Reasoning-data collection depends on a general reasoning teacher whose domain limitations can affect data quality and whose use may be financially costly.Source: G Limitations and Future Work · p. 35
• Future work may extend reasoning LLMs beyond RTL generation to PPA optimization and analog circuit synthesis.Source: G Limitations and Future Work · p. 35