Source-linked AI summary

Bee: A High-Quality Corpus and Full-Stack Suite to Unlock Advanced Fully Open MLLMs

Yi Zhang, Bolin Ni, Xin-Sheng Chen, Heng-Rui Zhang, Yongming Rao, Houwen Peng, Qinglin Lu, Han Hu, Meng-Hao Guo, Shi-Min Hu

arXiv:2510.13795v4cs.CVcs.AI

TL;DR

Open-source MLLM development faces a data-quality gap, with noisy SFT datasets and insufficient complex reasoning data. The paper introduces Honey-Data-15M and its HoneyPipe/DataStudio curation stack, and trains Bee-8B, which achieves SOTA among fully open MLLMs while competing with semi-open models such as InternVL3.5-8B (Wang et al., 2025b).

  • Problem

    Existing open-source SFT datasets contain widespread noise and insufficient complex reasoning data, hindering open-source MLLM development.

    Source: §1 · pp. 1–2
  • Method

    The paper builds Honey-Data-15M through systematic cleaning and dual-level short- and long-CoT enrichment using the modular HoneyPipe and DataStudio pipeline.

    Source: §1 · p. 2, §2 · p. 3
  • Results

    Bee-8B establishes a new SOTA among fully open MLLMs and performs on par with or surpasses recent semi-open models such as InternVL3.5-8B (Wang et al., 2025b).

    Source: §1 · pp. 2–3
  • Takeaways & Limitations

    The work provides an open dataset, curation pipeline, framework, training recipes, evaluation harness, and model weights for developing competitive fully open MLLMs.

    Source: §1 · p. 3, §7 · p. 10

    The single-pass verifier may discard valid CoT samples, reflecting a deliberate precision and cost-effectiveness tradeoff over recall.

    Source: §E.1 · p. 38

Abstract

from arXiv · show

Fully open multimodal large language models (MLLMs) currently lag behind proprietary counterparts, primarily due to a significant gap in data quality for supervised fine-tuning (SFT). Existing open-source datasets are often plagued by widespread noise and a critical deficit in complex reasoning data, such as Chain-of-Thought (CoT), which hinders the development of advanced model capabilities. Addressing these challenges, our work makes three primary contributions. First, we introduce Honey-Data-15M, a new SFT dataset comprising approximately 15 million QA pairs, processed through multiple cleaning techniques and enhanced with a novel dual-level (short and long) CoT enrichment strategy. Second, we introduce HoneyPipe, the data curation pipeline, and its underlying framework DataStudio, providing the community with a transparent and adaptable methodology for data curation that moves beyond static dataset releases. Finally, to validate our dataset and pipeline, we train Bee-8B, an 8B model on Honey-Data-15M. Experiments show that Bee-8B establishes a new state-of-the-art (SOTA) for fully open MLLMs, achieving performance that is competitive with, and in some cases surpasses, recent semi-open models such as InternVL3.5-8B. Our work delivers to the community a suite of foundational resources, including: the Honey-Data-15M corpus; the full-stack suite comprising HoneyPipe and DataStudio; training recipes; an evaluation harness; and the model weights. This effort demonstrates that a principled focus on data quality is a key pathway to developing fully open MLLMs that are highly competitive with their semi-open counterparts.

1 INTRODUCTION

Fully open MLLMs lag behind proprietary and semi-open models because open SFT data contains substantial noise, lacks complex-reasoning supervision, and is supported by opaque curation methods. The paper addresses these gaps with Honey-Data-15M, HoneyPipe/DataStudio, and Bee-8B, which together target competitive fully open MLLM performance.

Motivation and challenges

  • Open MLLMs lag behind proprietary and semi-open models because existing SFT data is noisy, lacks large-scale long-CoT supervision, and is difficult to curate transparently.Reported issues include factual inaccuracies, image-instruction mismatches, structural flaws, degraded reasoning, poor instruction following, and insufficient data for identifying tasks requiring deep multistep reasoning.Source: §1 · pp. 1–2

Validation

  • Bee-8B, trained on the full Honey-Data-15M dataset, establishes a new SOTA among fully open MLLMs and is competitive with or surpasses semi-open models such as InternVL3.5-8B (Wang et al., 2025b).Ablations show significant gains across multiple benchmarks over original, unprocessed data, supporting data quality as a strategy for narrowing the performance gap.Source: §1 · pp. 2–3

HoneyPipe and DataStudio

  • HoneyPipe, instantiated through DataStudio, offers a transparent, adaptable, MLLM-driven curation workflow that automates cleaning and reasoning enrichment.The pipeline uses fidelity verification to route complex instructions from short-CoT processing into a long-CoT enrichment loop when needed.Source: Figure 1 · p. 3

2 HONEY-DATA-15M AND HONEYPIPE

HoneyPipe is an automated, reproducible DataStudio workflow that cleans multimodal data and enriches it with short- and long-level Chain-of-Thought reasoning. It produces Honey-Data-15M, a 15-million-sample corpus spanning diverse domains and reasoning complexities.

2 HONEY-DATA-15M AND HONEYPIPE

  • HoneyPipe is an automated, reproducible DataStudio pipeline that transforms raw data into a transparent, adaptable, dual-level-CoT SFT dataset.Its architecture combines large-scale short-CoT enrichment with a specialized long-CoT loop for complex instructions.Source: §2 · p. 3

2.1 STAGE 1: DATA AGGREGATION AND PREPARATION

  • The pipeline aggregates approximately 24 million image-text pairs, removes pair-level duplicates using perceptual-image and textual-similarity hashes, and assigns source-level domain labels.The initial pool includes community datasets such as LLaVA-OneVision (Li et al., 2025a), PixMo (Deitke et al., 2025), and MAmmoth-VL (Guo et al., 2025c).Source: §2.1 · pp. 3–4

2.2 STAGE 2: NOISE AND IRRELEVANCE FILTERING

  • Noise filtering combines rules for image and text defects with Qwen2.5-VL-72B (Bai et al., 2025) checks for instruction answerability and image-instruction consistency.The combined operators produce a clean set of image-instruction pairs for enrichment.Source: §2.2 · p. 4

2.3 STAGE 3: SHORT COT ENRICHMENT AND VERIFICATION

  • Short-CoT enrichment uses Qwen2.5-VL-72B/32B to generate step-by-step responses, while Qwen2.5-VL-72B verifies factual agreement or semantic consistency with original answers.Samples failing verification are routed to long-CoT enrichment rather than immediately discarded, yielding approximately 12.1 million short-CoT samples.Source: §2.3 · p. 4

2.4 STAGE 4: LONG COT ENRICHMENT LOOP

  • The long-CoT loop processes failed verifications and preidentified complex sources, uses top proprietary MLLMs for deeper reasoning, and retains approximately 2.9 million responses that pass final fidelity verification.Responses failing final verification are discarded as erroneous, unsolvable, or too costly to annotate.Source: §2.4 · pp. 4–5

2.5 HONEY-DATA-15M: HIGH QUALITY CORPUS WITH THE DUAL-LEVEL COT

  • Honey-Data-15M comprises 15 million curated multimodal SFT samples, including approximately 12.1 million short-CoT and 2.9 million long-CoT examples.The corpus applies dual-level reasoning across domains including General and STEM.Source: §2.5 · p. 5

3 BEE-8B TRAINING RECIPE

Bee-8B is an 8B multimodal model trained through a five-stage recipe designed to progress from vision-language foundation building to complex reasoning and output refinement. Its architecture combines Qwen3-8B, SigLIP2-so400mpatch14-384, Anyres, and a two-layer GELU MLP projector.

3.1 MODEL ARCHITECTURE

  • Bee-8B uses a LLaVA-OneVision-inspired architecture with Qwen3-8B for reasoning and text generation, SigLIP2-so400mpatch14-384 for vision, Anyres for variable-resolution images, and a two-layer GELU MLP projector.The projector maps aggregated visual features into the LLM’s token embedding space.Source: §3.1 · p. 5

3.2 MODEL TRAINING

  • The five-stage recipe progressively builds a vision-language foundation, instills complex reasoning with Honey-Data-15M, and polishes outputs for reliability.The stages are detailed in Table 1.Source: §3.2 · p. 6, Table 1 · p. 6
  • Training proceeds from projector-only MLP warmup and full-parameter vision-language alignment to Honey-Data-15M SFT, Honey-Data-1M refinement SFT, and GRPO optimization.The third stage targets dual-level CoT reasoning, the fourth uses a rationally distributed subset, and GRPO mitigates issues such as text repetition.Source: §3.2 · p. 6, Table 1 · p. 6

4 EXPERIMENTS

Experiments show that Bee-8B sets a new bar among fully open MLLMs and competes strongly with semi-open models, especially in factual accuracy and complex reasoning. Ablations further attribute these gains to HoneyPipe’s filtering, selection, and CoT enrichment.

4.1 EVALUATION SETUP

  • Evaluation uses a customized VLMEvalKit with thinking mode, a 16,384-token maximum response length, and Qwen3-32B non-thinking LLM judging for benchmarks such as DocVQA.Additional evaluation details are provided in Section H.Source: §4.1 · p. 7

4.2 EVALUATION OF MODEL CAPABILITY

  • 50.7 on MMMU-Pro leads Qwen2.5-VL-7B by 3.6%, while Bee-8B also ranks first on CountBench at 93.0, MMVet at 83.9, and RealWorldQA at 73.1.On broader benchmarks, it scores 66.8 on MMMU and 71.4 on MMStar.Source: §4.2 · p. 7
  • 84.8 and 57.3 on CharXiv’s DQ and RQ tasks rank first, with RQ nearly 12% above Keye-VL’s 45.4; ChartQA reaches 86.7.These results demonstrate strong chart analysis and structured-document comprehension.Source: §4.2 · p. 7
  • 67.0 on MathVerse surpasses InternVL3.5-8B by 5.5%, while Bee-8B leads LogicVista at 61.3 and DynaMath at 41.3.The results connect CoT-enriched Honey-Data-15M with strong quantitative and logical reasoning performance.Source: §4.2 · pp. 7–8
  • Bee-8B delivers across-the-board gains over fully open models and remains highly competitive with recent semi-open MLLMs, with strongest advantages in factual accuracy and complex reasoning.These comparisons are reported in the comprehensive benchmark results of Table 2.Source: §4.2 · pp. 7–8, Table 2 · p. 8

4.3 ABLATION STUDY

  • Dcurated outperforms Dno-CoT, which outperforms Draw, showing that filtering and selection improve performance while CoT enrichment adds a further gain, especially on MathVista and CharXiv-RQ.The ablation compares Draw’s 1.2M raw samples with 960K-sample Dcurated and Dno-CoT subsets.Source: §4.3 · p. 8, Figure 4 · p. 9
  • Honey-Data-1M outperforms Random-1M and the Qwen2.5-VL-7B base model, surpassing the base model on nearly half of evaluated benchmarks.This demonstrates that curated data selection can cultivate advanced reasoning with only 1M samples.Source: §4.3 · pp. 8–9, Figure 5 · p. 9

5 RELATED WORK

MLLM progress has shifted from architectural innovation toward data quality and curation, as fully open models remain constrained by the scale and quality of public data. This work therefore focuses on a reproducible, cost-conscious data pipeline to narrow the gap with proprietary and semi-open models.

Data-centric MLLM development

  • Fully open MLLMs lag because their public training data is smaller and lower-quality than the private data used by proprietary and semi-open models.The passage attributes the performance hierarchy primarily to data access and curation pipelines rather than architectural differences.Source: §5 · p. 9

Open-source MLLM architectures

  • LLaVA (Liu et al., 2023c) established the open-source paradigm of a frozen vision encoder, lightweight connector, and visual instruction tuning for practical multimodal assistants.This paradigm showed that GPT-generated instructions could support multimodal assistant development.Source: §5 · p. 9

Origins of MLLMs

  • The field’s modern trajectory began with GPT-4V (OpenAI, 2023), which demonstrated that general language models could process visual inputs.MLLMs have since shown potential across diverse real-world scenarios.Source: §5 · p. 9

6 AVAILABILITY

The authors commit to open science by publicly releasing the complete resources needed to reproduce, analyze, and extend the work.

  • The release includes the complete Honey-Data-15M dataset and final Bee-8B model weights.Source: §6 · p. 10
  • HoneyPipe, DataStudio, and all scripts, configurations, and code for training and evaluation will be released.Source: §6 · p. 10
  • Intermediate checkpoints for every training stage and the corresponding training-data subsets will support detailed analysis and community research.Source: §6 · p. 10

7 CONCLUSION … A.1 ABLATION OF DIFFERENT REASONING MODES AND STAGES

The paper presents Honey-Data-15M and HoneyPipe as a transparent, quality-focused foundation for fully open MLLMs, validated by Bee-8B’s strong reasoning performance. Additional ablations show that refinement SFT and reinforcement learning progressively improve capability and reliability across reasoning modes and benchmarks.

7 CONCLUSION

  • Bee-8B establishes a new state-of-the-art among fully open models and is highly competitive with leading semi-open models, particularly in complex math and reasoning.The conclusion attributes this performance directly to targeted dual-level CoT enrichment and reports that ablations confirm the impact of the curation process.Source: §7 · p. 10
  • Honey-Data-15M contains 15M cleaned SFT samples enriched with dual-level CoT reasoning and constructed through the open-source HoneyPipe pipeline.The work argues that prioritizing data quality through transparent, reproducible methods is more effective than competing primarily on data volume.Source: §7 · p. 10

A ADDITIONAL EXPERIMENTS

  • The additional experiments examine contributions from training stages and model behavior under varied inference conditions to validate the training design and reasoning capabilities.The analysis focuses on the initial SFT checkpoint, refinement SFT checkpoint, and final reinforcement-learning checkpoint.Source: A ADDITIONAL EXPERIMENTS · p. 33, §A.1 · p. 33

APPENDIX

  • The appendix provides granular ablations that connect data curation, training stages, inference modes, and policy optimization to Bee-8B’s advanced reasoning behavior.These analyses collectively support the paper’s data-centric and multi-stage training approach.Source: A ADDITIONAL EXPERIMENTS · p. 33, §A.1 · p. 34

A.1 ABLATION OF DIFFERENT REASONING MODES AND STAGES

  • The ablation compares Stage 3, Stage 4, and Stage 5 models using short CoT and long CoT inference modes across general VQA, table, chart, and document benchmarks.Short CoT uses an explicit empty think block, whereas long CoT prompts the model to produce a more detailed reasoning process.Source: §A.1 · p. 33, Table 3 · p. 33, Table 4 · p. 34
  • The model performs strongly in both inference modes, indicating alignment between its vision module and the Qwen3 LLM’s dual-level reasoning capabilities.The benchmark comparisons cover general VQA as well as table, chart, and document understanding.Source: §A.1 · p. 33, Table 4 · p. 34, Table 5 · p. 34
  • Stage 4 refinement and Stage 5 reinforcement learning progressively improve performance, with reinforcement learning especially benefiting complex math, reasoning, and generation reliability.The gains are attributed to a curated 1M subset and reduced response repetition, supporting the necessity of the multi-stage recipe.Source: §A.1 · pp. 33–34

A.2 SENSITIVITY ANALYSIS OF JUDGE MODELS … B THE USE OF LARGE LANGUAGE MODELS

Across evaluator sensitivity, repeated runs, human assessment, contamination checks, and verifier agreement, the reported results support Bee-8B and HoneyPipe’s robustness, reproducibility, data quality, and reliability. The manuscript also states that LLMs were used only for language refinement, formatting, and naming assistance.

A.2 SENSITIVITY ANALYSIS OF JUDGE MODELS

  • Cross-family evaluation with GLM-4.5-FP8 was conducted to test potential self-preference bias from using Qwen3-32B to evaluate Bee-8B across all 29 benchmarks.The comparison covered both Bee-8B-SFT and Bee-8B-RL under the original Qwen judge and the independent GLM judge.Source: §A.2 · p. 34
  • Bee-8B-RL’s Global Average changed by only -0.3 points, from 70.2 with the Qwen judge to 69.9 with the GLM judge, across 29 benchmarks.Although GLM was stricter on ChartQA, DocVQA, and InfoVQA, the authors report that evaluator differences had minimal global impact and did not explain the model’s strong performance.Source: §A.2 · pp. 34–35, Table 7 · p. 35

A.3 EVALUATION STABILITY ANALYSIS

  • Five independent inference runs produced extremely low standard deviations and stable global averages for both Bee-8B-SFT and Bee-8B-RL across all benchmarks.Table 8 reports each run, the mean, and the standard deviation, supporting the authors’ conclusion that performance is consistent and reproducible.Source: §A.3 · p. 35, Table 8 · p. 36

A.4 HUMAN EVALUATION OF DATA QUALITY

  • Human evaluators preferred enriched responses for Reasoning (72.74%) and Expression Style (69.92%), while Accuracy produced an 83.65% Tie rate.The blind study involved 14 valid evaluators, 532 pairwise comparisons, and 38 randomly sampled questions across four quality dimensions.Source: §A.4 · pp. 35–36, Table 9 · p. 36

A.5 DATA CONTAMINATION ANALYSIS

  • 29 potential overlaps (<0.05%) and 2 exact matches were found among 66,682 evaluation samples, while MMMU, MMMU-Pro, MMStar, and CharXiv remained overlap-free.The authors conclude that this negligible overlap could not materially affect performance metrics and state that evaluation data was never intentionally included in training.Source: §A.5 · pp. 36–37, Table 10 · p. 37

A.6 INTER-MODEL AGREEMENT ON VERIFICATION

  • The three-verifier agreement included 63,203 retained samples, representing 96.5% of the reference model’s retained data and 91.3% of the chosen verifier’s.Agreement across Qwen2.5-VL-32B, Qwen2.5-VL-72B, and Qwen3-VL-235B-A22B-Instruct supports the claim that inconsistencies were mainly objective flaws rather than model-specific bias.Source: Table 11 · p. 37, §A.6 · p. 37

B THE USE OF LARGE LANGUAGE MODELS

  • The manuscript used LLMs for language polishing, clarity and grammar checks, idiomatic scientific phrasing, minor LaTeX formatting, and dataset and model naming discussions.No training, evaluation, or data-curation use is described in this disclosure.Source: B THE USE OF LARGE LANGUAGE MODELS · p. 37

C REPRODUCIBILITY STATEMENT · D ETHICS STATEMENT · E LIMITATIONS

The work commits to full reproducibility by releasing its data, models, checkpoints, training details, and code, while acknowledging dataset biases and potential harmful or inaccurate model outputs. It follows the ICLR Code of Ethics, involves no human or animal subjects, and promotes transparency and responsible use.

C REPRODUCIBILITY STATEMENT

  • The authors will publicly release Honey-Data-15M, Bee-8B weights, intermediate checkpoints, stage-specific training data, HoneyPipe code, and DataStudio code.This release is intended to enable full reproduction of the work.Source: C REPRODUCIBILITY STATEMENT · p. 38
  • The reproducibility release includes the specific training data used at each intermediate stage and the corresponding checkpoints.These materials expose the intermediate states of the stated training recipe, not only the final model.Source: C REPRODUCIBILITY STATEMENT · p. 38

D ETHICS STATEMENT

  • The work adheres to the ICLR Code of Ethics and involved no human or animal subjects.Its data sources are public academic datasets and MLLMs.Source: D ETHICS STATEMENT · p. 38

E LIMITATIONS

  • Honey-Data-15M may inherit biases from public sources despite filtering noise, factual errors, and image-instruction mismatches.Users should therefore remain aware of possible dataset limitations.Source: D ETHICS STATEMENT · p. 38
  • Bee-8B may generate inaccurate or harmful content, creating a limitation users should consider.The statement does not claim that pipeline filtering eliminates these model-output risks.Source: D ETHICS STATEMENT · p. 38

D ETHICS STATEMENT

  • The authors release the full suite and methodology to support open academic research, transparency, and community efforts to understand and mitigate model limitations.They strongly discourage malicious use of the released resources.Source: D ETHICS STATEMENT · p. 38

E.1 LIMITATION OF FIDELITY VERIFICATION · E.2 LIMITATION OF EVALUATION PARADIGM

The pipeline’s fidelity verification prioritizes precision and scalability over recall, while LLM-based evaluation introduces unavoidable subjectivity and nondeterminism. These limitations leave standardized, objective metrics for reasoning models as an open challenge.

E.1 LIMITATION OF FIDELITY VERIFICATION

  • E.1 LIMITATION OF FIDELITY VERIFICATION: Automated LLM-based fidelity verification requires trading valid-sample recall against computational cost.The pipeline uses an automated “LLM-as-a-Judge” mechanism for Fidelity Verification.Source: §E.1 · p. 38
  • E.1 LIMITATION OF FIDELITY VERIFICATION: A single-pass verifier can discard correct CoT samples when formatting, units, or synonyms differ from the original answer.Human annotation or verifying dozens of candidates could improve recall but is infeasible at tens-of-millions scale.Source: §E.1 · p. 38
  • E.1 LIMITATION OF FIDELITY VERIFICATION: The pipeline therefore prioritizes factual precision and cost-effectiveness, sacrificing some valid enriched data for scalability, reproducibility, and accessibility.This design avoids requiring industrial-scale resources.Source: §E.1 · p. 38
  • E.1 LIMITATION OF FIDELITY VERIFICATION: Although diverse verifier ensembles could improve robustness, a single strong open-source model balances high data quality against engineering complexity.The authors report that strong MLLMs generally agree on factual consistency, while disagreements typically indicate objective flaws or major deviations.Source: §E.1 · p. 38

E.2 LIMITATION OF EVALUATION PARADIGM

  • E.2 LIMITATION OF EVALUATION PARADIGM: Regex-based evaluation is insufficient for verbose CoT outputs because extraction failures can produce false negatives despite sound reasoning.LLM-based semantic consistency checking is presented as a necessary evolution for evaluating advanced reasoning models.Source: §E.2 · p. 38
  • E.2 LIMITATION OF EVALUATION PARADIGM: LLM judges make evaluation nondeterministic and dependent on judge capability, introducing subjectivity about formatting, semantic equivalence, and possible self-preference bias.Unlike rigid code-based metrics, their judgments are not fully objective.Source: §E.2 · p. 38
  • E.2 LIMITATION OF EVALUATION PARADIGM: Reasoning-model metrics are bounded by evaluator subjectivity, making standardized, objective, automated evaluation a critical open challenge.The challenge is especially important as reasoning models continue to evolve rapidly.Source: §E.2 · p. 39

F DETAILED TRAINING STAGE CONFIGURATIONS … F.5 STAGE 5: REINFORCEMENT LEARNING WITH GRPO

Bee-8B uses a five-stage progressive curriculum spanning visual-language alignment, multimodal SFT, efficient refinement, and GRPO reinforcement learning. The configurations progressively combine curated multimodal data, reasoning-focused text, targeted sampling, and rule-based rewards.

F DETAILED TRAINING STAGE CONFIGURATIONS

  • The overall recipe follows a progressive five-stage curriculum from foundational vision-language alignment through complex instruction SFT, targeted refinement, and reinforcement learning.The configuration section specifies the data sources and key settings for each stage to support clarity and reproducibility.Source: F DETAILED TRAINING STAGE CONFIGURATIONS · p. 39

F.1 STAGE 1: MLP WARMUP

  • Stage 1 trains only the MLP projector on approximately 1M image-caption pairs while freezing the vision encoder and LLM backbone.The data combines roughly 560K LLaVA-OneVision samples with around 440K recaptioned and rule-filtered COYO samples.Source: §F.1 · p. 39

F.2 STAGE 2: VISION-LANGUAGE ALIGNMENT

  • Stage 2 unfreezes all components and trains on approximately 12.6M vision-language pairs plus 1.43M text-only samples, preserving reasoning through long- and short-CoT Nemotron data.The vision-language data combines filtered LAION, COYO, and LLaVA-OneVision sources, while Nemotron-STEM, Nemotron-Math, Nemotron-Chat, and short-CoT Nemotron-Chat support reasoning and conversational ability.Source: §F.2 · p. 39

F.3 STAGE 3: MULTIMODAL SFT

  • Stage 3 trains for one full epoch on the entire Honey-Data-15M to develop advanced multimodal instruction-following and reasoning, including rare long-CoT patterns.Its dataset composition and dual-level CoT distribution are detailed in Fig. 3 and Tab. 12.Source: §F.3 · p. 39

F.4 STAGE 4: EFFICIENT REFINEMENT SFT

  • Stage 4 constructs a computationally efficient 1M subset using topic quotas, source-quality scores, and hybrid sampling that selects 60% longest responses and 40% random examples.The strategy preserves source diversity while emphasizing difficult conversations and maintaining a more rational topic distribution.Source: §F.4 · pp. 39–41

F.5 STAGE 5: REINFORCEMENT LEARNING WITH GRPO

  • Stage 5 applies GRPO with 512-sample rollout batches and 128-sample policy-update batches to address repetition, incomplete responses, and formatting errors.The process uses verl and candidate responses generated from MMK12 and a filtered, 95%-training split of ViRL39K.Source: §F.5 · p. 41
  • Stage 5 uses format and accuracy rewards weighted 0.2 and 0.8, respectively, requiring \boxed{} output and matching its extracted answer to ground truth.This targeted reward design provides final refinement of response reliability and output quality.Source: §F.5 · p. 41

G DATA LICENSING AND GOVERNANCE … Prompt for LLM-based Answer Grading

The paper pairs Honey-Data-15M with a dual-layer licensing and governance framework that preserves source-specific restrictions while licensing eligible modifications under CC-BY-NC-4.0. It also evaluates the model across broad VQA, OCR, chart, table, math, and reasoning benchmarks using customized judging procedures and a transparent final-answer grading prompt.

G DATA LICENSING AND GOVERNANCE

  • Honey-Data-15M remains governed by each constituent dataset’s original license, while eligible modified prompts, formatting, and generated CoT responses use CC-BY-NC-4.0.The corpus was aggregated largely from open-source collections such as LLaVA-OneVision and MAmmoth-VL, with fine-grained source tracing for attribution.Source: G DATA LICENSING AND GOVERNANCE · p. 41
  • The governance policy relies on original-curator anonymization for PII and public academic datasets for OCRed textbooks and K-12 material, while acknowledging potential future copyright or privacy concerns.A dedicated notice-and-takedown channel supports review and removal of contested content after valid infringement or PII notices.Source: G DATA LICENSING AND GOVERNANCE · pp. 41–42

H EVALUATION

  • The evaluation covers General VQA, Table & Chart & OCR, and Math & Reasoning benchmarks, with the complete benchmark grouping listed in Table 20.This organization is intended to assess model capabilities across a broad range of task types.Source: H EVALUATION · p. 48

H.1 EVALUATION DETAIL

  • The evaluation uses separate non-thinking and thinking inference configurations, setting temperatures to 0 and 0.6 and maximum output lengths to 8,192 and 16,384 tokens, respectively.The thinking configuration is designed to elicit more detailed reasoning.Source: §H.1 · p. 48
  • A customized VLMEvalKit (Duan et al., 2024) extends LLM judging to ChartQA, InfoVQA, DocVQA, and CountBench and fixes MathVerse extraction and judging errors with broader test cases.These modifications aim to improve evaluation coverage and reliability across benchmarks.Source: §H.1 · p. 48

H.2 PROMPT FOR LLM-BASED EVALUATION

  • The provided grading prompt evaluates only whether a candidate’s final answer matches the standard answer, allowing equivalent valid formats and assigning A, B, or C for correct, incorrect, or invalid responses.It treats incomplete, irrelevant, repetitive, or otherwise unusable responses as invalid and explicitly ignores intermediate reasoning errors.Source: §H.2 · p. 48, Prompt for LLM-based Answer Grading · p. 49

H.3 BENCHMARKS … K.1 FIDELITY VERIFICATION-FAILED DATA

The paper defines three benchmark categories and provides prompts for filtering noisy, irrelevant, low-quality, unanswerable, and fidelity-inconsistent multimodal data. Case studies illustrate these failure modes, including missing responses, poor images, repetition, image-instruction mismatch, unanswerable questions, and conflicting numerical answers.

H.3 BENCHMARKS

  • The evaluation covers General VQA, Table & Chart & OCR, and Math & Reasoning benchmarks, with the complete grouping listed in Table 20.These categories organize the study’s comprehensive benchmark evaluation.Source: §H.3 · p. 49, Table 20 · p. 50

I PROMPT

Prompt for Noise and Irrelevance Filtering

  • The noise filter removes image-question pairs that are unrelated, vague, non-question inputs, or contain languages other than Chinese or English.It returns false for relevant, clear pairs without issues and true for pairs failing relevance, ambiguity, contradiction, irrelevance, or language criteria.Source: Prompt for Noise and Irrelevance… · p. 51

Prompt for Fidelity Verification

  • The fidelity prompt retains answers when their facts, numerical values, reasoning conclusions, chart content, or comparisons remain consistent, and filters contradictions or fabricated content.Open-ended answers are retained unless clearly irrelevant; precise answers allow equivalent formats, minor rounding within 0.1, and valid unit conversions.Source: Prompt for Fidelity Verification · p. 52

J NOISE DATA CASE STUDY

  • HoneyPipe’s noise-case study removes malformed records, excessively low-quality images, and responses containing repeated text segments.The examples include missing responses, an image that is too small, and repeated reasoning passages.Source: §J.1 · p. 53, Case: Format flaw · p. 53, §J.2 · p. 53

J.4 IMAGE-INSTRUCTION MISMATCH

  • The pipeline filters image-instruction mismatches when instructions are unrelated to the image, as illustrated by a landscape question paired with an escalator image.Such mismatches impair alignment between visual features and semantic information.Source: §J.4 · p. 56, Case: Image-instruction mismatch · p. 56

J.5 UNANSWERABLE INSTRUCTION

  • It also removes unanswerable or low-quality instructions when images are corrupted or when questions can be answered without extracting visual information.Examples include an unlabeled parliamentary-seat pie chart and a Saturn image unrelated to a question about Titan’s discovery.Source: §J.5 · p. 56, §J.6 · p. 56, Case: Unanswerable instruction · p. 57

K.1 FIDELITY VERIFICATION-FAILED DATA

  • Fidelity verification rejects enriched responses that conflict with the original answer, exemplified by a radius answer of 18 conflicting with the original precise answer of 9.The example is a factual query whose enriched response introduces an inconsistent numerical value.Source: §K.1 · p. 58, Case: Factual inaccuracy · p. 58

K.2 SHOWCASE OF HIGH-QUALITY ENRICHED DATA … 3. Select Categories to Adjust

This section showcases Honey-Data-15M’s high-quality enriched multimodal responses across diverse tasks, including visual interpretation and short- or long-form reasoning. It also describes task-specific category rebalancing and illustrates grounded counting through an example of enumerating monitors.

K.2 SHOWCASE OF HIGH-QUALITY ENRICHED DATA

  • Honey-Data-15M showcases diverse enriched responses that provide more accurate answers, deeper knowledge, and detailed long-chain reasoning.The collection includes both short- and long-CoT examples across multimodal topics.Source: §K.2 · p. 59

Case: Caption (with short CoT)

  • A caption example combines a concise scene description with extended details about desert elements, visible text, and the image’s reflective spiritual interpretation.The response identifies dunes, a leafless tree, church-related text, and themes of patience, endurance, and faith.Source: Case: Caption (with short CoT) · p. 59

Case: General (with short CoT)

  • General visual reasoning expands an uncertain observation of people into evidence-based hypotheses about a managed public event, including crowd organization, traffic control, and decorations.The response considers walking or standing, possible parades or festivals, traffic signs and barriers, casual attire, and balloons.Source: Case: General (with short CoT) · p. 60

Case: Grounding & Counting (with short CoT)

  • Grounding and counting reasoning identifies scene objects, compares their matte appearances, and concludes that the large blue cube shares the small red sphere’s material.The final answer is yes.Source: Case: Grounding & Counting (with short… · p. 61

Case: Chart (with long CoT)

  • Long-CoT chart reasoning computes summed accuracies of 7 for artery and 17 for steel, selecting artery as the algorithm with the smaller total.The calculation compares bird and poet dataset values before giving the requested answer.Source: Case: Chart (with long CoT) · p. 62

Case: STEM (with long CoT)

  • Long-CoT STEM reasoning applies the median-to-hypotenuse property and the Pythagorean theorem to derive CD = 4, corresponding to answer C.The derivation uses CE = 5 and the resulting right-triangle relation.Source: Case: STEM (with long CoT) · p. 63

Case: Document (with long CoT)

  • Other long-CoT examples solve document and geometry queries by ordering renewal dates to select Netflix Standard Plan and using perpendicular-bisector equalities to obtain perimeter 10.Netflix follows Spotify’s November 1, 2023 renewal date with November 7, 2023; the geometry example relates AM + MN + AN to BC = 10.Source: Case: Document (with long CoT) · p. 64, Case: STEM · p. 65

1. Types of Visual Illusions Perceived

  • The examples cover ambiguous visual perception, where one contour can be read as two face profiles or a goblet, and dataset redesign tailored to the task’s category-coverage purpose.For rare categories such as OCR, prioritize increasing their share; for broad coverage, rebalance dominant categories and enhance smaller ones.Source: §1 · p. 66, §2 · p. 67

3. Select Categories to Adjust

  • Data distribution is adjusted by oversampling OCR, undersampling General, and rebalancing Chart and Caption to reach the new total.The stated goal is to align the distribution with application needs while maintaining robustness.Source: §3 · p. 67

Case: Grounding & Counting

Loading 2510.13795v4…