PaperlayerView exact revision on arXiv

Source-linked AI summary

K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs

Hao Liang, Qihan Lin, Zhaoyang Han, Xiaochen Ma, Zhen Hao Wong, Meiyi Qiang, Linzhuang Sun, Wentao Zhang

arXiv:2605.09635v3cs.CL

TL;DR

Existing K–12 benchmarks test exam question answering but not curriculum cognition: understanding structured knowledge and textbook visuals. K12-KGraph addresses this gap with a curriculum-aligned graph, benchmark, and training corpus, while experiments show current models remain weak and graph-guided textual and visual supervision improves educational performance.

  • Problem

    Existing K–12 benchmarks test exam question answering but not curriculum knowledge structure or textbook visual grounding.

  • Method

    The paper extracts K12-KGraph from official Chinese K–12 textbooks and derives graph-based evaluation questions and supervised training data.

  • Results

    K12-Train-Text outperforms matched mainstream instruction-tuning subsets, while K12-Train-Full reaches 39.9% Gaokao-MM accuracy and surpasses text-only and multimodal-only variants.

  • Takeaways & Limitations

    The results indicate that curriculum-structured training transfers across educational benchmarks and that textual and visual supervision provide complementary benefits.

Abstract

from arXiv · show

Large language models are increasingly used in K-12 education, but existing benchmarks mainly test exam question answering rather than understanding how curriculum knowledge is structured and visually presented. We call this capability curriculum cognition. It covers prerequisite chains, concept taxonomies, experiment-concept links, pedagogical sequencing, and visual grounding. We introduce K12-KGraph, a curriculum-aligned knowledge graph extracted from official People's Education Press textbooks in mathematics, physics, chemistry, and biology across primary, middle, and high school. It contains nine node types and fourteen relation types covering curriculum structure and visual grounding. From this graph, we derive K12-Bench, a 23,640-question multi-select benchmark with five task families: Ground, Prereq, Neighbor, Evidence, and Locate. We also build K12-Train, a graph-guided supervised fine-tuning corpus of 7,335 samples, including 2,267 text-only QA pairs and 5,068 multimodal VQA pairs. On K12-Bench, Gemini-3-Flash achieves only 57 percent exact match and Gemma-4-31B-IT reaches 46 percent, with Prereq and Neighbor being the hardest tasks. Our training experiments show that domain-specific supervision can reduce this gap. Under a matched 2,300-sample budget, K12-Train-Text consistently outperforms equally sized subsets of eight mainstream instruction-tuning corpora on GaokaoBench and EduEval. For vision-language models, K12-Train-Full achieves the best overall results on Gaokao-MM, MDK12-medium, and K12Vista among all compared training configurations, despite using fewer samples than the full DataFlow and WizardLM baselines. It also surpasses both text-only and multimodal-only variants, showing that textual and visual supervision are complementary. We release the graph, benchmark, training data, and complete construction pipeline.

1 Introduction

The paper argues that current K–12 benchmarks miss curriculum cognition: structured and visual understanding of curriculum knowledge. It introduces K12-KGraph and derives graph-based evaluation and training resources from official Chinese K–12 textbooks.

  • Motivation: Current K–12 benchmarks test exam-question answering but do not probe prerequisite structure, experiment–concept links, textbook locations, or visual grounding.The paper defines this missing capability as curriculum cognition.
  • K12-KGraph: The graph encodes curriculum structure and visual grounding, including prerequisite and taxonomic relations alongside figure composition, semantics, location, dependence, and evidence.Its visual component includes Figure and VisualElement nodes and relations such as contains_visual_element, refers_to, illustrates, appears_in, requires_figure, and supports_edge.
  • K12-KGraph: K12-KGraph is a heterogeneous, multi-subject curriculum graph extracted from official People’s Education Press textbooks across primary, middle, and high school.It covers mathematics, physics, chemistry, and biology, with textual and multimodal components.
  • Benchmarks and training: 23,640 questions form K12-Bench, a graph-derived multi-select benchmark organized into five task families.The listed families begin with Ground, Prereq, Neighbor, and Evidence; the supplied passage truncates the final family name.
  • Construction pipeline: The pipeline combines OCR parsing, hierarchical segmentation and merging, LLM extraction, structural validation, and lightweight human verification.This design targets scalability and correctness through automated, rule-based, and human-in-the-loop processing.
  • Results: 7,335 K12-Train samples outperform larger DataFlow and WizardLM baselines on Gaokao-MM, MDK12-Bench, and K12Vista, while textual and visual supervision remain complementary.The reported comparison uses 7,335 samples versus 10,000 and 142,759.

2 Related Work

Prior educational benchmarks largely emphasize question answering and broad capability evaluation, while existing educational knowledge graphs rarely align with official K–12 curricula or ground textbook figures. Existing instruction-tuning datasets are general-purpose or synthetically generated without structured educational knowledge, motivating KG-guided synthesis.

  • K–12 and education benchmarks: Existing Chinese K–12 benchmarks mainly evaluate multiple-choice factual questions or broader educational capabilities, with GaokaoBench additionally covering objective and subjective exam questions.C-Eval and CMMLU include K–12 categories; EduEval evaluates six educational capability dimensions.
  • Educational knowledge graphs: Existing educational knowledge graphs typically target a single subject or English-language courses rather than official K–12 curricula, and rarely ground textbook figures.They generally represent textbook knowledge as textual entities and relations.
  • Instruction tuning datasets: General-purpose instruction-tuning datasets cover diverse tasks but lack domain-specific educational knowledge, while Self-Instruct variants do not leverage structured knowledge sources.The cited general-purpose datasets include OpenHermes, UltraChat, WizardLM, Tulu-3, and SmolTalk.
  • Instruction tuning datasets: The KG-guided synthesis approach is positioned as a way to ground training data in structured knowledge and bridge the limitations of existing instruction-tuning resources.This approach connects supervised fine-tuning with structured educational knowledge sources.

3 K12-KGraph

K12-KGraph is a heterogeneous K–12 property graph integrating textual and multimodal curriculum knowledge through a schema with nine node types and fourteen directed edge types. Its automatic construction pipeline extracts, merges, validates, and expert-verifies textbook-grounded entities and relations.

  • Graph schema: Nine node types and fourteen directed edge types define K12-KGraph’s heterogeneous schema for textual and multimodal educational content.The graph uses typed properties for entities such as concepts and experiments.
  • Graph schema: Typed properties capture textbook-grounded concept information and experiment details, including definitions, importance, formulas, instruments, processes, phenomena, and conclusions.Concept definitions prefer textbook wording, while Experiment nodes record whether students perform the experiment.
  • Construction pipeline: Five automatic stages parse textbook PDFs, split sections and associate images, prompt GPT-5.2 for schema-aware JSON extraction, merge graphs across books, and validate relations.The pipeline preserves headings, formulas, raw text, image assets, and edge-level evidence citations or confidence scores.
  • Validation and quality control: Depth-first cycle detection and manual resolution yield valid DAGs for taxonomic and prerequisite relations, while experts verify triples and reconcile cross-book aliases.Quality control also uses confidence and evidence fields, hallucination constraints, and hierarchical name matching followed by expert review.

4 Benchmark and Training Data from K12-KGraph

K12-Bench and K12-Train derive complementary evaluation and training datasets from K12-KGraph through graph-grounded question construction. The benchmark tests five facets of curriculum cognition, while the training corpus combines node, edge, and exercise-assessment supervision across text and visual modalities.

  • Dataset construction: K12-Bench converts graph textual neighborhoods into multi-select questions, while K12-Train converts node attributes and edge semantics into structurally grounded QA pairs.The shared pipeline instantiates the same prerequisite subgraph into both a Prereq benchmark item and a KG-guided training pair.
  • K12-Bench: 23,640 multi-select items comprise five task families: Ground, Prereq, Neighbor, Evidence, and Locate.Each item presents four labeled candidates and requires the full set of correct labels, with gold answer cardinality ranging from 1 to 3.
  • K12-Bench: The benchmark probes grounding, prerequisite reasoning, neighbor recommendation, experiment evidence chains, and cross-chapter indexing through graph-defined subtasks and distractors.Questions and distractors are instantiated from graph queries rather than generated by an LLM, and graph validation is supported by 12 subject-qualified annotators with Fleiss’ κ = 0.84 overall.
  • K12-Train: K12-Train uses node-grounded, edge-grounded, and deterministic exercise-assessment QA paths to teach content, inter-node structure, and factually unambiguous exercise relations.LLM prompting is used for node and semantic-edge supervision, while tests_concept and tests_skill edges use deterministic templates to avoid fabrication or paraphrase risk.
  • K12-Train: 7,335 QA pairs form K12-Train-Full, partitioned into K12-Train-Text with 2,267 text-only pairs and K12-Train-MM pairs requiring figures or visual elements.K12-Train-Text uses source-balanced random subsampling to control quality and mitigate skew in the KG-derived candidate pool.

5 Experiments and Results

Experiments show that current LLMs struggle with curriculum structure, especially prerequisite and neighbor relations, while K12-Train improves text-only and multimodal educational performance across models and benchmarks. Textual and multimodal supervision are complementary, and the gains extend beyond the four source subjects.

  • K12-Bench: Gemini-3-Flash reaches only 57.1% overall EM on K12-Bench, while Gemma-4-31B-IT reaches 46.4%.Overall F1 peaks at approximately 73%, indicating systematic multi-label errors.
  • K12-Bench: Prereq and Neighbor are hardest, with Gemini-3-Flash below 35% EM on both tasks.Ground and Evidence are comparatively easier, exceeding 75% and 72% F1, respectively, for top models.
  • Text-only fine-tuning: 1009.96 is K12-Train-Text’s highest GaokaoBench total on Qwen3-4B-Base, exceeding DataFlow’s 985.91 by +24.1.It also achieves the best objective score rate (81.8%) and subjective score rate (89.5%).
  • Text-only fine-tuning: 66.76 is K12-Train-Text’s best EduEval average on Qwen3-4B-Base, surpassing WizardLM’s 66.70; on Llama3.1-8B-Base, it reaches 40.90 versus OpenHermes’s 40.05.These improvements remain consistent across both backbones under the same 2,300-sample budget.
  • Multimodal fine-tuning: K12-Train-Full achieves the best overall results on Gaokao-MM, MDK12-Bench, and K12Vista, reaching 39.9%, 52.94, and 79.95, respectively.It outperforms both text-only and multimodal-only variants, while combining textual and multimodal supervision improves grounding and relation-level reasoning.
  • Cross-subject transfer: K12-Train attains 120.18 on Chinese and 132.00 on Humanities Math despite receiving no in-domain supervision for those subjects.The results support cross-subject transfer of structurally grounded answering rather than content memorization.

6 Conclusion · Appendix · A K12-KGraph Construction Details

The paper introduces K12-KGraph, K12-Bench, and K12-Train as a curriculum-aligned graph, benchmark, and KG-guided SFT resource for K–12 education. Experiments indicate weak curriculum cognition in current LLMs and strong sample efficiency from KG-guided synthesis across textual and multimodal settings.

  • 6 Conclusion: K12-Bench evaluates curriculum cognition, while K12-Train supports knowledge-graph-guided supervised fine-tuning.
  • 6 Conclusion: 46% EM is achieved by a strong open-source model on K12-Bench, indicating that current LLMs lack robust curriculum cognition despite strong factual recall.
  • 6 Conclusion: KG-guided synthesis is reported as remarkably sample-efficient across both textual and multimodal settings.
  • 6 Conclusion: Approximately 2,300 samples form the matched budget used to compare K12-Train-Text with equally sized subsets of eight mainstream SFT corpora.
  • 6 Conclusion: K12-Train-Text outperforms equally sized subsets of eight mainstream SFT corpora under the matched approximately 2,300-sample budget.

A.1 Node and Edge Attribute Specification … A.3 Prompt for KG Extraction

The appendix specifies K12-KGraph’s ontology, source-document preprocessing, and extraction prompts. Its multimodal pipeline conservatively filters images, assigns instructional roles, localizes visual elements, and links them to the textual graph using only directly supported evidence.

  • A.1 Node and Edge Attribute Specification: Nine node types and fourteen edge types define K12-KGraph’s high-level ontology, with detailed attributes specified for every node type.The ontology and attribute specifications are summarized in Tables 9 and 10.
  • A.1 Node and Edge Attribute Specification: Textual attributes must be verifiable from textbook passages, while visual descriptions, localization, and textual evidence must be supported by image content and verbatim aligned text.The global constraints prohibit adding information beyond the source text and require rationales for model-inferred visual relations.
  • A.2 Source Documents and Preprocessing Setup: K12-KGraph uses official People’s Education Press PDFs spanning mathematics, physics, chemistry, and biology across primary, middle, and high school.The source files are obtained from a public repository.
  • SINGLE-TURN USER PROMPT FOR TEXTUAL K12-KGRAPH EXTRACTION: Textual extraction reads each textbook section and outputs schema-conformant nodes and edges from a single user-message template containing the detailed graph definitions.The prompt is designed for a knowledge-graph expert and has no separate system prompt.
  • SINGLE-TURN USER PROMPT FOR TEXTUAL K12-KGRAPH EXTRACTION: MinerU’s official pipeline and default CLI settings convert the textbook PDFs into structured text without custom parsing rules or modifications.The preprocessing directly adopts MinerU’s recommended configuration.
  • A.3 Prompt for KG Extraction: Retained figures receive one of six exact instructional-role labels, including explaining a concept, demonstrating an experiment, assisting problem solving, and presenting data.The role taxonomy also includes introducing a real-life context and summarizing and consolidating knowledge; it is not a classification of image format or topic.
  • A.3 Prompt for KG Extraction: Step 3 restricts outputs to supplied candidate nodes, edges, and visual elements, requiring normalized bounding boxes and direct image evidence while instructing the model to output less when uncertain.Relations are omitted when unsupported, and the highest thresholds apply to supports_edges and required_by_exercises.
  • A.3 Prompt for KG Extraction: The multimodal component has three stages: image relevance filtering, instructional-role and visual-element identification, then localization and relation extraction against the textual graph.Figure 4 summarizes this sequence, while the implementation constructs the multimodal component from images, section text, and the extracted textual graph.

B K12-Bench Construction Details and Evaluation Protocol … PROMPT FOR FILTERING K12-BENCH CANDIDATE DISTRACTORS

K12-Bench constructs distractors through a graph-driven, near-to-far pipeline that combines structural sampling, rule-based pruning, semantic ranking, and pedagogical validation. An experienced-teacher prompt retains only candidates judged valid distractors for final option sampling.

  • B.1 Distractor Pool and Structural Sampling Rules: Distractors are generated from graph queries and sampled from structurally relevant, non-gold nodes using a near-to-far expansion principle.The process begins with graph-local neighborhoods before expanding to broader curriculum pools.
  • B.1 Distractor Pool and Structural Sampling Rules: Candidate construction proceeds through pooling, rule-based filtering, pedagogical filtering, deduplication, stable ordering, and option sampling.Rule-based filtering removes gold answers, surface-form duplicates, and task-invalid candidates before LLM-based pedagogical filtering.
  • B.1 Distractor Pool and Structural Sampling Rules: Candidate pools expand from structurally proximate nodes to the same section, chapter, book, subject-stage, or subject only when earlier layers are insufficient.Previously selected candidates are removed, and graph distance uses the undirected union of relates_to, is_a, and prerequisites_for.
  • B.1 Distractor Pool and Structural Sampling Rules: Within each layer, candidates are ranked by averaged semantic similarity to the finalized question and to the most similar gold answer.Ranking uses BAAI/bge-smallzh-v1.5; node representations use names, while Exercise nodes preferentially use stems.
  • B.1 Distractor Pool and Structural Sampling Rules: Ground task distractors begin with 2-hop structural neighborhoods and expand through increasingly broader curriculum scopes while preserving the same node type.The task-specific construction rules vary by task family and direction, including exercise-to-concept and concept-to-exercise Ground instances.
  • B.2 Prompt for Pedagogical Filtering: After rule-based pruning, an LLM independently evaluates each candidate against the question and gold answer set to remove trivial or potentially acceptable options.The filter targets pedagogical validity and unambiguous incorrectness rather than improved semantic similarity.
  • PROMPT FOR FILTERING K12-BENCH CANDIDATE DISTRACTORS: The filtering prompt casts the model as an experienced K–12 teacher judging whether each candidate is a valid distractor for the multiple-choice question.The user prompt supplies the question, gold answer set, and candidate option; INVALID candidates are removed, while VALID candidates proceed to final sampling.

B.3 Benchmark Composition Statistics · B.4 Answering Prompt and Decoding Rules · B.5 Baseline EM/F1 Computation for the Random Predictor

The appendix details K12-Bench’s balanced multi-select composition, standardized answer-only evaluation protocol, and instance-level EM/F1 methodology, including a uniformly sampled random predictor.

  • B.3 Benchmark Composition Statistics: Subject coverage is broadly balanced across task families, while Evidence is concentrated in chemistry, biology, and physics and excludes mathematics.Most tasks distribute items evenly across biology, chemistry, mathematics, and physics.
  • B.3 Benchmark Composition Statistics: K12-Bench uses multi-select questions with four options, with Table 11 reporting sample counts and the graph relation probed by each subtask.Figure 6 additionally summarizes subject distributions and the number of correct options per subtask.
  • B.3 Benchmark Composition Statistics: The construction pipeline balances answer cardinality and label combinations, selecting 1, 2, or 3 correct options before assigning balanced combinations to slots A–D.This procedure prevents a natural skew toward single-answer instances.
  • B.4 Answering Prompt and Decoding Rules: Each evaluation sample is sent through an OpenAI-compatible chat interface as a fixed system message plus a benchmark-instantiated user message.The system prompt requires option-label-only answers, while the user prompt supplies the question stem and four labeled candidates.
  • B.4 Answering Prompt and Decoding Rules: The parser minimally normalizes delimiters and extracts labels without semantic repair; abnormal responses and API failures are automatically retried until valid.Examples of abnormal cases include empty responses.
  • B.5 Baseline EM/F1 Computation for the Random Predictor: F1 is computed per instance from gold and predicted label sets, with F1 set to zero when both precision and recall are zero.The formalization uses gold cardinality k = |G| and predicted cardinality m = |Ĝ|.
  • B.5 Baseline EM/F1 Computation for the Random Predictor: Task- and benchmark-level F1 uses instance-level macro averaging, weighting task averages by instance count and giving every instance equal weight regardless of gold cardinality.This is equivalent to scikit-learn’s average='samples' convention and keeps F1 on the same per-instance footing as EM.
  • B.5 Baseline EM/F1 Computation for the Random Predictor: The random predictor samples uniformly from the 15 non-empty subsets of {A, B, C, D}, with expected scores determined only by gold cardinality because label positions are symmetrically balanced.Thus, the expectations depend on k = |G| rather than the specific option letters.

B.6 Illustrative Example of KG-Grounded Resource Derivation … SINGLE-TURN PROMPT FOR EDGE-LEVEL QA SYNTHESIS

K12-Bench and K12-Train are derived from relation-specific local subgraphs, with benchmark queries preserving graph structure and training examples reformulated as instruction–response pairs. K12-Train QA synthesis uses separate node- and edge-level prompts that enforce factual grounding and relation-specific reasoning.

  • B.6 Illustrative Example of KG-Grounded Resource Derivation: K12-Bench and K12-Train begin by selecting a target relation and extracting its local subgraph before task-specific instantiation under structural constraints.The benchmark preserves the original graph query form, whereas training reformulates the same structure into instruction–response pairs.
  • C K12-Train Construction Details and SFT Protocol: The construction pipeline generates benchmark queries for evaluation and instruction–response pairs for K12-Train supervision from the same underlying graph structure.This shared derivation connects evaluation resources and training data while preserving their different output formats.
  • C.1 Prompt for QA Synthesis: Separate prompt templates generate node-level questions about key attributes and edge-level questions requiring reasoning about relationships.The edge-level prompt is shown with simplified input schema for readability, while implementation uses relation-specific multimodal templates.
  • SINGLE-TURN PROMPT FOR NODE-LEVEL QA SYNTHESIS: Node-level synthesis takes a Concept or Skill and its properties to produce factually grounded question–answer pairs for K–12 learners.The prompt frames the generator as a K–12 education expert transforming complex STEM textbook knowledge into logical QA materials.
  • SINGLE-TURN PROMPT FOR EDGE-LEVEL QA SYNTHESIS: Edge-level synthesis takes a typed relation, endpoint nodes, and properties as inputs for generating grounded question–answer pairs.The specified edge relations include is_a, prerequisites_for, relates_to, verifies, illustrates, refers_to, and requires_figure.
  • SINGLE-TURN PROMPT FOR EDGE-LEVEL QA SYNTHESIS: Each edge-level example contains exactly one question and one answer, remains strictly grounded in input properties, and explains why the edge holds.The constraints prohibit external knowledge and discourage merely restating endpoint names.
  • SINGLE-TURN PROMPT FOR EDGE-LEVEL QA SYNTHESIS: Relation-specific reasoning traces prerequisites through required knowledge, relates_to through concrete conceptual links, and verifies through experimental observations supporting a target concept.These templates require explanations rather than endpoint restatement.
  • SINGLE-TURN PROMPT FOR EDGE-LEVEL QA SYNTHESIS: Illustrates questions connect a figure’s overall visual content to a concept, skill, or experiment by explaining the represented definition, structure, procedure, process, or phenomenon.The answer should explain how the visual content represents the target rather than merely identifying it.

C.2 Training Configuration … D.2 KG Validation: Structural Checks and Human Verification

The experiments use shared configurations and controlled 2,300-instance comparisons, while validation applies tiered checks centered on K12-KGraph, including automatic structural tests, human verification, and confidence-based visual filtering.

  • C.2 Training Configuration: All models within each experiment group share identical hyperparameters for fair comparison, with unspecified settings following the training framework defaults.Table 12 covers text-only full-parameter SFT, while Table 13 covers multimodal LoRA SFT.
  • C.3 Baseline Subsampling and Fairness Controls: 2,300 instances are subsampled from each baseline dataset to match K12-Train, using stratified sampling when metadata is available.The procedure preserves source tags, task types, and length-category distributions.
  • C.3 Baseline Subsampling and Fairness Controls: Pilot experiments find similar performance trends across different random samples from the same dataset, indicating conclusions are not sensitive to a particular subset.
  • D Validation and Quality Assurance: Validation prioritizes K12-KGraph with automatic structural checks and full human verification, while K12-Bench and K12-Train receive targeted manual spot-checks.The knowledge graph underlies both downstream resources, motivating its more intensive validation.
  • D.2 KG Validation: Structural Checks and Human Verification: The is_a and prerequisites_- subgraphs are checked as directed acyclic graphs, with detected cycles and inconsistencies reviewed for curriculum-correct edge decisions.Subject annotators may remove, modify, or retain edges, resolving ambiguities through discussion.
  • D.2 KG Validation: Structural Checks and Human Verification: Each node and edge undergoes independent annotation by three same-subject annotators after structural conflicts are resolved.
  • D.2 KG Validation: Structural Checks and Human Verification: Consensus decisions resolve disagreements by retaining, modifying, or removing graph elements, with the consensus label treated as final.Visual relations are filtered by confidence: refers_to and illustrates require at least 0.85, requires_figure and supports_edge at least 0.95, and bounding boxes at least 0.9 localization confidence.

D.3 Spot-Check Validation of K12-Bench and K12-Train · E Extended Results and Sanity Checks

The paper validates K12-Bench and K12-Train through stratified manual sampling, finding high correctness among reviewed K12-Bench instances. It also reports inter-annotator agreement and per-subtask benchmark results as extended validation and sanity checks.

  • D.3 Spot-Check Validation of K12-Bench and K12-Train: Stratified manual sampling is used to validate K12-Bench and K12-Train.The sampling procedure supports spot-check validation of both resources.
  • D.3 Spot-Check Validation of K12-Bench and K12-Train: 15% of K12-Bench instances are manually reviewed across task families, subjects, and grade levels.Review checks consistency among question text, gold answers, distractors, and the underlying graph structure.
  • D.3 Spot-Check Validation of K12-Bench and K12-Train: 98.4% of sampled K12-Bench instances are fully correct.The remaining cases primarily concern phrasing ambiguity or borderline distractor quality rather than semantic errors.
  • D.3 Spot-Check Validation of K12-Bench and K12-Train: The K12-Bench review evaluates agreement between each question, its gold answer set, and its distractor options.These checks are performed with respect to the underlying graph structure.
  • D.3 Spot-Check Validation of K12-Bench and K12-Train: Inter-annotator agreement is reported before adjudication using Fleiss’ κ across subjects and node/edge types.The agreement results are presented in Table 14.
  • D.3 Spot-Check Validation of K12-Bench and K12-Train: Per-subtask K12-Bench results are reported as percentages using exact match and option-label-level F1.Table 15 defines EM as exact match and F1 at the option-label level.

E.1 Full Benchmark Results · E.2 Stability Across Random Seeds · E.3 Overlap and Leakage Analysis

The appendix expands K12-Bench evaluation into fine-grained subtask and subject analyses, tests SFT robustness across three random seeds on stratified subsets, and examines potential leakage into external benchmarks. The leakage analysis combines source and content distinctions with lexical and manual checks, finding no measurable contamination.

  • E.1 Full Benchmark Results: Table 15 and Table 16 report per-subtask and per-subject K12-Bench results, exposing variation behind aggregate task-family averages.The per-subject table reports EM and instance-level macro F1 in percent across curriculum domains.
  • E.2 Stability Across Random Seeds: Three seeds—{42, 123, 2026}—are used to repeat SFT on fixed 20% stratified subsets of GaokaoBench and EduEval.The subsampling preserves each benchmark’s original subtask distribution; full multi-seed evaluation was not conducted because of computational cost.
  • E.1 Full Benchmark Results: The per-subject breakdown complements the fine-grained subtask results by showing performance variation across curriculum domains.EM denotes exact match, while F1 denotes instance-level example-based macro F1.
  • E.2 Stability Across Random Seeds: Table 17 summarizes performance variation across the three random seeds using means and standard deviations from the 20% stratified subset.The reported statistics are computed over three runs.
  • E.3 Overlap and Leakage Analysis: K12-Train, GaokaoBench, and EduEval are institutionally and functionally distinct, supporting an analysis of potential contamination between training and evaluation data.K12-Train uses K–12 textbook content and curriculum relations, whereas GaokaoBench contains standardized high-stakes examination questions and EduEval has diverse evaluation objectives.
  • E.3 Overlap and Leakage Analysis: K12-Train emphasizes textbook-grounded concepts and relations, while GaokaoBench and EduEval primarily evaluate problem-solving and applied reasoning.The differing content forms and objectives further reduce the likelihood of overlap.
  • E.3 Overlap and Leakage Analysis: Negligible n-gram overlap and manual inspection finding no duplicated or near-duplicated question-answer pairs indicate no measurable leakage into the evaluation benchmarks.These empirical checks jointly support the conclusion that K12-Train does not contaminate GaokaoBench or EduEval.
Loading 2605.09635v3…