PaperlayerView exact revision on arXiv

Source-linked AI summary

Distilling Directional Verification

Jungseob Lee, Sugyeong Eo, Seongtae Hong, Seungyoon Lee, Chanjun Park, Jaehyung Seo, Heuiseok Lim

arXiv:2610.00997v1cs.CLcs.AIcs.LG

TL;DR

Teachers can know a relation in one direction yet give unreliable answers in reverse, making generated answers risky distillation targets. Directional label distillation instead scores candidates in the known direction and trains students on the selected targets. On parent–child facts, this improves trained-query open-ended accuracy over prior-corrected reverse labels.

  • Problem

    A pretrained teacher may fail to answer a reverse query despite knowing the relation forward, leaving distillation without reliable reverse targets and risking transfer of generated errors.

    Source: §1 · pp. 1–2

  • Method

    Frozen teachers score candidate answers in the relation direction they know, and the highest-scoring candidate becomes the student’s reverse-query training target.

    Source: §1 · p. 2

  • Results

    13 to 15 points higher open-ended accuracy on trained queries comes from known-direction labels versus prior-corrected reverse labels; label advantages also hold on unscreened queries and without inserted correct answers.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Directional verification provides a practical source of reverse supervision when teacher generation fails, with the scoring direction on these facts tied to the notable entity rather than the prompt template.

    Source: §7 · p. 9

    The study covers one relation with a fixed name inventory; open-ended answers need other candidate-proposal methods, and scoring costs one teacher call per candidate.

    Source: §7 · pp. 9–10

Abstract

from arXiv · show

Knowledge distillation aims to transfer the factual knowledge of large language models to smaller models for efficient deployment. Yet a teacher may recall a relation in one direction while failing to generate the answer in the reverse direction. Distillation from its generated answers can therefore propagate this directional limitation to the student. The same teacher can nevertheless recognize such an answer by scoring the relation in the direction it knows. We introduce directional label distillation, in which frozen teachers score candidate answers in that known direction and the best-scoring candidate becomes the student's training target. On facts about parents and their children, known-direction scoring yields more accurate labels than scoring the requested direction, even after tuned corrections for name priors. With prior-corrected scores, the better direction depends on the facts rather than the template, and reverses on mined facts whose notable entity is the parent rather than the child. With the evaluated children's forward facts withheld, students trained on known-direction labels improve open-ended accuracy on their trained queries by 13 to 15 points over students trained on prior-corrected reverse labels. After generated answers are matched to a fixed name list by lexical similarity, students reproduce nearly all selected labels. Their accuracy largely follows label quality. The label advantage holds on unscreened queries and when candidates are retrieved without inserting correct answers. Our findings show that directional verification mitigates the transfer of errors from teacher-generated answers to students by providing more accurate training targets. Code is available at https://github.com/js-lee-AI/directional-verification.

1 INTRODUCTION

Directional label distillation uses teacher verification in a relation’s known direction to select reverse-query training targets. On parent–child facts, those labels improve student accuracy over prior-corrected reverse labels and outperform several alternative labeling approaches.

1 INTRODUCTION

  • Prior approaches alter training data or objectives, but do not address obtaining accurate reverse labels from a pretrained teacher whose reverse answers are unreliable.Generated sequences can pass incorrect reverse answers to students, while filtering cannot recover correct answers absent from candidates.Source: §1 · pp. 1–2
  • Directional label distillation scores candidate answers in the teacher’s known relation direction and uses the highest-scoring candidate as the student’s reverse-query target.The student learns without teachers at inference.Source: §1 · p. 2
  • Known-direction labels outperform reverse-score labels under every tested aggregation rule and beat tuned prior correction, teacher generation, and name heuristics.The advantage also holds on unscreened queries and when candidate retrieval does not insert correct answers.Source: §1 · p. 3
  • With prior-corrected scores, the better sentence direction switches between facts about notable children and notable parents, rather than following the query template.Known-direction scoring also avoids directly rewarding candidates for the probability of their own names.Source: §1 · p. 3
  • 13 to 15 points higher open-ended accuracy on trained queries comes from known-direction labels versus prior-corrected reverse labels.The gains persist for whole-answer accuracy, and students reproduce nearly all selected labels after lexical matching.Source: §1 · p. 3

2 DIRECTIONAL LABEL DISTILLATION

Frozen teachers score candidate children by evaluating the same parent continuation in the child-to-parent direction, then select a pseudo-label for reverse-query student training. The section also specifies candidate construction, reverse-score comparators, and student training and inference.

2 DIRECTIONAL LABEL DISTILLATION

  • For each parent query, teachers average parent-token log probabilities after each candidate child’s parent prompt, and the highest-scoring candidate becomes the pseudo-label.The parent continuation stays fixed across candidates while the child context changes.Source: §2 · p. 3

2.1 CANDIDATE ACQUISITION AND LABEL CONSTRUCTION

  • For screened queries, three teachers contribute their eight highest-scoring candidates from the inventory; a fourth teacher scores the union, and the four main-template scores are averaged.Matched comparisons hold candidate pools and selection rules fixed, including queries whose pool contains no correct answer.Source: §2.1 · p. 3

2.2 REVERSE COMPARATORS

  • Reverse comparators score the requested continuation and may subtract context-only name probabilities; tuned variants select correction parameters on other query sets.Teacher generation is also compared, with completions mapped onto the candidate list.Source: §2.2 · p. 4

2.3 STUDENT TRAINING AND INFERENCE

  • Masked diffusion students learn answer slots with bidirectional attention and cross-entropy, while true forward examples remain mixed into reverse supervised fine-tuning.At inference, teachers are discarded; the student fills ten slots in confidence order, and an autoregressive student trained on the same labels provides a second prediction objective.Source: §2.3 · p. 4

3 EXPERIMENTAL SETUP

The experiments use screened and unscreened parent–child query cohorts, with training and evaluation designed to examine label quality under controlled forward exposure. They separately measure label accuracy and student internalization.

3 EXPERIMENTAL SETUP

  • Label accuracy and student internalization are measured separately; screened-cohort scoring choices use gold evaluations, while unscreened choices are fixed before teacher inference.Student comparisons use 0.6B masked diffusion models, three seeds, and matched GPU class within paired runs.Source: §3 · pp. 4–5
  • The corpus contains 10,505 Wikidata parent–child pairs; the screened cohort retains 2,476 of 4,400 children, and 1,500 pairs supply reverse-label training queries.Its acquired pool has 28,463 candidate slots and an accepted answer for 97.13% of queries.Source: §3 · p. 4
  • The unscreened cohort contains 512 new parent queries, each paired with two direction-independent 64-name lists using uniform or lexically similar distractors.Both lists include the recorded answer, and separate students are trained on this cohort.Source: §3 · pp. 4–5
  • Primary screened comparisons remove forward edges for trained children and evaluate 1,390 queries whose parent appears in no retained forward example.Unscreened students withhold each query parent’s children’s forward edges, making all 512 queries per pool exposure-free.Source: §3 · p. 5
  • Open accuracy requires a normalized full-name substring match, whole-answer accuracy requires exact normalized output, and inventory-matched accuracy requires exact membership after lexical selection.The evaluation includes no bare-surname credit.Source: §3 · p. 5

4 RESULTS

Known-direction scoring produces better labels than reverse scoring, and students trained on those labels achieve higher open-ended accuracy. The best scoring direction also varies with the facts, while the advantage persists against stronger comparators and unscreened candidate pools.

4.1 STUDENTS TRAINED ON DIRECTIONAL LABELS

  • After inventory matching, students reproduce their selected labels for more than 97% of queries on average, and student accuracy largely follows label accuracy.This indicates that better labels account for most of the student gain.Source: §4.1 · p. 5
  • 14.77 points on screened queries and 13.09 to 15.36 points on unscreened queries: known-direction labels improve open accuracy over summed DC or transferred tuned-reverse labels.The training recipe is held fixed; gains hold across paired seeds and under whole-answer evaluation.Source: §4.1 · p. 5, Table 1 · p. 6

4.2 SCORING DIRECTION AGAINST AGGREGATION RULES

  • 12.47 to 16.73 points: known-direction labels exceed summed DC labels under all six aggregation rules, while rules within a score form differ by at most 2.33 points.Eight-channel agreement performs on par with simple score means on the same vectors.Source: §4.2 · p. 5, Figure 2 · p. 6
  • Reverse-score errors often favor names with higher priors; subtracting the prior over-corrects for high-prior children, while known-direction accuracy stays similar across prior strata.Over 90% of summed-reverse errors select a name with a higher prior than the true child.Source: §4.2 · p. 6

4.3 STRONGER REVERSE COMPARATORS

  • 10.07 to 14.06 points: known-direction labels lead transferred tuned-reverse labels across three cohorts, and they also outperform teacher generation and teacherless name heuristics.Tuned reverse scores improve on summed DC, but leave a similar gap when tuned on each cohort’s own gold labels.Source: §4.3 · p. 7
  • Known-direction scoring leads tested reverse scorers on uniform and lexical unscreened lists, including pools built without teacher scores that contain the recorded answer.Checks across four teachers and two preregistered template pairs also favor known-direction scoring over summed reverse and summed DC scores.Source: §4.3 · p. 7, Figure 4 · p. 7
  • 25.59 points: known-direction scoring retains this advantage over summed DC on the same union pool in 512 unscreened queries without inserted answers.The results show gains in candidate retrieval and label selection, while candidate coverage still limits accuracy.Source: Table 2 · p. 7, §4.3 · p. 7

4.4 SENTENCE DIRECTION AND NOTABILITY

  • On corpus facts, child-to-parent sentences win; on mined facts with a notable parent, parent-to-child sentences win, with the mean difference shifting 18.68 points toward that direction.The direction switches between fact sets sharing template pair, list construction, and scoring forms; differing eras and name distributions mean the comparison does not isolate a single cause.Source: §4.4 · pp. 7–8, Figure 4 · p. 7

5 SUPPORTING CONTROLS

Controls indicate that student accuracy tracks label quality, while name-cue and decoding checks show the observed gains are not explained by surname matching alone.

5.1 LABEL QUALITY AND SURFACE REALIZATION

  • Without reverse-label supervision, format adaptation, MLM-U, and identity-bridge data each leave open accuracy below 3%; reverse-label supervision drives improvement in these controls.MLM-U directly tests an any-order remedy for reversal failures, yet leaves these reverse queries largely unanswered.Source: §5.1 · p. 8

5.2 NAME CUES AND DECODING

  • On surname-mismatched queries, known-direction-trained students outperform summed-DC students by 22 to 29 points on average, with a gain in every seed.Adding 9,498 parent-only names halves surname-only lookup accuracy and cuts warm-start accuracy by roughly three quarters, but changes trained-student accuracy by less than one point.Source: §5.2 · p. 8

6 RELATED WORK

Prior work studies directional failures, alternative adaptation objectives, and verification-based supervision; this paper instead uses frozen directional scores to build reverse targets and tests them without the evaluated children’s forward edges. Its controls show that any-order training alone leaves reverse accuracy near zero on these facts.

Directional access and adaptation

  • Prior reversal studies examine failures to infer reverse relations after forward learning, while factorization and next-token prediction offer broader accounts [Berglund et al., 2024; Wang & Sun, 2026; Kitouni et al., 2024; Bachmann & Nagarajan, 2024].Source: §6 · p. 8
  • Any-order training alone leaves reverse accuracy near zero on these facts, whereas the same student answers once reverse labels exist.The evaluation withholds the evaluated children’s forward edges.Source: §6 · p. 9
  • Unlike reverse-training, identity-bridge, or alternative prediction-order approaches, this work uses frozen directional scores to construct reverse targets [Ma et al., 2026; Nie et al., 2025].Source: §6 · p. 9

Verification and weak supervision

  • The method scores inventory candidates in the opposite direction from the requested answer, relating it to noisy-channel scoring and prior-corrected direct-prediction comparators.Source: §6 · p. 9
  • Unlike verifier-ensemble work, this study trains an answer generator and separates candidate coverage, label correctness, and student reproduction to locate the source of gains [Lee et al., 2026a].Source: §6 · p. 9

7 CONCLUSION AND LIMITATIONS

Directional label distillation turns a teacher’s ability to verify a relation in one direction into targets for the other. The paper reports stronger known-direction labels and student reproduction, while noting constraints on relation scope, candidate generation, and choosing the scoring direction.

7 CONCLUSION AND LIMITATIONS

  • Known-direction labels outperform reverse labels under every tested aggregation rule and stronger comparator, including unscreened queries and candidate retrieval without inserted answers.Students reproduce the selected labels almost exactly.Source: §7 · p. 9
  • On these facts, the better scoring direction tracks the notable entity rather than the prompt template, making the more frequently described entity a natural direction-selection criterion.Source: §7 · p. 9
  • The study covers one relation and a fixed name inventory; open-ended answers require other ways to propose candidates.Source: §7 · p. 9
  • Scoring costs one teacher call per candidate, larger inventories need a cheaper proposal stage, and the corpus fixes scoring direction by construction.A label-free direction-selection rule remains a next step when the notable entity varies.Source: §7 · p. 10

ETHICS STATEMENT

The study uses public Wikidata parent–child relations and publicly released language models, with no human subjects, annotators, or personal data beyond those public records.

ETHICS STATEMENT

  • Experiments use public Wikidata parent–child relations and publicly released language models, without human subjects, annotators, or additional personal data.Source: ETHICS STATEMENT · p. 10
  • Because directional verification could surface memorized relations about real people, experiments are restricted to documented Wikidata facts and collect no new information about individuals.Source: ETHICS STATEMENT · p. 10
  • The notable-parent cohort includes people without English Wikipedia articles, and the study uses only their relations already recorded in Wikidata and reports aggregate accuracies.Source: ETHICS STATEMENT · p. 10

A IMPLEMENTATION AND SUPERVISION

The appendix specifies the Wikidata corpus, teacher scoring and label-selection procedures, masked-diffusion student training, and evaluation processing. It also documents candidate screening and the distinct budgets of the autoregressive control.

A IMPLEMENTATION AND SUPERVISION

  • The corpus contains 10,505 deduplicated parent–child pairs mined from Wikidata, with humans selected by occupation and English Wikipedia presence and names cleaned to two-to-four ASCII tokens.Source: A IMPLEMENTATION AND SUPERVISION · p. 14
  • Known-direction prompts score the parent as a continuation of child-to-parent templates, while reverse prompts score candidate children; mean continuation-token log probabilities supply scores.Teacher candidates come from three model pools, with a fourth model rescoring them.Source: A IMPLEMENTATION AND SUPERVISION · p. 14
  • Soft agreement combines teacher–template channel probabilities into candidate distributions using fitted weights and selects the highest-posterior eligible candidate.The weight update is a heuristic inspired by latent-annotator aggregation, not a Dawid–Skene likelihood estimate.Source: A IMPLEMENTATION AND SUPERVISION · p. 14
  • Alternative label rules use raw-score or z-score means, probability means, Borda, and reciprocal-rank fusion, with self candidates excluded from final selections.Source: A IMPLEMENTATION AND SUPERVISION · p. 14
  • Masked-diffusion students train on ten answer slots with a uniformly sampled subset masked, using a masked-answer loss over target tokens.Qwen3-4B or Qwen3-0.6B supplies initial weights; training uses LoRA.Source: A IMPLEMENTATION AND SUPERVISION · p. 14
  • Mixed training repeats reverse examples relative to forward examples, and greedy confidence-first inference fills the ten answer slots in ten passes.Source: A IMPLEMENTATION AND SUPERVISION · p. 15
  • Open accuracy matches normalized full-name substrings against true children, while postprocessing maps outputs to inventory names by token overlap and character-level similarity.The matching inventory has 8,218 names, and fixed inventory order breaks ties.Source: A IMPLEMENTATION AND SUPERVISION · p. 15
  • Gold facts support screening, forward adaptation, configuration evaluation, and answer insertion in unscreened candidate lists; the autoregressive control receives about 1.8 times as many realized supervised tokens.The MDM-4B/AR-8B comparison also changes model size, objective, and budget.Source: A IMPLEMENTATION AND SUPERVISION · p. 15

B MATCHED LABEL COMPARISONS

Known-direction labels outperform reverse-score controls, and the advantage persists in student training with forward facts withheld. Aggregation differences are small, while acquired-pool accuracy is measured only where the recorded answer is available among candidates.

B MATCHED LABEL COMPARISONS

  • The known-direction label advantage over mean-token DC reverse scores is 32.47–37.78 points, while the reverse-versus-DC interval includes zero.On exposure-free queries, known-direction scores exceed summed DC by 12.95–16.83 points across six rules.Source: Table 4 · p. 15, B MATCHED LABEL COMPARISONS · p. 16
  • Agreement and the three known-direction score means differ by less than one point.The six prespecified comparisons use the same eight score vectors for the aggregation rules.Source: Table 6 · p. 16, B MATCHED LABEL COMPARISONS · p. 16
  • Known-direction accuracy is about 94% on 1,346 exposure-free acquired-pool queries with the recorded child among candidates; 44 other queries lack it.This analysis measures label selection when the recorded answer is available.Source: B MATCHED LABEL COMPARISONS · p. 16

C MATCHED FORWARD-WITHHELD STUDENTS

  • Known-direction labels yield higher open and whole-answer student accuracy than summed-token DC labels in all three seeds, with a 15.54-point label-accuracy gap.Known-direction labels are correct for 1,261/1,390 primary queries (90.72%), versus 1,045/1,390 (75.18%) for summed DC.Source: C MATCHED FORWARD-WITHHELD STUDENTS · p. 18, Table 10 · p. 18

D SELECTED-LABEL TRANSFER

  • In the reported student runs, label accuracy is an upper bound on whole-answer accuracy because no student turns a wrong label into a whole-answer match.This holds for known-direction, reverse, and summed-DC students across the evaluated seeds.Source: D SELECTED-LABEL TRANSFER · p. 19, Table 12 · p. 19

E COHORT WITH DIRECTION-INDEPENDENT CANDIDATES

On unscreened parent queries, known-direction scoring improves label and student accuracy over summed-DC controls across candidate pools. The advantage also holds with alternate templates and full-inventory candidate retrieval without inserted answers.

E COHORT WITH DIRECTION-INDEPENDENT CANDIDATES

  • Averaging both templates gives known-direction gains over summed DC of 13.28 points on uniform lists and 15.62 points on lexical lists.Known-direction labels are more accurate than summed-DC labels in both pools; within a teacher, mean- and summed-token known-direction rankings coincide.Source: E COHORT WITH DIRECTION-INDEPENDENT… · p. 20, Table 14 · p. 21
  • The known-direction advantage persists on all 228 surname-mismatched queries, reaching 73.25% in the uniform pool against at most 44.30% for summed DC.These comparisons use unchanged candidate lists.Source: E COHORT WITH DIRECTION-INDEPENDENT… · p. 20, Table 15 · p. 21
  • Without inserted answers, known-direction full-inventory scans place a true child among the top eight names for 63.67–75.39% of parents, versus 26.37–50.20% for summed DC.The known-direction pipeline exceeds summed DC by 27.15 points, and by 25.59 points on the union of both candidate pools.Source: E COHORT WITH DIRECTION-INDEPENDENT… · p. 21, Table 16 · p. 22
  • Known-direction students exceed summed-DC students in open, whole-answer, and inventory-matched accuracy in every seed, with paired gains of 12.30–16.41 points.The comparison covers both unscreened candidate pools.Source: E COHORT WITH DIRECTION-INDEPENDENT… · p. 21, Table 17 · p. 22

F STRONGER COMPARATORS AND REVERSED QUERIES

Known-direction labels remain more accurate than tuned reverse, generation, and lexical-rule comparators. Student results and reversed-query analyses show that label quality and the preferred scoring direction vary with the facts and query setting.

F STRONGER COMPARATORS AND REVERSED QUERIES

  • Known-direction student gains over summed DC persist when parent and recorded child have different surnames.The reported known-direction label advantage over transferred tuned-reverse labels is about 22–27 points across cohorts.Source: F STRONGER COMPARATORS AND REVERSED… · p. 23
  • Known-direction labels are more accurate in all three cohorts than reverse labels using each cohort’s best tuned correction or a transferred tuned-reverse comparator.The comparison includes four tuned reverse rules and a transferred comparator selected on other cohorts.Source: F STRONGER COMPARATORS AND REVERSED… · p. 23, Table 19 · p. 24
  • Mapped teacher generations reach at most 17.05% label accuracy, and four-teacher greedy-majority labels trail the best individual teacher in every cohort.The tested mapping and voting rules leave most reverse labels incorrect.Source: F STRONGER COMPARATORS AND REVERSED… · p. 23, Table 20 · p. 24
  • Known-direction labels exceed surname and character-trigram rules in all three pools, with acquired-pool gains of 17.99 and 25.25 points, respectively.The advantage also holds on surname-mismatched queries.Source: F STRONGER COMPARATORS AND REVERSED… · p. 24, Table 22 · p. 25
  • Inverted generation labels are mostly correct but cover fewer than 30% of queries in every cohort, leaving their accuracy below known-direction scoring.Replacing known-direction labels with inverted labels wherever votes are available changes accuracy by at most 0.20 points per cohort.Source: F STRONGER COMPARATORS AND REVERSED… · p. 24, Table 21 · p. 25
  • Known-direction students exceed transferred tuned-reverse students by 12.11–14.65 points on uniform lists and 13.09–15.04 points on lexical lists.After inventory matching, selected-label reproduction averages 99.5–99.9% for known-direction, tuned-reverse, and generation-majority labels.Source: Table 23 · p. 25, F STRONGER COMPARATORS AND REVERSED… · p. 25
  • On reversed child queries, child-to-parent sentence scoring also wins; best-λ gains are 11.52 points on uniform lists and 6.45 points on lexical lists.At λ = 1, the lexical gain is 5.86 points.Source: F STRONGER COMPARATORS AND REVERSED… · p. 25, Table 24 · p. 26
  • The preferred sentence direction tracks the notable entity: child-to-parent wins on corpus facts, while parent-to-child wins on notable-parent facts.The averaged differences are 11.82 points on corpus facts and −6.86 points on notable-parent facts.Source: F STRONGER COMPARATORS AND REVERSED… · p. 26, Table 24 · p. 26

G DEFAULT-EXPOSURE CONTROL RECIPES

Default-exposure controls compare label sources, teacher sizes, decoding, training budgets, fact counts, and selective supervision. Student accuracy generally follows label quality, while longer training improves accuracy and filtering supervision can reduce coverage.

G DEFAULT-EXPOSURE CONTROL RECIPES

  • Across student sizes and objectives, single-teacher, agreement, and gold labels keep the same ordering, while larger MDM mainly raises open accuracy at fixed label correctness.The comparison spans all 1,500 trained queries.Source: Table 25 · p. 26, G DEFAULT-EXPOSURE CONTROL RECIPES · p. 26
  • The control recipes include forward-only warm training, a one-stage masked objective without reverse labels, random or self-ranked labels, and alternate-template or identity-bridge controls.The identity-bridge control adapts Ma et al. (2026) to the total warm and SFT token budget.Source: G DEFAULT-EXPOSURE CONTROL RECIPES · p. 27
  • Label accuracy rises with Qwen3 teacher size, and matched student accuracy follows it closely; the student’s Qwen3-4B base supplies known-direction labels at 78.40% accuracy.The pipelines also differ in candidate acquisition and self-identity filtering.Source: Table 26 · p. 27, G DEFAULT-EXPOSURE CONTROL RECIPES · p. 27
  • Decoding order shifts open accuracy by up to 7.5 points but matched accuracy by at most 0.2 points.The decoding control retrains single-teacher-label checkpoints and evaluates each under three orders.Source: G DEFAULT-EXPOSURE CONTROL RECIPES · p. 27, Table 29 · p. 28
  • Longer training raises open and matched accuracy and brings matched accuracy closer to label accuracy; at fixed steps, accuracy declines as trained fact counts increase.With single-teacher labels, matched accuracy remains close to label accuracy across nested query sets.Source: G DEFAULT-EXPOSURE CONTROL RECIPES · p. 28, Table 30 · p. 29
  • Full-coverage agreement gives the highest matched accuracy over all trained queries because precision filtering leaves many queries without reverse supervision.The compared selective-supervision strategies are margin filtering, unanimity, and full-coverage agreement.Source: Table 31 · p. 29, G DEFAULT-EXPOSURE CONTROL RECIPES · p. 29
Loading 2610.00997v1…