PaperlayerView exact revision on arXiv

Source-linked AI summary

Label-Centric Knowledge Retrieval for Zero-Shot Text Classification

Daoping Wu, Dong Wang, Siyu Liu, Chen Liang, Jian Geng, Ziqi Li, Shuguang Xiong, Ying Zhang

arXiv:2610.16182v1cs.AI

TL;DR

Repeated flagship-LLM inference is costly for large-scale labeling, and comprehensive labeled coverage is difficult in broad, diverse data. SLKR constructs reusable label knowledge offline and uses retrieval with a lightweight LLM for zero-shot classification. Across benchmarks and a real-world multilingual dataset, it improves lightweight zero-shot classification and compares favorably with few-shot and retrieved-ICL baselines across two model families.

  • Problem

    Flagship LLM inference costs and latency limit large-scale labeling, while broad and diverse data make comprehensive labeled coverage difficult.

    Source: §1 · pp. 1–2

  • Method

    SLKR builds label definitions, retrieval keywords, and boundary rules offline, then retrieves candidate-label knowledge and input-relevant rules for lightweight zero-shot classification.

    Source: §1 · pp. 2–3, §3.4 · p. 5

  • Results

    SLKR consistently improves lightweight zero-shot classification and compares favorably with few-shot and retrieved-ICL baselines across two model families.

    Source: §6 · p. 9

  • Takeaways & Limitations

    Selecting a small number of relevant boundary rules can retain the benefit of larger boundary contexts while avoiding irrelevant distinctions.

    Source: §6 · p. 10

    SLKR targets prediction from a closed-set inventory of semantically meaningful labels; task-specific labeled instances are isolated from its knowledge construction and classifier adaptation.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

Flagship large language models (LLMs) offer rich general knowledge and strong reasoning capabilities, but their high inference cost and latency limit their use for large-scale data labeling. We study zero-shot text classification with semantically meaningful labels and task descriptions, without task-specific labeled examples or classifier feedback. We propose \textbf{SLKR} (Semantic Label Knowledge Retrieval), a label-centric retrieval-augmented generation framework that separates offline knowledge construction from lightweight online inference. Offline, a flagship LLM independently generates label definitions, retrieval keywords, and boundary rules for potentially confusable label pairs, forming a reusable label-knowledge store. Because this construction operates over the label space rather than the input corpus, its one-time flagship cost can be amortized across large-scale inference. Online, hybrid semantic and lexical retrieval first identifies the top-$K$ candidate labels and then selects up to $M$ input-relevant boundary rules connecting candidate-label pairs. A lightweight LLM performs the final classification in a single call using the retrieved definitions and rules. Across three public benchmarks and a real-world multilingual book-classification dataset, SLKR achieves the best results on all four datasets with Qwen3.5-4B and on three of four with Ministral-3-3B, while using compact input contexts and no task-specific labeled examples. Boundary ablations further show that retrieving a small set of input-relevant rules preserves performance close to using all boundaries with substantially less context, supporting efficient reuse of flagship-generated label knowledge for lightweight classification.

1 INTRODUCTION

SLKR uses meaningful label semantics to organize reusable label knowledge, addressing the cost of repeated flagship-LLM classification and the difficulty of comprehensive labeled coverage. It retrieves candidate-label knowledge and input-relevant boundaries for lightweight zero-shot classification.

1 INTRODUCTION

  • Repeated flagship inference is costly, while broad and diverse data make labeled coverage difficult and conventional annotation, verification, and fine-tuning add iteration overhead.Meaningful labels also offer a natural interface to pretrained LLM knowledge, motivating reusable label semantics.Source: §1 · pp. 1–2
  • A flagship LLM independently generates label definitions, retrieval keywords, and boundary rules offline; online, hybrid retrieval selects labels and rules for a lightweight LLM.The flagship augmentations are based on the label space, while the final classifier reads the input and retrieved knowledge.Source: §1 · p. 2, Figure 1 · p. 2
  • SLKR retrieves reusable knowledge about candidate labels rather than source documents or labeled demonstrations, enabling classification without task-specific labeled data.This label-centric retrieval shifts from instance-level evidence to class-level decision knowledge.Source: §1 · pp. 2–3
  • Boundary rules distinguish potentially confusing labels across semantic, domain, functional, and application-level overlap, with only input-relevant rules retrieved at inference.The rules provide targeted guidance for fine-grained classification.Source: §1 · p. 3
  • Externalizing label understanding to offline construction reduces recurring inference costs and context requirements while maintaining competitive classification quality.The reusable knowledge is applied across inputs rather than reconstructed for every instance.Source: §1 · p. 3

2 RELATED WORK

Related work draws on label semantics, retrieval-augmented classification, and pairwise reasoning about ambiguous labels. SLKR differs by organizing reusable label-side knowledge and constructing boundaries from labels and task descriptions without task-instance supervision or classifier feedback.

Label semantics for classification

  • Label-description methods enrich class semantics for zero-shot classification, while LOTClass and X-Class combine label names with unlabeled task corpora.SLKR organizes definitions, retrieval keywords, and pairwise boundaries into a reusable label-side knowledge base.Source: §2 · p. 3

Retrieval-augmented classification

  • Conventional RAG retrieves external documents, whereas retrieved in-context learning selects labeled demonstrations relevant to an input.The cited dynamic few-shot retrieval methods assume task-instance supervision absent from SLKR.Source: §2 · p. 3

Boundary-aware classification

  • Prior boundary-aware methods use task instances or classifier feedback, while SLKR builds boundary knowledge from the label inventory and task description alone.It retrieves input-relevant rules among candidate labels.Source: §2 · p. 3

3 LABEL-CENTRIC RETRIEVAL-AUGMENTED CLASSIFICATION

SLKR builds reusable label-side knowledge offline, then uses staged retrieval and a lightweight LLM to classify inputs; its flagship construction cost depends on the label space, not the number of inputs.

3.1 SETTING AND RETRIEVAL OBJECT

  • For a closed-set inventory of meaningful labels, SLKR predicts a primary label using reusable label-side knowledge rather than instances from the classified corpus.Each label record includes an augmented definition, retrieval keywords, and associated boundary rules.Source: §3.1 · p. 4

3.2 THREE INDEPENDENT FLAGSHIP AUGMENTATIONS

  • Offline, the flagship model independently generates label definitions, retrieval keywords, and pairwise boundary rules from labels and task description, then freezes them for reuse.The branches do not use one another’s outputs or task-specific labeled examples.Source: §3.2 · pp. 4–5

3.3 HYBRID LABEL RETRIEVAL

  • Hybrid semantic and lexical retrieval combines rankings of label definitions and label-name or keyword matches with RRF to pass Top-K candidate labels onward.Boundary rules are excluded from label embeddings because they express contrasts among classes rather than positive representations of individual labels.Source: §3.3 · p. 5

3.4 CANDIDATE-CONDITIONED BOUNDARY RETRIEVAL

  • Boundary retrieval restricts rules to candidate-label pairs, then fuses semantic and lexical relevance to select Top-M rules, reducing irrelevant distinctions and keeping context compact.Rules involving labels outside the candidate set are excluded.Source: §3.4 · p. 5

3.5 LIGHTWEIGHT CLASSIFICATION

  • The lightweight LLM receives the input, candidate-label names and definitions, and selected boundary rules, then predicts one label from the candidate set.Retrieval scores and keyword lists are omitted from its final prompt.Source: §3.5 · p. 6

3.6 AMORTIZED COST ANALYSIS

  • Flagship construction costs O(N^2) in the worst case and is independent of T, while direct flagship classification requires a flagship call for every input.As T increases for a fixed label inventory, the per-input flagship augmentation contribution vanishes, leaving retrieval and lightweight inference as recurring costs.Source: §3.6 · p. 7

4 EXPERIMENTAL SETUP

The experiments assess classification quality, context efficiency, recurring costs, and boundary retrieval across public benchmarks and a real-world book-classification setting. They compare SLKR with zero-shot, few-shot, and retrieval-based methods, while isolating the contribution of input-conditioned boundary selection.

4.1 DATASETS

  • Book-38 contains 719 multilingual books across 38 broad disciplines, complementing three public benchmarks with longer, more heterogeneous documents.The public datasets are WOS-46985, BANKING77, and CLINC150; Book-38 includes 540 training and 179 test instances.Source: §4.1 · p. 7

4.2 BASELINES AND IMPLEMENTATION

  • SLKR is compared with label-only zero-shot prompting, label-description augmentation, few-shot demonstrations, and retrieved in-context examples.The full method uses K = 15 candidate labels and up to M = 5 boundary rules; offline knowledge is built from labels and task descriptions without dataset instances or classifier feedback.Source: §4.2 · pp. 7–8

4 EXPERIMENTAL SETUP

  • Recall@15 exceeds 95% on all four datasets, motivating a fixed candidate budget of K = 15 for subsequent experiments.The setup evaluates classification quality, context efficiency, amortized inference cost, and the contribution of boundary rules.Source: §4.2 · p. 8

4.3 BOUNDARY RETRIEVAL ABLATION

  • Four settings isolate boundary effects: no rules, all eligible rules, five random rules, and the five most input-relevant rules.The candidate budget is fixed at K = 15, and Random M = 5 controls for rule count when comparing with Hybrid Top-M = 5.Source: §4.3 · p. 8

4.4 METRICS

  • Evaluation reports accuracy and macro-F1, Recall@K, reader input tokens, selected-boundary tokens, and separate one-time construction and recurring inference costs.Recall@K measures whether the gold label remains among the Top-K candidates.Source: §4.4 · p. 8

5 RESULTS

SLKR performs strongly across the four datasets with lightweight readers, while selective boundary retrieval retains performance with less context than including every rule. A BANKING77 comparison also shows a quality–cost trade-off between direct flagship inference and reusable SLKR knowledge.

Main results

  • SLKR achieves the best results on all four datasets with Qwen3.5-4B and on three of four with Ministral-3-3B; retrieved ICL is slightly better on CLINC150 with Ministral.SLKR uses 1.4k input tokens on average, compared with 4.7k for 1-shot-per-label prompting, and compares favorably with few-shot methods without task-specific labeled examples.Source: §5 · p. 8, Table 4 · p. 9

Boundary retrieval

  • Hybrid Top-M = 5 uses 0.35k boundary tokens versus 2.51k for All Boundaries, with similar performance on public benchmarks; on Book-38, selective retrieval performs substantially better.Random M = 5 consistently underperforms Hybrid Top-M = 5, indicating that the benefit is not simply from adding more rule text.Source: §5 · pp. 8–9, Table 5 · p. 9

Flagship and cost comparison

  • Direct GPT-5.6 Sol scores 86.01/85.30 accuracy/macro-F1 versus SLKR's 77.45/76.63 on BANKING77, while recurring online cost is $4.111 versus $0.0596 per 1k inputs.Direct flagship inference has higher classification quality, but incurs inference cost for every input; SLKR pays the flagship cost once to construct reusable label knowledge.Source: §5 · p. 9, Table 6 · p. 9

6 CONCLUSION

SLKR externalizes label understanding into reusable knowledge built by a flagship LLM, then retrieves candidate-label knowledge and relevant boundary rules for lightweight zero-shot classification. Results and boundary ablations support compact-context reuse across public and real-world classification settings.

6 CONCLUSION

  • Across three public benchmarks and a real-world multilingual book-classification dataset, SLKR improves lightweight zero-shot classification and compares favorably with few-shot and retrieved-ICL baselines across two model families.The framework retrieves candidate-label knowledge and input-relevant boundary rules rather than documents or labeled demonstrations.Source: §6 · p. 9
  • Boundary ablations show that selecting a small number of relevant rules can retain the benefit of larger boundary contexts while avoiding irrelevant distinctions.The conclusion frames reusable offline label knowledge as a practical path toward scalable, cost-efficient classification in large-scale cold-start settings.Source: §6 · p. 10

AI USE STATEMENT

The authors used generative AI to polish the paper and assist with code development and debugging, then reviewed and verified AI-assisted content and code.

AI USE STATEMENT

  • Generative AI assisted with writing and code development/debugging; authors reviewed and verified the outputs and take responsibility for the final work.Source: AI USE STATEMENT · p. 10

A ADDITIONAL METHOD DETAILS

SLKR builds reusable label knowledge offline, retrieves candidate labels and input-relevant boundary rules online, and uses a lightweight LLM to classify in one call. The candidate budget yields Recall@15 above 95% on all four datasets.

A.1 INDEPENDENT LABEL-KNOWLEDGE CONSTRUCTION

  • Independent offline branches generate label definitions, retrieval keywords, and pairwise boundary rules, then combine them in a frozen label-side knowledge base.Construction uses the label inventory and task description, not task-specific labeled examples, and is performed once over the label space.Source: §A.1 · p. 13

A.2 HYBRID CANDIDATE-LABEL RETRIEVAL

  • Candidate labels are ranked by combining semantic and lexical retrieval with reciprocal-rank fusion (RRF).Semantic ranking uses cosine similarity; lexical retrieval uses label names and generated keywords, while boundary rules and keywords are excluded from the final classification context.Source: §A.2 · p. 13
  • Recall@15 exceeds 95% on all four datasets with the main candidate budget K = 15.The main configuration fixes K = 15 across datasets.Source: §A.2 · p. 13, §A.3 · p. 14

A.3 CANDIDATE-CONDITIONED BOUNDARY RETRIEVAL

  • Boundary retrieval considers only rules whose two labels are in the Top-K set, then uses hybrid ranking to retain up to M rules.The main configuration uses up to five rules; this retrieval changes decision guidance, not the candidate-label set.Source: §A.3 · p. 14

A.4 LIGHTWEIGHT CLASSIFICATION

  • The lightweight LLM receives the input, retrieved candidate names and definitions, and selected boundary rules, then predicts one candidate label in a single call.Retrieval scores and keyword lists are not included in its classification context.Source: §A.4 · p. 14

B.1 PUBLIC BENCHMARKS

The evaluation covers three public classification benchmarks and Book-38, a real-world multilingual book dataset. SLKR knowledge construction is instance-free, and K = 15 yields Recall@15 above 95% on each dataset.

B.1 PUBLIC BENCHMARKS

  • The public benchmarks comprise fine-grained research-area classification on WOS-name-143, 77 BANKING77 intents, and in-scope examples from CLINC150’s 150 intent classes.WOS-name-143 uses Level-2 labels from the hierarchical WOS-46985 taxonomy.Source: §B.1 · p. 14

B.2 BOOK-38

  • Book-38 contains 719 multilingual books across 38 broad disciplines, with 540 training and 179 test instances.Its longer, more heterogeneous documents provide a complementary practical datacuration evaluation.Source: §B.2 · p. 14

C.1 MODELS

  • SLKR knowledge construction uses only labels and the task description, while few-shot and retrieved-ICL baselines use labeled training examples.The experiments use Qwen3.5-4B and Ministral-3-3B as lightweight readers, with GPT-5.6 Sol constructing frozen offline knowledge; reported main-paper performance is averaged over three seeds.Source: §C.1 · p. 15

C.2 CANDIDATE BUDGET

  • Recall@15 exceeds 95% on every dataset, supporting the use of K = 15 for subsequent classification and boundary retrieval.The experiments fix K = 15 across all four datasets.Source: §C.2 · p. 15

C.3 BOUNDARY-RETRIEVAL ABLATION

The boundary ablation compares removing rules, including all eligible rules, random selection, and input-conditioned selection of five rules. Hybrid Top-M = 5 uses substantially fewer boundary tokens than including all eligible rules.

C.3 BOUNDARY-RETRIEVAL ABLATION

  • The four settings separate boundary knowledge from input-conditioned selection: no rules, all eligible rules, five random rules, and five hybrid-retrieved rules.The comparison fixes K = 15; eligible rules connect labels in the retrieved candidate set.Source: §C.3 · p. 15
  • 0.35k boundary tokens per input for Hybrid Top-M = 5, versus 2.51k for All Boundaries.Random M = 5 serves as a rule-count-matched control.Source: §C.3 · p. 15

C.4 EVALUATION METRICS

The evaluation measures classification quality, retrieval coverage, and context use, while cost accounting separates one-time construction from recurring inference. For a fixed label inventory, construction overhead per input falls as more inputs are classified, unlike direct flagship inference costs.

C.4 EVALUATION METRICS

  • Classification uses accuracy and macro-F1; label retrieval uses Recall@K, and context efficiency counts input tokens supplied to the lightweight reader.Boundary experiments also report tokens contributed by selected rules.Source: §C.4 · p. 15

D COST ACCOUNTING

  • SLKR cost accounting separates one-time flagship knowledge construction from average recurring online inference cost per classified input.The total cost for T inputs combines the offline construction cost with recurring online costs.Source: D COST ACCOUNTING · p. 15
  • For a fixed label inventory, the one-time construction cost contributes less per input as the number of classified inputs increases.Direct flagship classification instead requires a flagship inference call for every input, so its inference cost grows linearly with T.Source: D COST ACCOUNTING · pp. 15–16

Related papers

Loading 2610.16182v1…