Zero-Shot Visualization: Exploring Text Corpora with User-Prompted Axes

1 Introduction

• Conversational LLM tools return answers or summaries, while this work asks whether LLM knowledge can instead structure exploration through explicit semantic axes.The motivating contrast is with classical visualization methods that rely on explicit feature representations.Source: §1 · p. 1
• Zero-Shot Visualization (ZSV) maps texts to user-written natural-language concepts as axes, without task-specific training.A scoring function measures each text’s alignment with each concept, producing coordinates for projection, visualization, and clustering.Source: §1 · pp. 1–2
• Practical ZSV scorers must be semantically faithful, efficient enough for interactive use, and runnable on commodity hardware.Embedding methods are faster but have inferior, poorly scaled scores, while prompted verbalizers are inefficient on commodity hardware.Source: §1 · p. 2
• The paper contributes a formal framework, a visualization-first benchmark across datasets and use cases, and practitioner guidance for end-to-end baselines.The guidance covers method and LLM choice, scaling, inference cost, and preprocessing.Source: §1 · p. 2

2 Related Work

• Natural-language data-exploration systems translate requests into visualizations over existing tabular attributes rather than defining new semantic dimensions.These include chart generation, instruction grounding, agentic systems, and exploratory-analysis benchmarks.Source: §2 · p. 2
• Chart understanding, summarization, and visual question answering interpret fixed visualizations, whereas ZSV addresses how semantic axes are constructed.Source: §2 · p. 2
• Zero-shot label-generation methods do not generally provide projections specified by written concepts, while ZSV benchmarks user-input concepts for interactive corpus exploration.Classical projections are unsupervised; closer concept-scoring work either induces concepts from data or embeds scoring in a trained classifier.Source: §2 · p. 2

3.1 Problem Setup

• ZSV maps each text to a K-dimensional vector of zero-shot alignment scores, with each user-defined concept forming an explicit, human-interpretable axis.No labeled text–concept pairs are assumed; activations can also support clustering and downstream tasks without retraining.Source: §3.1 · p. 3

3.2 Design Requirements

• Interactive exploration requires bounded, faithful scores, near-real-time inference, and execution on commodity hardware.The experiments use small open-source models that fit on a personal computer and expose next-token probabilities needed by some scorers.Source: §3.2 · p. 3

3.3 Zero-Shot Scoring Functions

• ZSV accepts any bounded text–concept alignment scorer, and the framework groups candidates into retriever-based and LLM-based families.Retrievers use fixed lexical or semantic vector spaces; LLM scorers estimate alignment through language-model inference.Source: §3.3 · p. 3

3.3.1 Retriever-Based Scorers

• Retriever-based scorers embed texts and concepts into shared spaces, offering efficient deterministic approximations for scoring or first-stage retrieval.Sparse Lexical Retrieval uses TF–IDF cosine similarity but relies on exact lexical overlap; Dense Neural Retrieval uses normalized embeddings and cosine similarity, while Sparse Neural Retrieval combines lexical matching with learned term expansion.Source: §3.3.1 · pp. 3–4

3.3.2 LLM-Based Scorers

• LLM-based scorers directly judge text–concept alignment through generated ratings or probabilistic inference, with greater semantic expressiveness but higher exhaustive cost.Direct Rating normalizes a generated scalar to [0, 1] and may be prompt-sensitive; PZS normalizes affirmative and negative next-token probability mass, while EM marginalizes alignment over sampled explanations.Source: §3.3.2 · p. 4

3.3.3 Hybrid Coarse-to-Fine Scoring

• EM-Hybrid applies a higher-fidelity LLM reranker only to a retriever-selected subset, leaving unselected pairs with coarse scores.Its final quality is limited by the first-stage scorer, which must not misrank candidates beyond recovery.Source: §3.3.3 · p. 4

4.1 Tagged Corpora: Semantic Faithfulness

• PZS leads on all reported semantic-faithfulness metrics for MMLU and Banking77, while DNR has notably worse ground-truth alignment on Brier.Table 1 reports means over 57 MMLU subjects and 77 Banking77 intents.Source: Table 1 · p. 5, §4.1 · p. 5
• Across four comparable open-weight models, PZS results are consistent, indicating the reported behavior is not specific to one LLM.The cross-model comparison is reported in Table 2.Source: Table 2 · p. 5, §4.1 · p. 5
• PZS offers the strongest practical compromise in the timing test: 2.9 s per single requirement, versus 68 ms for SLR and 54.0 s for EM.EM's higher runtime reflects its linear scaling in sampled explanations; PZS and DR use a single prompt.Source: §4.1 · p. 5
• PZS concept projections show expected structure: orthogonal subjects have negligible joint activation, while related physics concepts co-activate.These configurations make the induced concept space interpretable from natural-language requirements.Source: §4.1 · pp. 5–6, Figure 2 · p. 6

4.2.1 Binary axes

• On hotel-review binary axes, retrieval methods concentrate or collapse prototypes, while probabilistic LLM scorers yield the clearest separation across activation quadrants.PZS projections also ground the quadrants in example reviews.Source: §4.2.1 · p. 6, Figure 3 · p. 6

4.2.2 Graded axes

• Graded satisfaction axes capture intensity, but a neutral score can mean either genuine neutrality or an unmentioned topic; pairing them with binary relevance separates four mention regimes.The corresponding graded scores can be filtered to reviews mentioning both topics.Source: §4.2.2 · p. 7, Figure 4 · p. 7, Figure 5 · p. 7
• Off-topic reviews with extreme positive or negative sentiment show inflated or deflated cleanliness scores, a compositional sentiment bias that motivates binary-relevance filtering.The authors leave a full study of this phenomenon to future work.Source: §4.2.2 · pp. 7–8, Figure 6 · p. 8

4.2.3 UN speeches: multi-axis exploration

• UN General Debate projections show contrasting decadal-scale patterns of economic and environmental emphasis between two countries, aggregated by leadership era.The example applies ZSV to a non-consumer corpus without fine-tuning or task-specific training.Source: §4.2.3 · p. 8

5 Practitioner Guidance

• Use PZS by default: it produces the best metrics with reasonable processing times.The recommendation references Table 1.Source: §5 · p. 8
• Choose DNR when throughput matters more than faithfulness; it offers competitive AUROC at roughly two orders of magnitude less computation, but has poorer scaling and ground-truth alignment.The poorer alignment is reflected in Brier and Figure 3.Source: §5 · p. 8
• Choose among 3B–9B open-weight LLMs by Brier, not AUROC, because AUROC barely moves while Brier varies sharply.This comparison is reported in Table 2.Source: §5 · p. 8
• Pair graded axes with binary-relevance filtering because unfiltered low scores conflate unmentioned topics with neutral opinions, and off-topic scores show compositional sentiment bias.The bias is noted in Section 4.2.2.Source: §5 · p. 8

6 Conclusion

• Across tagged and untagged corpora, probabilistic zero-shot scoring offers the strongest practical compromise between semantic faithfulness and computational cost.Different scorer families also induce sharply different visualization geometries.Source: §6 · p. 8
• Pairing graded scores with binary relevance queries resolves the ambiguity between genuinely low intensity and a topic that is not mentioned, yielding an interpretable multi-axis visualization.The paired approach combines graded axes with a relevance signal.Source: §6 · p. 8
• Off-regime points expose a sentiment-driven halo effect in off-topic scores, which the paper leaves for future characterization.Source: §6 · p. 9
• The paper also demonstrates that PZS-scored concept activations can serve as interpretable features for a shallow decision-tree surrogate on ScienceQA.The experiment is presented as a possible future extension.Source: §6 · p. 9

Scope of evaluation

• The quantitative benchmark covers two tagged corpora and one untagged hotel-review corpus, all in English, and does not evaluate multilingual, specialized, or long-form text.The benchmark domains are general knowledge, customer intent, and consumer reviews.Source: §7 · p. 9

EM-Hybrid underperformance

• EM-Hybrid does not improve over its components on the evaluated benchmarks, and alternative selection policies or first-stage scorers that might mitigate this behavior are not explored.The authors attribute the result to a known limit of two-stage retrieval when a weak first-stage scorer misranks candidates.Source: §7 · p. 9

Graded axes and halo effect

• Graded axes are demonstrated only on the untagged hotel corpus, so their semantic faithfulness relative to PZS is not quantitatively benchmarked on tagged data.Source: §7 · p. 9
• The off-topic halo effect is a qualitative finding; its magnitude and dependence on scorer and prompt remain uncharacterized.Source: §7 · p. 9

Concept design

• Visualization interpretability and informativeness depend on how analysts phrase concepts, while paraphrase robustness and prompt sensitivity are not studied.Source: §7 · p. 9

A Implementation Details and Prompt Templates

• All scorers return normalized alignment scores in [0, 1], and the appendix records the implementation and prompt templates used.The text and requirement placeholders are substituted at scoring time.Source: A Implementation Details and Prompt… · p. 11

A.1 Decoding and Inference

• LLM scorers use one instruction-tuned open-weight model in half precision; DR generates one greedy token, whereas PZS and EM scoring steps read the last-position vocabulary softmax without generating tokens.EM also samples three short reasoning traces per text–requirement pair.Source: §A.1 · p. 11

A.2 Retriever-Based Scoring

• Retriever alignment is computed in vector space: SLR uses TF–IDF cosine similarity, DNR uses rescaled cosine similarity of normalized embeddings, and SNR uses a sparse dot product of learned-weight vectors.Source: §A.2 · p. 11

A.3 LLM-Based Scoring (Binary Mode)

• DR linearly maps generated ratings from [0, 10] to [0, 1], while PZS estimates alignment from next-token True/False probability mass.PZS collects surface variants, deduplicates their first token ids, and computes pT /(pT + pF + ε), with ε = 10^-6.Source: §A.3 · pp. 11–12

A.4 Graded Scoring

• Graded scoring replaces True/False with ordinal multiple-choice anchors whose analyst-supplied labels map to numeric values; experiments use five evenly spaced values from 0 to 1.Anchor labels vary by axis, while the values remain 0, 0.25, 0.5, 0.75, and 1.Source: §A.4 · p. 12, Table 3 · p. 12

A.5 Hybrid Configuration

• EM-Hybrid reranks the top 10% of texts per requirement selected by SLR, using EM with N = 3; unselected pairs retain their SLR scores.The implementation also supports alternative first-stage scorers, rerankers, and selection policies.Source: §A.5 · p. 13

• ZSV concept activations are tested as interpretable features for downstream prediction without task-specific representation learning.Source: B Concept-to-Decision Surrogates · p. 13
• On ScienceQA, fine-grained topical categories define PZS-scored concept axes, while coarse labels are language, natural, or social science.Source: B Concept-to-Decision Surrogates · p. 13
• A shallow depth-3 tree is distilled from a train-split embedding teacher using validation-split concept vectors and teacher probabilities.Source: B Concept-to-Decision Surrogates · p. 13
• Held-out evaluation reports surrogate fidelity to teacher probabilities and task accuracy against ground-truth labels; Table 4 summarizes both.Source: B Concept-to-Decision Surrogates · p. 13, Table 4 · p. 13
• Table 5 contrasts a correct path with a faithful reproduction of the teacher's wrong answer, supporting interpretation and post-hoc error diagnosis.Source: B Concept-to-Decision Surrogates · p. 13

• Each internal test asks whether text is about a concept; leaves predict Nat., Soc., or Lang. ScienceQA subject.Source: C Hotel Quadrant Examples · p. 14, Figure 8 · p. 14
• Figure 8's decision-tree surrogate thresholds human-defined requirements at internal nodes and predicts ScienceQA subject labels at leaves.Source: Figure 8 · p. 14
• Table 5 lays out alignment scores, thresholds, and error sources along paths for one correct and one incorrect classification.Source: Table 5 · p. 14
• Table 6 summarizes PZS cluster counts by activation regime and the prototype nearest each Figure 3 corner.Source: C Hotel Quadrant Examples · p. 13, Table 6 · p. 14