PaperlayerView exact revision on arXiv

Source-linked AI summary

Concept Driven Domain Adaptation: Finding an Abstract Needle in a Haystack

Haiming Zhao, Tai Wang, Kun Zhang, Xicheng Peng, Zhiyang Li

arXiv:2610.00973v1cs.CVphysics.ed-ph

TL;DR

Instructional video search starts with an abstract curriculum concept, whereas existing moment retrieval generally expects a description of an observable scene. CDDA adapts two-tower vision–language models using educational example–concept pairs and staged documentary supervision. On a curated physics benchmark, it improves pedagogically oriented retrieval while preserving concrete image–description alignment.

  • Problem

    Teachers need documentary segments that illustrate abstract curriculum concepts, but standard moment retrieval generally uses queries describing observable scenes rather than the principle to be taught.

    Source: §1 · p. 1, §2 · p. 3

  • Method

    CDDA structures the language space with textbook and handbook example–concept pairs, transfers that structure to documentary visuals with a frozen visual encoder, then jointly fine-tunes both encoders.

    Source: §1 · p. 2

  • Results

    CDDA improves pedagogically oriented retrieval on the curated physics benchmark while preserving concrete image–description alignment.

    Source: §4.3 · p. 8, §5 · p. 9

  • Takeaways & Limitations

    The benchmark results suggest curriculum concepts can serve as effective semantic anchors for instructional video retrieval.

    Source: §5 · p. 9

    The curated physics setting is constrained by segments that may correspond to multiple concepts and subtitle-based segmentation that can yield redundant or incomplete visual evidence.

    Source: §5 · p. 9

Abstract

from arXiv · show

Science teachers frequently search for documentary excerpts not by describing what appears on screen, but by querying the abstract concepts they intend to teach. This use case exposes a limitation of existing language-based video moment retrieval methods, which typically assume that queries describe observable events, whereas instructional search requires retrieving concrete visual phenomena that instantiate an underlying scientific principle. We study this setting as concept-to-example video retrieval, an abstract-needle-in-a-haystack problem where compact curriculum concepts must be grounded in temporally sparse documentary evidence. To bridge this abstraction gap, we propose Concept-Driven Domain Adaptation (CDDA), a three-stage framework for adapting two-tower vision-language models to concept-level retrieval. CDDA treats concepts as intermediate semantic anchors: it first structures the textual embedding space with textbook and teacher-handbook example-concept pairs, then transfers this concept-aware geometry to documentary visuals under a frozen visual encoder, and finally jointly adapts both encoders with sparse visual concept supervision. From a geometric perspective, this staged alignment reduces text-concept and vision-concept angular gaps, thereby encouraging concept-level adaptation while preserving the pretrained model's concrete image description alignment. On a curated middle-school physics retrieval benchmark, CDDA achieves stronger pedagogically oriented concept retrieval than several competitive multimodal baselines, including Qwen3-VL-Embedding-2B, while maintaining concrete image-text matching after adaptation.

1 Introduction

Teachers need short documentary segments that illustrate abstract curriculum concepts, but existing retrieval methods generally expect queries to describe observable scenes. CDDA structures concepts through educational examples and adapts vision–language models to retrieve concrete phenomena that instantiate them.

1 Introduction

  • Educational video search requires finding brief documentary segments that exemplify curriculum concepts, a task teachers otherwise address by manually searching long-form videos.Short clips can make phenomena observable and support classroom explanation, discussion, and demonstration.Source: §1 · p. 1
  • Existing video retrieval typically matches concrete scene descriptions, while instructional search begins with a principle whose relevant visual examples may have little lexical overlap with it.A universal-gravitation query could be relevant to planetary motion, falling bodies, tides, or historical demonstrations.Source: §1 · p. 2
  • Concept-to-example video retrieval asks models to retrieve short documentary segments that visually instantiate compact, abstract curriculum concepts.The evidence is concrete, multimodal, temporally sparse, and often indirectly related to query words in narrative documentaries.Source: §1 · p. 2
  • CDDA adapts two-tower vision–language models in three stages: structure text with educational example–concept pairs, align it to documentary visuals with a frozen visual encoder, then jointly fine-tune both encoders.The stages address the limited structure available from sparse segment–concept annotations alone.Source: §1 · p. 2
  • The paper formalizes concept-to-example retrieval and reports that concept-driven alignment improves abstract concept retrieval over original CLIP and a strong multimodal embedding baseline while preserving concrete image–text matching.Its framework combines textbook-derived textual supervision with sparse documentary annotations.Source: §1 · p. 2

2 Related works

Prior video-moment retrieval generally assumes queries describe what should be observed, while concept-to-example retrieval instead links curriculum principles to phenomena. Related work on model adaptation and abstract grounding motivates adding conceptual structure for this different relation.

2 Related works

  • Video-moment retrieval includes proposal-based [15], proposal-free [16, 17], and query-based or DETR-style methods [18–21, 5], but its query generally describes the moment to observe.The paper distinguishes concept-to-example retrieval as matching a curriculum principle to a phenomenon it governs.Source: §2 · pp. 2–3
  • Vision–language adaptation commonly targets image–caption or description alignment, whereas CDDA first structures language with example–concept supervision before adapting visual segments.The paper notes that sparse video–concept fine-tuning can fit associations without organizing the underlying principle–phenomenon relation.Source: §2 · p. 3
  • Prior abstract-grounding work includes metaphorical retrieval and structured concept grounding, while this task concerns latent scientific principles instantiated by observable phenomena.Educational texts provide intermediate supervision for aligning explanatory principles with the visual evidence they govern.Source: §2 · p. 3

3 Method

CDDA frames documentary search as ranking clips that instantiate curriculum concepts, rather than requiring literal query-to-scene matches. It uses concept-aware text supervision and staged transfer to align clips with those concepts.

3.1 Problem Formulation

  • The task ranks documentary clips as concrete visual examples of a curriculum concept, using textual examples and sparse visual annotations as complementary supervision.Relevant clips need not literally describe or name the queried concept.Source: §3.1 · p. 3

3.2 Model Overview

  • CDDA adapts a two-tower vision–language model in three stages: structure text with example–concept pairs, transfer that structure to video, then jointly adapt both encoders.Stage 2 keeps the visual backbone frozen; Stage 3 uses textual replay.Source: §3.2 · pp. 3–4, Figure 2 · p. 4

3.3 Design Rationale: Concepts as Semantic Anchors

  • Reducing text–concept and video–concept angular distances lowers an upper bound on video–example distance, motivating concepts as intermediate semantic anchors.The argument explains the staged design but does not claim concept supervision alone guarantees video–text alignment.Source: §3.3 · pp. 4–5

3.4 Multi-Positive Concept Alignment

  • CDDA uses multi-positive concept alignment because multiple text descriptions or clips can instantiate one concept, avoiding their incorrect treatment as negatives.Concept prototypes come from normalized textual concept representations; when a concept appears once per batch, the objective reduces to symmetric contrastive learning.Source: §3.4 · p. 5

3.5 Stage 1: Concept-Text Structuring

  • Stage 1 updates the text encoder and projection head to bring concept embeddings close to examples, explanations, and phenomenon descriptions that instantiate them.The visual encoder is not used in this stage.Source: §3.5 · p. 5

3.6 Stage 2: Video-Concept Alignment

  • Stage 2 aligns documentary clips to Stage-1 concept prototypes while freezing the visual backbone and updating projection heads and text-side parameters.Freezing provides a stable visual anchor and reduces overfitting under sparse video supervision.Source: §3.6 · p. 5

3.7 Stage 3: Joint Concept-Driven Adaptation

  • Stage 3 unfreezes both encoders for documentary-domain adaptation, while textual replay regularizes the concept space learned in Stage 1.The video loss adapts the model to visual instances of scientific concepts without discarding the educational-text structure.Source: §3.7 · p. 6

3.8 Inference

  • At inference, CDDA embeds the concept query and precomputed candidate clips, ranks clips, and can merge adjacent high-ranking clips into longer classroom excerpts.The returned clips serve as visual examples of the queried scientific concept.Source: §3.8 · p. 6

4 Experiments

Experiments evaluate concept-to-example retrieval on a curated middle-school physics benchmark, finding stronger retrieval while testing whether adaptation preserves concrete image–description matching and which stages contribute to gains.

4.1 Experimental Setup

  • The benchmark uses 62 image-concept examples across five force-unit concepts and evaluates 146 non-overlapping segments from a separate 52.5-minute documentary.Two senior pedagogical experts cross-check relevance labels; evaluation includes top-10 pedagogical hits and MAP/NDCG at K=5 and 10. Stage 1 uses 150 textual example-concept pairs, including 118 handbook sentences.Source: §4.1 · pp. 6–7

4.2 Main Results

  • CDDA ties Gemini Embedding 2 in pedagogical score and outperforms open or trainable baselines on the same candidate pool.The Force query is especially challenging: CDDA retrieves 5 of 6 relevant segments, versus at most 2 for general-purpose multimodal baselines.Source: Table 2 · p. 7, §4.2 · p. 8
  • Against Qwen3-VL-2B, CDDA improves MAP@10 from 0.3723 to 0.3848 and NDCG@5 from 0.4814 to 0.5094.Gemini Embedding 2 obtains the strongest MAP values, while CDDA remains competitive with a smaller CN-CLIP backbone.Source: Table 3 · p. 8, §4.2 · p. 8

4.3 Diagnostics and Ablations

  • CDDA preserves CN-CLIP’s image–description correctness pattern on six held-out images while improving several image–concept decisions.This diagnostic tests whether curriculum-level alignment can be added without erasing concrete visual-textual correspondence.Source: Table 4 · p. 8, §4.3 · p. 8
  • Removing Stage 1 lowers MAP@10 from 0.3848 to 0.2974, NDCG@10 from 0.5402 to 0.4309, and pedagogical score from 24 to 20.Literal-description variants score 12 or 13 pedagogical points; the reported gain comes from structuring text with example-concept supervision before cross-modal adaptation, not merely exposing concept tokens.Source: §4.3 · pp. 8–9
  • CDDA ranks the apple-to-Earth non-trivial match under Universal Gravity; Fig. 4 highlights are qualitative context, not a reproducible baseline.The example bridges near-surface falling with orbital motion, whereas commercial highlights emphasize biography or person-centered moments.Source: §4.3 · p. 9, Figure 4 · p. 9, Figure 5 · p. 9

5 Limitations and Conclusion

The paper introduces concept-to-example retrieval and CDDA for searching documentaries with curriculum concepts. On a curated physics benchmark, CDDA improves pedagogically oriented retrieval while preserving concrete image–description alignment, though evaluation is constrained by concept overlap and subtitle-based segmentation.

5 Limitations and Conclusion

  • Evaluation is less exact when one segment corresponds to multiple concepts, and subtitle-based segmentation can produce redundant or incomplete visual evidence.The authors call for broader benchmarks and further separation of visual, subtitle, narration, and audio cues.Source: §5 · p. 9
  • CDDA structures text embeddings with educational example–concept pairs and transfers that concept-aware geometry to documentary visuals through staged adaptation.The paper presents curriculum concepts as semantic anchors for connecting abstract concepts with visual evidence.Source: §5 · p. 9
  • On the curated physics benchmark, CDDA improves pedagogically oriented retrieval while preserving concrete image–description alignment.The paper suggests curriculum concepts can serve as semantic anchors for instructional video retrieval.Source: §5 · p. 9

Technical Appendices and Supplementary Material

The supplementary material identifies three discussion topics: retrieval diversity, pedagogical exactness, and model parameter scales.

A Discussion

A.1 On the diversity of retrieval results

The discussion examines whether models retrieve varied documentary examples for “Force” and relate them to the broader curriculum concept. It reports that several multimodal models returned only one of the illustrated force examples, while results for “Force” overlapped substantially with those for “Gravity” and “Universal Gravity.”

A.1 On the diversity of retrieval results

  • Only Fig. A1d among the illustrated “Force” segments was retrieved by Gemini Embedding 2, Qwen3-VL-Embedding-2B, ImageBind, and InternVideo-Next.Fig. A1 shows representative images from different segments; the caption describes forces other than gravity or universal gravity.Source: §A.1 · p. 12, Figure A1 · p. 13
  • The authors argue that the models likely understand the individual scenes but fail to connect some scenes, such as metaphorical gears, to a superordinate curricular concept.The documentary was divided into 146 segments averaging 21.6 seconds, most showing one or two short, complete scenes.Source: §A.1 · p. 12
  • Results for “Force” largely overlapped with those for “Gravity” or “Universal Gravity,” including scenes involving a lever-like solar panel and Newton narratives.Source: §A.1 · p. 13

A.2 On the pedagogical exactness of retrieval results

The discussion finds that concept-example supervision helps CDDA avoid a pedagogically inexact “Gravity” result, while showing the difficulty of exact retrieval for abstract and identity-based queries. It also compares the models’ parameter scales and video-embedding procedures.

A.2 On the pedagogical exactness of retrieval results

  • CDDA and CDDA (II&III) avoid returning the non-trivial segment in the top 10 for “Gravity,” while CN-CLIP and CDDA (II&III)d inherit the inexact match.Source: §A.2 · p. 13
  • CDDA (II&III)d+c’s result on the non-trivial segment supports the authors’ claim that Stage 1 training is necessary; combining a phenomenon description and concept in one text item was less effective.Source: §A.2 · p. 13
  • For the “Newton’s First Law” query, Gemini Embed. 2 was the only Table 2 model to retrieve the Fig. 5 segment, ranking it fifth among its top 10 results.The authors excluded cannonball-flying pictures from training and instead paired a picture similar to Fig. A2d with the concept.Source: §A.2 · p. 13
  • The discussion treats “Force” as a test of result diversity, “Gravity” queries as tests of pedagogical exactness, and “Newton’s First Law” as a ceiling-level difficulty.Gemini Embedding 2 performed outstandingly on the last query but failed on “Newton.”Source: §A.2 · p. 13

A.3 On the Gemini Embed. 2 ’s retrieval results for the “Newton” query

  • For “Newton,” Gemini Embedding 2’s top 10 excludes the portrait-based ground truth and instead returns scenes of actors portraying Newton or narrating his contributions.The authors speculate that the model may treat Newton’s contributions as the core essence of the symbol “Newton.”Source: §A.3 · p. 14

A.4 On the parameter scale

  • CDDA uses the relatively small CN-CLIP model yet achieves competitive results; only Gemini-Embedding-2 directly returns a video-segment embedding.Table A1 presents the parameter scales of the five large models; several models sample frames and pool their embeddings for video segments.Source: §A.4 · p. 14, Table A1 · p. 14

B.1 Proof of Eq. (7) in Subsection 3.3

The proof relates the text–vision angle to each vector’s angle from the concept and their residual angle. The appendix also documents Stage 1 concept–example pairs and the mapping-preservation diagnostic.

B.1 Proof of Eq. (7) in Subsection 3.3

  • When both vectors lie within 45° of the concept, their cosine similarity remains positive regardless of their residual angle, indicating some semantic similarity.The proof derives this from the angular relation in Eq. (A5).Source: §B.1 · pp. 14–15

B.2 Samples of Concept–Example Pairs

  • Stage 1 uses 150 concept–example pairs across six concepts, split into 103 training, 31 validation, and 16 test items.The examples describe phenomena, facts, or explanations that can instantiate a concept in instruction, rather than literal captions.Source: §B.2 · p. 15, Table A2 · p. 15

B.3 Details on the Mapping Preservation Examination

  • The mapping-preservation diagnostic checks whether adaptation retains concrete image–description matching while improving image–concept assignment.Table A3 distinguishes image descriptions from associated or predicted concepts; “identical to CN-CLIP” refers only to the generated description.Source: §B.3 · p. 15

C Additional Experimental Details

The appendix consolidates experimental details for reproducibility and interpretation, with its tables and figures tied to the main experiments’ held-out retrieval corpus. It also provides detailed outputs for the mapping-preservation examination.

C Additional Experimental Details

  • The appendix covers dataset construction, segmentation, baseline protocols, ablations, training settings, metrics, query images, and generated illustrations.These details are presented to support reproduction and interpretation of the Section 4 experiments.Source: C Additional Experimental Details · p. 15
  • All appendix tables and figures are tied to the same held-out retrieval corpus used in the main experiments.Source: C Additional Experimental Details · p. 15
  • Table A3 reports image descriptions alongside associated or predicted concepts for the mapping-preservation examination.Phrases such as “identical to CN-CLIP” refer to generated descriptions, not concept predictions.Source: Table A3 · p. 16

C.1 Dataset Construction and Annotation

The supplementary section specifies the retrieval corpus, annotation rules, candidate segmentation, model protocols, training settings, and metrics. It also explains the pedagogical treatment of a non-trivial match and cautions about generated illustrations.

C.1 Dataset Construction and Annotation

  • Image–concept examples come from listed documentaries and Motion Mountain, while Newton’s Dark Secrets is held out from adaptation.Two pedagogical experts cross-check labels; a segment is relevant when it can serve as a concrete teaching example, even without the concept word in subtitles.Source: §C.1 · pp. 16–17, Table A4 · p. 16
  • The five queries are Force, Gravity, Newton, Newton’s First Law, and Universal Gravity; relevant-segment counts define MAP@K denominators and NDCG@K ideal rankings.The evaluation corpus contains 146 segments; the non-trivial match receives additional pedagogical credit only for Universal Gravity in the top-10 analysis, but is binary relevant for MAP/NDCG.Source: §C.1 · pp. 16–17, Table A5 · p. 16, §C.5 · p. 18

C.2 Candidate Segmentation

  • Subtitle embeddings and semantic continuity groupings produce 146 fixed, non-overlapping candidate segments from 832 subtitles for all compared methods.Only segment and query representations vary across methods.Source: §C.2 · p. 17

C.3 Model and Baseline Protocols

  • CDDA uses CN-CLIP with Chinese RoBERTa; InternVideo-Next serves as a visual-retrieval diagnostic using one manually selected query image per concept.Native retrieval models rank segment embeddings against concept queries, while text-only baselines search segment subtitles.Source: §C.3 · p. 17, Figure A2 · p. 17, Table A6 · p. 18

C.4 Implementation Details

  • Visual adaptation stages share batch size, learning rate, epoch count, and temperature; validation supports model selection, while six test images are reserved for the preservation diagnostic.Table A7 lists the training and inference settings.Source: §C.4 · p. 17, Table A7 · p. 18

C.5 Metric Definitions

  • MAP and NDCG are averaged over five queries, with the non-trivial match counted as one binary-relevant segment; its extra pedagogical credit applies only to top-10 analysis.The ranking metrics use binary relevance.Source: §C.1 · p. 16, §C.5 · p. 18

C.6 Generated Cannonball Illustrations

  • Generated cannonball illustrations may contain physical inconsistencies, ambiguous trajectories, or prompt-intent deviations; retrieval instead targets vetted documentary evidence for teaching.Source: §C.6 · p. 19, Figure A3 · p. 19

C.7 Valid loss curve

Related papers

Loading 2610.00973v1…