PaperlayerView exact revision on arXiv

Source-linked AI summary

Behind Video Generation: Observations and Insights on Multimodal Understanding from Kling

Kling Team, Dong Chen, Jialu Chen, Xinlong Chen, Yifan Dai, Haotian Dong, Binzong Geng, Yushuo Guan, Minghui Guo, Guanlin He, Guanxiao He, Jingyun Hua, Yanwei Huang, Yiyan Ji, Xueping Jia, Yizhuo Jia, Bozhou Li, Jiaze Li, Min Li, Shihao Li, Shujie Li, Shengyi Liao, Weihong Lin, Shiyuan Liu, Xinyu Liu, Siyang Mao, Xiaoru Qu, Dianmo Sheng, Yang Shi, Jiafu Tang, Longxiang Tang, Yuqi Tang, Chengzhuo Tong, Kuo Wang, Qixun Wang, Rui Wang, Tianduo Wang, Yuran Wang, Zekun Wang, Zihao Wang, Boyang Xia, Yifan Xu, Liu Yang, Zhenlong Yuan, Bohan Zeng, Mujin Zhang, Yuanxing Zhang, Zhicheng Zhang, Zhuoran Zhang, Yuanhong Zheng, Kun Gai, Chengru Song, Pengfei Wan

arXiv:2611.04916v1cs.CV

TL;DR

Video generation’s move to coordinated sequences makes semantic control a compositional challenge, and visual fidelity alone does not ensure correct actions or continuity. The report examines multimodal understanding across the generation chain; it reports that agentic prompt enhancement improves reasonableness, faithfulness, and vividness together.

  • Problem

    Sequence-level generation requires semantic control because constraints interact over time, while visual fidelity alone does not ensure correct actions or narrative continuity.

    Source: §1 · p. 3

  • Method

    The report synthesizes Kling’s multimodal-understanding modules across the data, conditioning, alignment, and evaluation chain, including captioning, prompt enhancement, text encoding, reward modeling, and task-specific evaluation.

    Source: §3.3 · p. 13, §5.6 · p. 24, §7.2 · p. 29

  • Results

    Agentic prompt enhancement improves reasonableness, faithfulness, and vividness together by refining contradictory requests and retrieving captions of compelling videos.

    Source: §1 · p. 4

  • Takeaways & Limitations

    Evaluators should verify atomic, source-grounded facts and their temporal scopes rather than rely on surface-text matching.

    Source: §3.3 · p. 13

    In DiT-in-the-loop reinforcement learning, the rewards do not measure excitement, and directly rewarding ambition without reintroducing artifacts remains open.

    Source: §5.3 · p. 22

Abstract

from arXiv · show

Video generation has progressed from isolated clips to coherent short-form audiovisual sequences, and the bottleneck is shifting from rendering to semantic control. This report summarizes the observations and insights on multimodal understanding gained from successive generations of Kling. We argue that generation quality is the projection of multimodal understanding through the entire data--conditioning--alignment--evaluation chain, and we examine this chain module by module. Kling-MLLM, a shared multimodal foundation model that unifies image, video, audio, synchronized audio--video, and interleaved inputs, is specialized for each downstream module. Because a DiT is trained on far fewer training samples than a language model, we treat captioning as a semantic contract that maximizes description density, and extend it to reference-grounded captioning that makes reference bindings explicit. Prompt enhancement translates user intent into this caption distribution: basic PE is derived from captions of compelling videos and refined with text-level and generator-in-the-loop reinforcement learning, while agentic PE refines the story and retrieves compelling examples. We show that a reliable text encoder comes from perceptual training rather than from stronger language ability, and that candidates can be screened from text alone before end-to-end training. Reward modeling pairs a general reward model for explainable, localized critiques with a decision reward model for fast multidimensional scoring, treating perceptual bandwidth as a first-class design parameter. Finally, we review task-specific automatic evaluation, focusing on protocols that expose scoring shortcuts and help trace failures to their upstream source. We hope these lessons offer practical guidance for building the understanding stack behind controllable video generation.

1 Introduction

As video generation advances to coordinated sequences, semantic control becomes compositional: constraints interact across shots and errors accumulate over time. The paper frames generation quality as multimodal understanding projected through the data–conditioning–alignment–evaluation chain.

1 Introduction

  • The unit of progress expands from frames and clips to shots and complete sequences, where constraints interact and errors accumulate over time.Decisions in one shot can affect how the next is interpreted.Source: §1 · p. 3
  • Generation failures are not diagnoses: similar symptoms can arise from omitted caption supervision, lost temporal structure, reward bias, or an evaluator that misses the requested event.The paper therefore treats understanding as a property of the full chain, not a score attached to one model.Source: §1 · p. 3
  • The bottleneck is shifting from rendering to semantic control, since visual fidelity alone does not ensure correct actions, camera language, reference binding, or narrative continuity.These capabilities are weakly represented in general-purpose multimodal benchmarks despite their importance for controllable and editable generation.Source: §1 · p. 3
  • Because a DiT trains on far fewer samples than a language model, captions must maximize description density, with explicit reference bindings and perceptual text encoding carrying supervision to the generator.The passage identifies description density per unit of video duration as the central measure of data efficiency.Source: §1 · p. 3
  • Basic prompt enhancement translates requests into the generator’s caption distribution, while agentic refinement repairs contradictory requests and retrieves compelling examples to improve reasonableness, faithfulness, and vividness together.The paper presents these agentic steps as dividing work that would otherwise place conflicting goals on one model.Source: §1 · pp. 3–4
  • Task-specific, decomposed evaluation and inexpensive text-only proxies can expose scoring shortcuts and help trace failures upstream before costly end-to-end training.The report traces these practices alongside Kling-MLLM, captioning, prompt enhancement, text encoding, and reward modeling.Source: §1 · p. 4

2 Multimodal Understanding Foundation Models

Kling-MLLM is a shared multimodal foundation model specialized through task-specific post-training, with its architecture and curriculum designed to integrate diverse modalities and longer temporal evidence. Evaluation shows the clearest gains in video perception and temporal understanding, audio temporal grounding, and audio–video captioning, while results in several other areas remain mixed.

2.1 Objective and Scope

  • Kling-MLLM centralizes multimodal perception and reasoning in a reusable foundation model, then specializes checkpoints for downstream tasks and accuracy–latency needs.Its autoregressive interface unifies images, videos, audio, synchronized audio–video, and interleaved media.Source: §2.1 · p. 4

2.2 Kling-MLLM Architecture

  • A single LLM consumes a common sequence of text and continuous visual and acoustic embeddings, with dynamic timestamped video frames and jointly serialized synchronized audio–video.The scale-dependent audio towers share an embedding interface; synchronized audio tokens are inserted before visual blocks at matching timestamps.Source: §2.2 · pp. 4–6, Figure 2 · p. 5

2.3 Four-Stage Training Curriculum

  • The four-stage curriculum progresses from audio-only initialization to joint multimodal alignment, long-sequence temporal training, and long-context consolidation.The stages broaden task and modality coverage; merging Stage 1 and Stage 4 checkpoints yields a better checkpoint than either alone.Source: §2.3 · pp. 6–7

2.4 Data Construction

  • Data construction combines public, Kling-specific, and agent-oriented sources, with automated ingestion, modality-specific processing, iterative audits, and stage assignment.Internal task data target perception, reference understanding, editing, and multimodal agent tasks.Source: §2.4 · pp. 7–8

2.5 Evaluation

  • Evaluation results are mixed across modalities: Kling variants lead selected audio and interleaving measures, while other systems lead several audio–video and image benchmarks.Kling-MLLM-397B-A17B leads both OmniCap-IF metrics and is competitive on interleaving; Doubao2.0-Lite leads LibriSpeech other, WorldSense, and some image measures.Source: Table 2 · p. 8, §2.5 · pp. 8–9, Table 3 · p. 9
  • Kling-MLLM-397B-A17B shows the clearest gains in video perception and temporal understanding, audio temporal grounding, and audio–video captioning, while results in several other areas remain mixed.It leads five video benchmark columns; other systems lead several video, image, speech-recognition, and interleaving results.Source: §2.5 · pp. 9–10, Table 5 · p. 10

3 Caption: From Pixels to Trainable Semantics

Generation-oriented captioning treats captions as conditional supervision, requiring faithful, complete, temporally organized descriptions rather than fluent summaries. The section develops a production pipeline and evaluation approach, then reports Kling-Captioner’s results across three benchmarks.

3 Caption: From Pixels to Trainable Semantics

  • A caption is conditional supervision: omissions leave controllable factors underspecified, while inaccuracies inject conflicting signals; fluent captions can still misstate actions, timing, sound, or text.Lexical-overlap scores can conceal these errors, and question-based verification risks conflating captioning with VQA capability.Source: §3 · p. 10

3.1 The Semantic Contract: Content and Conventions

  • Generation-oriented captions should function as compact temporal data structures, preserving identity, action, timing, spatial and camera information, visible text, and inspected audio when relevant.Mandatory content and optional fields depend on input evidence and downstream use; retrieval favors short captions, while generation benefits from dense ones.Source: §3.1 · pp. 11–12

3.2 The Supervision Pipeline: Separating Complex Discovery from Efficient Deployment

  • A multi-expert supervision pipeline routes videos, preserves perceptual evidence, integrates specialist claims, and applies conjunctive quality gates before distilling outputs into a single-pass student.Mining unobserved dimensions and resolving expert conflicts expands coverage and accuracy beyond simply scaling a general annotator.Source: §3.2 · p. 12

3.3 Acceptance Criteria: Conjunctive Quality and Atomic Benchmarking

  • Caption usability requires factual accuracy, semantic coverage, beneficial diversity, and internal consistency, assessed through atomic source-grounded facts and complementary evaluation views.Judges require stability checks, human calibration, ranking robustness, and caption-length control.Source: §3.3 · pp. 12–13

3.4 Downstream Validation: The Generator as the Final Consumer

  • Readable captions are not necessarily usable supervision; controlled downstream experiments should match lengths and verified fact counts and test whether targeted caption changes transfer to corresponding generation capabilities.Human preferences and fine-grained metrics can expose control failures hidden by global text–video similarity scores.Source: §3.4 · pp. 13–14

3.5 Experiments

  • Kling-Captioner outperforms baselines across VidCapBench, UGC-VideoCap, and the internal benchmark, leading in overall accuracy, precision, and coverage or in every evaluated dimension.On the internal benchmark, it leads across motion, camera, and sound targets.Source: Table 8 · p. 14, §3.5 · p. 14

3.6 Discussion

  • Open challenges include balancing temporal and spatial fidelity against context capacity, managing long-video caption length and referential consistency, and aligning audiovisual events and speakers.Source: §3.6 · p. 14

4 Reference-Grounded Captioning: Orchestrating Coherent Multimodal Anchors

Reference-grounded captioning (RGC) makes reference-to-target correspondences and their scopes explicit, so captions can specify what references supply and what the target description must add. The section presents relational supervision and a unified representation, then reports Kling-RGC’s benchmark results.

4 Reference-Grounded Captioning: Orchestrating Coherent Multimodal Anchors

  • Reference-conditioned generation requires binding each reference to the correct target entity, property, modality, and temporal interval; errors can cause identity swaps, attribute leakage, or broken state continuity.Source: §4 · pp. 14–15

4.1 Task Definition: Reference Semantics and Systemic Challenges

  • RGC faces reference-target ambiguity, asymmetric modality authority, fragile identity binding across disruptions, and shortcuts that omit IDs or complex edits.Evaluation must check required coverage as well as correctness.Source: §4.1 · p. 16

4.2 The Semantic Contract: Scoped Binding, Residual Sufficiency, and Authority Consistency

  • Reliable RGC combines scoped bindings, residual descriptions of generation-relevant facts unavailable from references, and consistency with each reference’s authority.Together, these make the caption a control schematic that delegates reference-governed properties while filling contextual gaps.Source: §4.2 · pp. 16–17

4.3 Relational Supervision: Learning from Arbitrary Video Pairs

  • Relational supervision uses sufficiently similar video pairs as reference-target training examples, stating what to retain from the reference and what to change in the target.A fixed target can be paired with multiple references, expanding training examples without requiring a separate edited result for each pair.Source: §4.3 · p. 17

4.4 Unified Representation: Integrating Captions with Reference Pointers

  • A unified representation lets standard captioning and RGC share a captioner, text encoder, prompt-enhancement policy, and validators, combining broad semantic supervision with reference-binding logic from scarcer multi-reference data.The representation uses stable pointers and explicit change scopes while treating standard captioning as a zero-reference case.Source: §4.4 · p. 17

4.5 The Supervision Pipeline: Extending for Cross-Modal Correspondences

  • The cross-modal pipeline extends the multi-expert design with reference parsing, correspondence checks, and conflict resolution, while a hybrid representation connects scoped bindings to shot or event timestamps.Source: §4.5 · p. 18, §4.4 · p. 18

4.6 Experiments

  • Kling-RGC leads across all ViDiC metrics for difference and similarity captioning; on MRVBench, it has the highest keypoint coverage, VQA accuracy, and reference-binding precision, while remaining highly competitive with Gemini-4-Argon.It substantially outperforms Doubao2.1-Lite across the remaining MRVBench metrics.Source: Table 12 · p. 19, §4.6 · p. 19

4.7 Discussion

  • Future work includes diverse reference inputs such as regions, sketches, and trajectories, complex multimodal combinations for one target, and finer-grained temporal binding.These directions require targeted supervision and evaluation.Source: §4.7 · p. 19

5 Prompt Enhancement: From User Intent to the Caption Distribution

Prompt enhancement (PE) translates user requests into the generator’s caption dialect while separating story decisions from final condition writing. Basic PE is built from compelling-video captions and refined through text-level and generator-in-the-loop rewards; agentic PE refines plots and retrieves examples.

5 Prompt Enhancement: From User Intent to the Caption Distribution

  • Basic PE rewrites requests into the caption distribution, while agentic PE makes the story coherent and retrieves compelling examples before basic PE writes the final condition.Figure 5 depicts three basic-PE training stages and the inference-time relationship between the two modules.Source: §5 · p. 19, Figure 5 · p. 20

5.1 Basic PE: Caption Alignment through Backward Construction

  • Training from compelling-video captions gives PE a more vivid creative ceiling than uniform training-corpus captions, which produce faithful but generic framing, lighting, and action.Source captions are selected using aesthetic and motion scores and vividness, with domain and length quotas to avoid genre collapse.Source: §5.1 · p. 21

5.2 Text-Level Quality: Faithfulness, Reasonableness, and Vividness

  • Text-level faithfulness, reasonableness, and vividness checks can filter training pairs and optimize PE before video generation, making end-to-end RL cheaper and its rewards easier to interpret.Faithfulness checks user requirements; reasonableness checks spatial and interaction consistency; vividness rewards contextually appropriate expressive elements.Source: §5.2 · pp. 21–22

5.3 DiT-in-the-Loop RL

  • DiT-in-the-loop RL makes severe failures rarer and worst-case quality better, but reduces peak vividness by avoiding high-risk choreography, camera moves, and dense interactions.The authors suggest that rewards penalizing risk without measuring excitement may explain this trade-off; rewarding ambition without reintroducing artifacts remains open.Source: §5.3 · p. 22

5.4 Agentic PE: Story Refinement and Retrieval

  • Story refinement resolves contradictions in AI-written prompts upstream, allowing basic PE to preserve the refined intent while translating it into caption language.The refiner makes minimal edits, preserves non-conflicting hard constraints, and can pass an unresolved conflict through with a flag rather than guess.Source: §5.4 · pp. 22–23
  • Agentic PE retrieves five compelling captions as optional context, and strict rejection sampling filters targets that import unsupported entities, attributes, or events.Training includes irrelevant retrieval examples so basic PE learns that retrieved captions are optional evidence.Source: Figure 5 · p. 20, §5.4 · p. 23

5.5 Experiments

  • Table 13 reports cumulative PE-component contributions using held-out prompt text scores and fixed-generator video benchmarks, but the supplied passages provide no component-level values.The video measures cover story completion, joint audio-video generation, audiovisual editing, and multi-reference generation.Source: §5.5 · p. 23, Table 13 · p. 24

5.6 Discussion

  • PE divides responsibilities between caption-dialect translation and content planning; open directions include rewarding ambitious executable content and jointly training both modules against shared verifiers.Tool use, previews, and post-generation verification are also being explored, with latency and compute identified as bottlenecks.Source: §5.6 · p. 24

6 Text Encoder: Perceptual Grounding for Conditioning

The text encoder turns prompts into the sequence representations consumed by the video generator, but general language ability and standard proxy benchmarks do not ensure reliable video conditioning. The paper advocates sequence-level screening, perceptual adaptation, multi-layer conditioning, and stable fusion; all three refinements improve human preference over the Qwen baseline.

6.1 Representation Evaluation Beyond General Proxies

  • General QA and pooled retrieval proxies do not directly evaluate the sequence of prefill hidden states that a DiT consumes for conditioning.Minimal semantic contrasts on those hidden-state sequences can screen candidate backbones and checkpoints before costly end-to-end DiT training.Source: §6.1 · p. 25

6.2 Adapting Foundation Backbones for Video Conditioning

  • Broad multimodal instruction tuning offers no reliable conditioning gain, while reasoning-oriented post-training can degrade hidden-state grounding; targeted perceptual adaptation instead strengthens visual and temporal detail encoding.The proposed adaptation builds on Kling-MLLM using dense video descriptions and spatiotemporal perception tasks.Source: §6.2 · pp. 25–26

6.3 Multi-Granular Semantic Conditioning

  • Aggregating complementary encoder layers gives the DiT one sequence representation that can preserve lexical details, entity relations, temporal structure, and global context.The paper links the hierarchy of prompt information to complementary abstraction levels across encoder depth.Source: §6.3 · p. 26

6.4 Stable Conditioning During Diffusion Training

  • Learnable fusion weights can drift as diffusion training shifts from coarse structure toward fine details, making the condition non-stationary; stabilizing fusion prevents that moving target.The authors identify changes in training stages, timestep sampling, and video resolution as shifting optimization targets.Source: §6.4 · p. 26

6.5 Empirical Evaluation

  • The evaluation compares generated videos under controlled training settings, using paired human judgments for overall preference, instruction following, and aesthetics.Each Kling-MLLM configuration is compared with the same Qwen text-encoder baseline using (G + S)/(S + B).Source: §6.5 · pp. 26–27, Table 14 · p. 27
  • All three text-encoder refinements improve preference over the Qwen baseline across every dimension, with gains increasing as multimodal adaptation, multi-granular conditioning, and stability are added.The reported increases in overall preference and aesthetics exceed those in instruction following.Source: Table 14 · p. 27, §6.5 · p. 27

6.6 Discussion

  • The text encoder maps structured text into an implicit, multi-granular scene representation that is screened, perceptually adapted, aggregated across layers, and stabilized during diffusion training.The paper frames this encoder as the interface translating multimodal understanding into the generator’s continuous conditioning space.Source: §6 · p. 24, §6.6 · p. 27

7 Reward Models: Turning Understanding into an Optimization Signal

The reward-model section treats reward as an executable utility signal and separates explainable, localized critique from fast multidimensional scoring. Its design addresses perceptual, attribution, and latency challenges, and the DRM leads on most internal dimensions and three public benchmarks.

7.1 Objectives, Challenges, and Application Scenarios

  • Reward modeling must address perceptual bottlenecks, coupled quality dimensions and loser-centric labels, and the trade-off between localized explanations and low-latency scoring.The GRM provides explainable, localized critiques; the DRM scores many dimensions quickly across five production scenarios.Source: §7.1 · pp. 27–28

7.2 Model Design

  • Scoring motion magnitude separately counteracts the stillness incentive created by optimizing against defect penalties alone.Slow or frozen videos can contain fewer anatomical or physical errors.Source: §7.2 · p. 28
  • Stage 1 trains the vision encoder and aligner for low-level artifact perception, using dense sampling for motion and artifacts and sparse, high-resolution frames for visual quality.The LLM stays frozen, and 20% Stage 1 replay is retained in later stages.Source: §7.2 · p. 28, Figure 7 · p. 29
  • The GRM learns structured, localized critiques from annotated defects and checklists, then uses verifiable rewards for rating calibration, defect detection, tagging, and spatiotemporal localization.Unmatched ground-truth defects are penalized, preventing reward gains from reporting fewer defects.Source: §7.2 · p. 29, Figure 7 · p. 29
  • The DRM answers fine-grained defect, prompt-verification, and pairwise questions in one prefill, with a shared two-video context improving subtle-motion judgments.Fixed answer placeholders support independent field scoring; joint pointwise–pairwise training and temporal grounding are used, while confidence-gated order-swapped inference invokes a second pass only for low-margin cases.Source: §7.2 · pp. 29–31, Figure 7 · p. 29

7.3 Experimental Evaluation

  • The DRM leads on eight of nine internal dimensions and all three public benchmarks, while Gemini-4-Argon remains stronger on multi-shot prompt following.The GRM already exceeds Gemini-4-Argon in macro accuracy; Stage 3 adds 3.1 points at single-prefill inference cost.Source: Table 17 · p. 31, Table 18 · p. 31, §7.3 · p. 31

7.4 Discussion

  • The shared taxonomy links DRM scores, GRM-localized defects, and checkpoint or data-screening slices, supporting attribution of reward-model failures to specific dimensions and upstream sources.Because generators optimize reward signals, those signals alone cannot reveal optimization shortcuts; evaluation protocols must remain independent.Source: §7.4 · p. 31

8 Automatic Evaluation: Measuring What Each Task Requires

Automatic evaluation should follow the generation task because success criteria and score-inflating shortcuts differ across tasks. The section reviews task-specific criteria, routes properties to suitable measurement tools, and recommends protocols that support reliable diagnosis.

8.1 Task Taxonomy: What Should Be Evaluated?

  • Evaluation should be task-specific because success criteria and ways to inflate scores differ across generation tasks.A universal quality score is not the objective; evaluation identifies the target task, relevant failure modes, suitable tools, and measurement reliability.Source: §8.1 · p. 32, §8 · p. 32
  • Caption evaluation favors semantic coverage over lexical overlap, while long-video captions also require referential consistency and comparative captions require separate similarity and difference checks.VidCapBench reports 0.89–0.92 correlations between caption accuracy and downstream text-to-video generation quality; metric gains should translate into downstream gains.Source: §8.1 · p. 32
  • Text-driven generation evaluation separates instruction following, perceptual quality, physical plausibility, and audio dimensions because success on one does not guarantee success on others.VBench decomposes quality into 16 dimensions, while audiovisual benchmarks separately assess video and audio quality, alignment, synchronization, instruction following, and realism.Source: §8.1 · pp. 32–33
  • Reference and keyframe evaluation checks identity, detail, role binding, integration, and temporal response rather than embedding similarity or keyframe presence alone.Keyframe response includes visual fidelity, order, position, persistence, and uniqueness, distinguishing temporal integration from flashes, freezes, and reversed sequences.Source: §8.1 · p. 33
  • Editing evaluation jointly checks requested changes and preservation of unrelated source content, with response gating applied to preservation and realism measures.The multiplicative Editing Intent score prevents unchanged-source and indiscriminate-regeneration shortcuts from achieving high scores.Source: §8.1 · p. 33

8.2 Evaluation Tools and Reliability: How Can Each Property Be Measured Accurately?

  • Route each property to the simplest reliable tool: signal and embedding metrics for global measures, specialist models for localized evidence, and MLLM checklists for semantic requirements.Specialist evidence can ground semantic judgments.Source: §8.2 · p. 33
  • Reliable protocols use atomic observable criteria, jointly score competing requirements, control sampling and token budgets, calibrate judges, and report results by dimension and difficulty.Temporal metrics should change when temporal evidence is removed or shuffled; hidden prompts and periodic human review guard against benchmark overfitting.Source: §8.2 · p. 33

8.3 Operational Evaluation: How Should Evaluation Be Used in Practice?

  • Evaluation should retain criteria, sampled evidence, tool outputs, and judge rationales so failures can be traced to likely upstream sources.The operational levels range from hourly smoke tests to daily regression boards and release evaluations with full routed protocols and human calibration.Source: §8.3 · pp. 33–34

9 Conclusion

The conclusion presents Kling’s successive-generation experience as a view of video-generation quality shaped by multimodal understanding across the full pipeline. It summarizes the shared foundation model and downstream practices for controllable generation.

9 Conclusion

  • Kling-MLLM provides a shared perceptual foundation for downstream modules, while generation quality reflects understanding across data, conditioning, alignment, and evaluation.The report frames these observations as practical guidance for building the understanding stack behind controllable video generation.Source: §9 · p. 34
  • Captioning supplies a dense semantic contract, reference-grounded captioning makes bindings explicit, and prompt enhancement translates user intent into that contract.Agentic prompt enhancement refines the story and retrieves compelling examples.Source: §9 · p. 34
  • The conclusion pairs an explainable GRM with a fast DRM, emphasizes perceptual training for reliable text encoders, and recommends task-specific evaluation that exposes shortcuts and traces failures upstream.It also notes that text-only screening can filter candidate text encoders before end-to-end training.Source: §9 · p. 34

Related papers

Loading 2611.04916v1…