Source-linked AI summary
Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding
Ming Wang, Yuqing Zhang, Tingna Xie, Xiangju Li, Xiaocui Yang, Daling Wang, Shi Feng, Yifei Zhang
TL;DR
Creative understanding in MLLMs remains difficult to evaluate, especially when meaning depends on non-obvious conceptual relations. C4 addresses this gap with a chengyu-based cross-concept decoding benchmark, finding that the strongest model achieves only 50.7% primary accuracy.
Problem
Creative capabilities remain difficult to evaluate systematically because creative tasks lack explicit targets and direct reward signals common in accuracy-oriented benchmarks.
Method
C4 evaluates cross-concept decoding through a reviewed chengyu network with structured encoding, explicit bridge paths, difficulty levels, and exact answers.
Results
50.7% primary accuracy is the strongest model score, while candidate constraints add 17.3–56.0 points over open recovery and open models lag behind closed models.
Takeaways & Limitations
Current MLLMs remain far from solving cross-concept decoding, with open answer-space search identified as the dominant bottleneck.
Takeaways & Limitations
The benchmark assumes reviewed annotations license a recoverable path to each target, and those annotations are unavailable to models except through general bridge hints.
Abstract
from arXiv · showhide
Creative capabilities of MLLMs matter in design, communication, education, and human--AI collaboration, yet remain difficult to evaluate because explicit targets and reward signals are scarce compared with accuracy-oriented tasks. Cross-concept understanding is a core cognitive capacity underlying receptive creativity. It enables a perceiver to recover intended meaning from non-obvious but meaningful conceptual relations. We operationalize item construction as cross-concept encoding and model inference as cross-concept decoding. We introduce C4, a cognition-inspired evaluation framework for Chengyu (Chinese idiom)-based Cross-Concept Creativity. Its encoding component maps target slots to imageable substitute concepts along bridge paths in a manually annotated and third-party-reviewed cross-concept network, enabling batch generation with explicit structure, difficulty indexed by bridge count and depth, and exact answers. Using this framework, we instantiate the C4 Evaluation Set (C4-Eval), comprising 184 synthetic items and 37 human-created cross-concept chengyu figures collected from online sources. We manually construct and review cross-concept relations, bridge paths, and reasoning processes for the collected figures. Each C4-Eval item is instantiated in five task settings, yielding 884 primary answer-recovery cases. Across ten evaluated MLLMs, the strongest closed models reach 50.7% and 48.0% primary accuracy, while open-source models remain substantially lower. Candidate constraints improve accuracy sharply, but bridge hints and explanation requests provide only modest gains. These results expose a substantial gap in how current MLLMs decode creatively encoded meaning through cross-concept relations. The code is in the supplementary material.
1 Introduction
The introduction frames cross-concept understanding as a core capacity for receptive creativity and formalizes it as encoding and decoding through meaningful associative paths. It presents C4 and C4-Eval as a structured chengyu-based benchmark, showing that current MLLMs remain far from saturated on creative meaning recovery.
- Motivation: Creative MLLM capabilities matter across design, education, communication, and human–AI collaboration, but their evaluation is difficult because creative tasks lack explicit targets and direct reward signals.Unlike accuracy-oriented tasks, creativity may involve novelty, appropriateness, and multiple valid interpretations.
- Conceptual framing: Cross-concept understanding enables observers to recover intended meaning from displaced but meaningfully related cues, supporting receptive creative interpretation.The paper illustrates this with clues linked to an apple through a well-known anecdote.
- Conceptual framing: C4 defines cross-concept encoding as projecting a source concept through associative paths into imageable cues and decoding as reversing those relations to recover the source.The formulation separates the communicator’s substitution process from the observer’s recognition process.
- Benchmark: C4-Eval contains 184 synthetic items and 37 web-collected figures across 84 targets and 884 primary cases, with difficulty levels indexed by bridge count and bridge depth.The benchmark uses a manually annotated and reviewed chengyu-oriented network and four answer-bearing primary settings, T1 through T4.
- Findings: 50.7% primary accuracy is achieved by the strongest model, indicating that the task remains far from saturated.Candidate constraints add 17.3–56.0 points over open recovery, while open models lag behind the strongest closed models under the stated evaluation protocol.
2 Related Work
Prior multimodal and creativity benchmarks cover image-grounded reasoning, compositionality, lateral thinking, and generative creativity, but C4 evaluates decoding of conventional creative expressions linked indirectly to images through annotated bridge paths. Its task views and fixed targets enable exact scoring and analysis by source and difficulty.
- Multimodal benchmarks: Multimodal benchmarks span image-conditioned question answering, compositional reasoning, external knowledge, grounded language, and image-text compositionality.VQA/VQAv2 emphasize image-grounded answering; CLEVR/GQA probe compositional structure; MME, MM-Vet, MMMU, and CMMMU broaden perception, cognition, and domain reasoning.
- C4’s distinction: C4 targets conventional creative expressions linked to images indirectly through manually annotated bridge paths rather than standard image-grounded benchmark tasks.This formulation supports cross-concept evaluation through explicit conceptual links between expressions and images.
- Evaluation design: C4 separates open recovery, candidate recognition, and explanation, while fixed targets and explicit paths support exact scoring and analysis by source and difficulty.These task views provide complementary ways to assess cross-concept understanding.
- Cognitive foundations: Cognitive accounts connect creativity with novelty and appropriateness, retrieval, recombination, imagery, exploration, remote conceptual connections, analogical alignment, and conceptual integration.These perspectives motivate framing creative understanding as interpreting new but meaningful conceptual structures.
- Creativity benchmarks: Creativity benchmarks assess divergent generation, lateral problem solving, multimodal creation, literary outputs, visual humor, and image-conditioned creative responses.Related studies also examine causal interventions and human–AI effects on novelty and diversity, using diversity proxies or task success.
3 Benchmark Design
C4’s benchmark design uses a reviewed, reusable cross-concept network to encode chengyu into solvable image clues with controlled synthetic difficulty. C4-Eval combines synthetic and human-created figures, then tests decoding under five levels of answer and explanation support.
- Cross-concept network: The reusable network contains 47 target chengyu, 168 anchored slots, and 758 deduplicated bridges.It is manually annotated, reviewed, provenance-preserving, and reused across synthetic item generation.
- Synthetic encoding: The encoder selects target slots, follows bridge paths to imageable substitute concepts, composes substituted phrases, and renders them as visual clues.Each item retains a known latent target and annotated bridge structure for reverse decoding.
- Difficulty control: Synthetic difficulty spans four levels defined by target-slot count and bridge depth, from one slot and one step at Level 1 to two slots with paths of at least two steps at Level 4.Levels 2 and 3 combine two non-overlapping slots, with Level 3 mixing one-step and deeper paths.
- Evaluation set: C4-Eval combines synthetic items with web-collected human-created figures whose cross-concept relations, bridge paths, and reasoning processes are manually reconstructed and reviewed.Both sources use the same annotation-oriented representation, while only synthetic items receive L1–L4 labels.
- Task settings: Each item is evaluated in five task settings that vary image-only decoding, bridge hints, candidate constraints, explanation requests, and explanation-focused answer search.T1 probes unconstrained recovery, T3 collapses the candidate space, and T4–T5 separate answer recovery from explanation quality and formatting.
4 Experiments
Experiments show that current MLLMs remain far from solving cross-concept chengyu decoding, with answer-space search and multi-slot encoding creating major difficulties. Models can often recognize visual evidence and recover answers, yet their explanations frequently fail to reconstruct the intended bridge paths.
- RQ1: 50.7% top accuracy sits far above the best open score of 18.1%, showing that cross-concept decoding remains far from solved.Even the strongest model misses roughly half of the primary cases.
- RQ2: Candidate constraints produce the largest accuracy gains, indicating that open answer-space search is the central decoding bottleneck.Bridge hints yield much smaller gains, while T4 remains near other open settings rather than candidate-level performance.
- RQ2: Roughly half to the majority of closed models’ T3 successes are missed by all three open settings, rising to at least three quarters for most open models.This candidate-only recovery pattern directly quantifies the search bottleneck.
- RQ3: The L1-to-L2 slot transition causes roughly three times the mean accuracy drop of the L2-to-L3 depth transition, making slot count the dominant difficulty factor.Additional depth beyond the first multi-step path produces no further measurable difficulty, though comparisons are associative rather than controlled causal effects.
- RQ4: GPT-5.5’s known-answer explanation matches recognized visual cues but diverges at the bridge layer, demonstrating fluent yet unfaithful path reconstruction.The correct T4 answer shows that the image contains sufficient evidence for target recovery and intended paths.
- RQ4: 83 of 221 items are unsolved by every model in open settings, while 6 remain unsolved even with candidates, concentrated in higher synthetic levels.A representative item shows models selecting a distractor despite reconstructing both encoded paths once the answer is provided.
5 Conclusion
C4 makes receptive creative understanding measurable through cross-concept encoding and decoding. Its C4-Eval instantiation combines synthetic and human-created chengyu figures under a shared annotation schema, while results across ten MLLMs show that even the strongest model recovers only about half of primary cases.
- C4 makes receptive creative understanding measurable through cross-concept encoding and decoding.
- C4-Eval combines synthetic items with human-created figures collected from online sources under a shared annotation schema.
- C4 integrates structured item construction, task instantiation, answer extraction, exact scoring, and stratified analysis around a manually annotated chengyu-oriented cross-concept network.
- Across ten MLLMs, even the strongest model recovers only about half of the primary cases.
A Theoretical Scope and Operational Assumptions
C4 frames receptive creative understanding as cross-concept connection, formalized through encoding and decoding over a structured network. Its evaluation assumes fixed conventional targets, nonliteral visible concepts, reviewed bridge paths, and exact recovery of intended answers while allowing alternative interpretations.
- Theoretical contribution: C4 connects cognitive theories of creative interpretation to encoding and decoding over target slots, bridge paths, landing concepts, and recoverable answers.The cross-concept network makes the cognition-grounded formulation operational for MLLM evaluation.
- Operational assumptions: The evaluation assumes fixed conventional targets, nonliteral visible concepts, reviewed bridge paths, and exact recovery of the intended target.Annotated paths specify intended construction but do not rule out other plausible interpretations of an image.
- Formal evaluation: C4 formalizes items, bridge paths, four synthetic levels, recoverability, five task views, and a primary score to test benchmark-level predictions.Candidate constraints isolate answer-space search; T1/T4 test prompt-form sensitivity, level comparisons separate slot count from bridge depth, and T5 supports visual-recognition and bridge-reconstruction inspection.
B Data and Resource Release
The publication release will provide the complete C4-Eval benchmark and its supporting resources, with code and data distributed under separate open licenses.
- Release contents: The release includes C4-Eval, reviewed cross-concept annotations, synthetic and collected images, task instances, model outputs, and benchmark-construction and evaluation code.The evaluation package will also include the collected figures.
- Licensing: Code is licensed under Apache-2.0, while data, annotations, prompts, and benchmark metadata use CC BY-NC 4.0.
B.1 Data Card · C Cross-Concept Network Annotation · D Synthetic Construction and Quality Control
The benchmark combines 221 chengyu-based items with structured annotations and five task views, while its synthetic subset is built through reviewed bridge chains, deterministic filtering, and manual image-quality control. Cross-concept annotations enforce aligned, recoverable, non-identical, imageable relations that define four difficulty levels.
- B.1 Data Card: 221 items cover 84 target chengyu, including 184 synthetic items and 37 human-created figures.The synthetic subset contains 47 L1, 47 L2, 46 L3, and 44 L4 items.
- B.1 Data Card: 1,105 evaluation cases arise from five task views per item, with 884 answer-bearing T1–T4 cases determining the primary score.The primary score excludes the remaining task-view cases because only T1–T4 are answer-bearing.
- B.1 Data Card: Each item stores its source, image, canonical answer, prompt, and accepted answers, with synthetic and collected items carrying additional reviewed construction metadata.The benchmark contains no personal attributes or private user records.
- C Cross-Concept Network Annotation: Two annotators independently selected contiguous character-span slots and constructed one- or multi-step chains from each slot to an imageable concept.Their annotation sets were retained separately during the first round.
- C Cross-Concept Network Annotation: A third reviewer adjudicated flagged slot and bridge disagreements, after which annotations were normalized, alternatives retained, and duplicate chains merged by target, slot, and node sequence.The second round checked lexical-unit boundaries and recoverable relations before adjudication.
- C Cross-Concept Network Annotation: Retained chains must align with the target span, avoid conceptual identity, support interpretable adjacent relations, and end in a concrete imageable concept.These criteria define the required validity conditions for cross-concept bridge paths.
- D Synthetic Construction and Quality Control: 184 synthetic items result from four bridge-depth levels: L1 uses one one-step chain, L2 two one-step chains, L3 combines one one-step and one deeper chain, and L4 uses two deeper chains.The build retains at most one item per target and level and removes unchanged substitutions, overlapping slots, malformed phrases, and duplicates from 188 possible target-level cells.
- D Synthetic Construction and Quality Control: 184 final images were manually reviewed to remove gold-idiom appearances, repeated substituted-phrase titles, and explicit task instructions before GPT Image 2 generation at 1024 by 1024 pixels.The release includes images and construction metadata, and selection, candidate construction, and task instantiation are deterministic once annotations are fixed.
E Task Instantiation and Scoring
The five task settings use Chinese prompts and task-specific answer requirements, with standardized scoring for predicted chengyu and separate treatment of target-supplied T5. T5 explanations are additionally evaluated against annotated visual cues, bridge chains, and target fragments.
- Task settings: All prompts are written in Chinese, while T3 distractors come from the 84-answer inventory and are deterministically ranked by shared Chinese characters with the gold target.Ties are resolved deterministically.
- Scoring: T1–T3 score one predicted chengyu after normalization, accepting only the canonical four-character target; T5 is excluded from primary accuracy because its prompt supplies the target.Normalization removes whitespace and common ASCII and Chinese punctuation.
- Implementation: GLM-4.1V-9B-Thinking is scored from its explicit final-answer payload, whereas Qwen3.5-9B is scored only when it produces a final answer; early termination remains incorrect.The publication release includes original responses and final scored predictions.
- Explanation analysis: T5 explanations are analyzed by comparing perceptual cues and cue-to-target links with annotated visual elements, bridge chains, and target fragments, using same-item T4 as an answer-hidden cross-check.The analysis proceeds in two stages.
F Experimental Configuration
The evaluation uses fixed task prompts, candidate sets, normalization, and scoring rules, with each reported result based on one complete pass over 1,105 task cases. API and local models follow distinct serving configurations, while local inference uses specified Linux, Python, CUDA, vLLM, and NVIDIA-accelerator settings.
- Evaluation protocol: 1,105 task cases comprise one complete pass for each official evaluation row.Each reported result is computed from one complete evaluation pass.
- Inference conditions: Temperature is set to zero when available, no output-token cap is applied, and API models use provider-managed serving contexts and documented default interaction modes.API hardware is managed by the corresponding provider.
- Inference conditions: Local inference runs on Linux with Python 3.11.11, CUDA 13.0.2, and vLLM 0.23.0, using one NVIDIA accelerator with 40 to 141 GB of GPU memory per process.The accelerator memory range is specified per local process.
- Evaluation protocol: Fixed prompts, candidate sets, normalization, and scoring rules are used across all models.These conditions are held constant for reported comparisons.
G Extended Results · G.1 Additional Construction Cases · H Chengyu Reference
The extended results specify how task-level and subgroup accuracies are computed, illustrate additional synthetic and collected construction cases, and provide a six-part Chengyu reference for C4-Eval. The supplementary material also documents exact case counts and reviewer-oriented reference conventions.
- G Extended Results: 221 cases comprise every task, while Primary accuracy sums correct counts from T1–T4 across 884 cases.T5 measures exact recovery of its supplied answer field and is excluded from Primary.
- G Extended Results: 148 primary cases come from 37 collected items; level totals are 188 for L1, 188 for L2, 184 for L3, and 176 for L4.Released item-level predictions permit recomputation of every reported percentage and paired count.
- G.1 Additional Construction Cases: The target zhˇı lù wéi mˇa demonstrates all four construction levels by varying slot count and bridge depth across two homophonic paths.L1 uses “give directions,” L2 adds “feed a horse,” L3 extends to a horse keeper, and L4 extends to a traffic officer.
- G.1 Additional Construction Cases: The collected máo sè dùn k¯ai case uses culturally grounded paths linking a blocked toilet, Newton, and an opening action to its two target fragments.Annotation separates recognition of the depicted entities and action from reconstructing the target-fragment links.
- H Chengyu Reference: Tables S6–S11 provide the complete C4-Eval Chengyu reference across six parts.Pinyin is italicized, and English text gives conventional meanings for reviewer orientation.