PaperlayerView exact revision on arXiv

Source-linked AI summary

Structural Graph Probing of Vision-Language Models

Haoyu He, Yue Zhuo, Yu Zheng, Qi R. Wang

arXiv:2603.27070v1cs.CV

TL;DR

VLM interpretability lacks a population-level account of how multimodal computation is organized across neurons. The paper models each layer with neuron-correlation graphs and probes their structure across modalities, depth, and interventions. It finds that topology carries recoverable behavioral signal and provides a meaningful intermediate scale for interpreting multimodal computation.

  • Problem

    How multimodal computation is organized within interacting neuron populations remains incompletely understood, limiting interpretability beyond salient inputs or components.

  • Method

    The paper constructs within-layer neuron-correlation graphs from hidden activations and analyzes their graph representations for behavioral predictability, multimodal structure, and intervention targets.

  • Results

    Graph-based probes improve over linear baselines on most model–dataset pairs, with CLEVR gains of 7.7% on LLaVA, 4.3% on Qwen2.5-VL, and 1.3% on InternVL3.

  • Takeaways & Limitations

    Neural topology offers an intermediate-scale interpretability perspective that is richer than local attribution while remaining tractable for comparing layers, modalities, and models.

Abstract

from arXiv · show

Vision-language models (VLMs) achieve strong multimodal performance, yet how computation is organized across populations of neurons remains poorly understood. In this work, we study VLMs through the lens of neural topology, representing each layer as a within-layer correlation graph derived from neuron-neuron co-activations. This view allows us to ask whether population-level structure is behaviorally meaningful, how it changes across modalities and depth, and whether it identifies causally influential internal components under intervention. We show that correlation topology carries recoverable behavioral signal; moreover, cross-modal structure progressively consolidates with depth around a compact set of recurrent hub neurons, whose targeted perturbation substantially alters model output. Neural topology thus emerges as a meaningful intermediate scale for VLM interpretability: richer than local attribution, more tractable than full circuit recovery, and empirically tied to multimodal behavior. Code is publicly available at https://github.com/he-h/vlm-graph-probing.

1. Introduction

The paper addresses how multimodal computation is organized across populations of neurons in VLMs, beyond local attribution signals. It proposes neural topology as a population-level lens for connecting internal correlation structure with behavior.

  • Motivation: Existing VLM analyses emphasize attention, saliency, patch attribution, and component inspection, but these approaches poorly characterize distributed population-level organization.Transformer-based VLM computation is distributed across interacting units rather than isolated pathways.
  • Approach: The paper constructs within-layer neuron correlation graphs from hidden activations recorded during image–question inference.These graphs provide a topology-centered representation of multimodal computation.
  • Approach: Neural topology is used to test behavioral predictability, track cross-modal organization across depth, and identify structurally important neurons through perturbation.The framework treats these analyses as complementary views of internal computational organization.
  • Implication: The results establish neuron-correlation organization as a meaningful source of information about VLM behavior.This reframes interpretability around computation organized across populations rather than isolated attribution effects.

2. Neural Topology

The paper represents each transformer layer as a weighted neuron-correlation graph derived from hidden responses across multimodal tokens. It compares joint, visual, and textual topologies and uses graph representations to analyze population structure.

  • 2.1. Neuron Correlation Topology: Each layer is represented as a complete weighted graph whose nodes are neurons and whose edge weights measure Pearson correlation between neuron activation profiles.The resulting graph captures within-layer co-activation structure rather than literal model wiring.
  • 2.1. Neuron Correlation Topology: Hidden activations are collected from a frozen VLM for an image and text prompt, with each neuron represented by its response across joint multimodal tokens.The downstream analysis uses the inferred correlation structure rather than activation values themselves.
  • 2.2. Vision, Text, and Multimodal Topology: At each layer, multimodal, visual, and textual graphs are constructed from the full hidden state or token-type-specific subsets within the same forward pass.Differences among these graphs reflect specialization to visual, textual, and joint contexts.
  • 2.3. Neural Topology Representation: A GCN processes graph structure with learnable one-hot neuron identity embeddings, producing representations centered on topology rather than raw hidden-state content.The graph input includes node identities and correlation structure, while the resulting representation reflects neuron organization within a layer.
  • 2.3. Neural Topology Representation: Complementary mean and max pooling convert node representations into a fixed-dimensional graph signature for comparing layers and studying behavior.Mean pooling captures overall correlation tendency, while max pooling preserves salient high-response structure.

3. Predictability

The section tests whether layerwise neuron-correlation topology predicts VLM behavior across reasoning, recognition, counting, and hallucination tasks. Graph probes generally outperform linear baselines, while predictive depth varies by architecture and remains robust under sparse graph construction.

  • Probing Performance: Graph-based probes outperform linear baselines on most model–dataset pairs, with the clearest gains on CLEVR counting and TDIUC.On CLEVR, GCN improvements over the linear baseline are 7.7% for LLaVA, 4.3% for Qwen2.5-VL, and 1.3% for InternVL3.
  • Probing Performance: 7.7%, 4.3%, and 1.3% are the CLEVR improvements of GCN probes over linear baselines for LLaVA, Qwen2.5-VL, and InternVL3, respectively.
  • Regression: Graph-based probes improve CLEVR object-counting regression across all three VLMs, reducing MSE and improving R2 and Pearson correlation.This extends the observed topology signal beyond discrete classification to finer-grained numerical estimation.
  • Sparsity and Graph Construction: Probe accuracy remains largely stable when retaining the top 1%–20% of correlations, and high predictive performance does not require dense connectivity.The strongest correlations already contain most of the predictive structural signal under this setup.
  • Layerwise Predictability: Qwen2.5-VL-3B peaks around layer 27, whereas LLaVA-1.5-7B and InternVL3-1B show flatter or gradually declining depth trends.Thus, the most behaviorally informative depth differs across architectures.
  • Hallucination Detection: Graph probes consistently outperform word2vec text-only baselines for hallucination classification across all three models.The result indicates structural information beyond simple lexical cues, while serving as evidence of informativeness rather than a competitive detector.

4. Multimodal Structure and Alignment

The paper examines multimodal organization through token-, neuron-, and graph-level correlation structure, finding depth-dependent coupling, persistent graph hubs, and partial cross-modal alignment.

  • Behavioral Signal: Graph-based probes outperform text-only baselines across all three models for hallucination classification, showing that correlation topology carries behavioral signal.The result indicates informativeness about whether responses are grounded or hallucinatory, rather than establishing a competitive hallucination-detection system.
  • Cross-Modal Graph Alignment: The analyses jointly characterize how visual and linguistic information become coupled, stabilized, and organized within VLM internal topology.The study combines token-level dynamics, neuron-level persistence, and graph-level alignment rather than treating these as isolated analyses.
  • Cross-Modal Correlation Dynamics: Token-level correlations reveal increasing vision–text and text–text coupling with depth, while vision–vision correlations remain comparatively flat.The authors interpret this pattern as consistent with stronger multimodal integration in later layers, while noting that the statistic is descriptive rather than mechanistic.
  • Structural Hub Stability: Graph-defined hubs recur more reliably across samples than activation-based and modality-specific alternatives, indicating more persistent structural centrality.This comparison separates full multimodal topology from unimodal structure and activation magnitude as possible sources of hub persistence.
  • Structural Hub Stability: Top 1% graph-derived hub stability peaks in intermediate layers, where cross-modal coupling is also broadly strongest.The authors interpret this as recurring population-level organization around compact structural loci, not as direct evidence of mediation.
  • Cross-Modal Graph Alignment: Graph alignment reaches 0.9598 GAUC for multimodal–multimodal matching, compared with 0.8188 for LLaVA text–image matching and 0.6803 against the LLaMA backbone.These results support partial correspondence between visual and linguistic pathways while preserving meaningful structural differences after multimodal finetuning.

5. Causal Intervention Analysis

Causal interventions test whether topology-defined neurons and edges identify behaviorally influential components. Across node and edge interventions, graph-based selections produce substantial performance changes, with effects depending on coordinated activity and model depth.

  • Ablation of Top Neurons: Degree-ranked neuron ablation causes the largest performance drop across TDIUC and CLEVR, outperforming activation-magnitude selection.InternVL3-1B is most sensitive around layer 11, whereas Qwen2.5-VL-3B is most sensitive at layer 0.
  • Edge-Level Intervention: Graph-defined edge interventions show that replacing an endpoint with its partner’s activation preserves performance better than random or opposite replacement.Opposite replacement is most destructive, especially for Qwen2.5-VL-3B on color and counting tasks; InternVL3-1B shows the same ordering but greater robustness.
  • Edge-Level Intervention: The behavioral importance of a strong graph edge depends on the sign and alignment of coordinated endpoint activity, not only on endpoint identity.Identical intervention causes little degradation and can slightly improve performance, whereas random replacement causes a moderate decline.
  • Perturbation of Hub Activations: Small positive or negative perturbations to topology-defined hub neurons substantially degrade performance while other activations remain fixed.The tested hubs are neuron 62 at layer 11 in InternVL3-1B and five neurons at layer 0 in Qwen2.5-VL-3B.
  • Perturbation of Hub Activations: Approximately symmetric sensitivity to hub amplification and suppression suggests that these hubs operate within a relatively narrow functional range.Across intervention types, topology-defined hubs consistently occupy behaviorally influential positions.

6. Related Work

Prior VLM interpretability methods emphasize local signals such as attention, saliency, gradients, and visualization, while related neural-network work studies representations, connectivity, modularity, and interventions. This paper addresses the missing population-level topology of VLM layers in relation to multimodal behavior.

  • Interpretability in Vision–Language Models: Existing VLM analyses rely on attention flow, saliency and gradient attribution, or interactive visualization to study difficult-to-interpret internal mechanisms.These approaches focus on interpreting multimodal systems through localized or inspectable signals.
  • Neural Topology and Representation Structure: Prior neural-network research examines representational similarity, connectivity patterns, emergent transformer modularity, and causal interventions on internal components.These strands provide related tools for studying representation structure and functional importance.
  • Neural Topology and Representation Structure: Unlike this work, the cited approaches do not examine population-level topology in VLM layers as a way to analyze multimodal behavior.The paper positions within-layer correlation structure as the missing link between neural organization and VLM behavior.

7. Discussion

The discussion argues that within-layer co-activation topology is a useful mesoscopic explanatory unit for VLMs. It links organized populations, persistent hubs, and coordinated relations to behavior while remaining more tractable than full circuit recovery.

  • 7. Discussion: Within-layer co-activation topology offers a population-level view of computation beyond isolated tokens, heads, or neurons.The paper compares this mesoscopic perspective to neuroscience’s emphasis on population dynamics.
  • 7. Discussion: Topology is positioned between local attribution and full circuit recovery, balancing behaviorally consequential structure with tractability across layers, modalities, and models.The graphs are not literal wiring diagrams, but they expose structured populations, persistent hubs, and coordinated relations.
  • 7. Discussion: The findings suggest that multimodal reasoning may be better understood as an emergent property of organized neural populations than as a sum of independently interpretable components.This is presented as a broader implication of the topology-centered interpretability perspective.

8. Additional Experimental Setup

The experiments cover three representative VLMs, seven multimodal benchmarks, and graph-probing procedures with specified datasets, splits, probes, and evaluation reporting. Additional setup also includes text-only hallucination baselines and precision–recall reporting.

  • Models: The study evaluates InternVL3-1B, Qwen2.5-VL Instruct-3B, and LLaVA-1.5-7B across different visual-encoder designs and VLM families.Layerwise hidden activations are extracted from each model’s multimodal transformer backbone.
  • Datasets: Predictability is evaluated on CLEVR, TDIUC, MMMU, MMMU-Pro, BLINK, EMMA, and MHaluBench.CLEVR uses 10,000 counting examples; TDIUC uses 4,634 sports-recognition examples; MHaluBench uses 2,110 validation examples balanced between hallucinating and non-hallucinating responses.
  • Training Details: Each dataset uses an 80% training and 20% test split, with linear and GCN probes trained using task-appropriate objectives and Adam optimization.The reported result is the best test performance across epochs, using a single NVIDIA L40S GPU.
  • Text-only Baselines for Hallucination: MHaluBench hallucination detection includes text-only baselines based on question–answer mean embeddings and prompt token counts.These baselines measure predictability from shallow textual information before graph-based multimodal representations are introduced.
  • Additional Reporting: Table 5 reports precision and recall for linear and graph-based probes across TDIUC, CLEVR, MMMU, MMMU-Pro, BLINK, and EMMA.The best result in each model–dataset pair is highlighted in the table.
Loading 2603.27070v1…