Source-linked AI summary
Mu-DisCoCat: A Variational Pipeline for Compositional Generalization on Quantum Processors
Mina Abbaszadeh, Matilda Karabina Moore, Raem Haq, Martha Lewis, Mehrnoosh Sadrzadeh
TL;DR
Mu-DisCoCat addresses the limited multimodal reach of compositional models for CoCoGen with a staged variational quantum approach that learns objects before relations. Simulations report stronger relational generalization than the evaluated CLIP baseline, while hardware executions retain positive correlation with simulated fidelities despite noise.
Problem
Multimodal methods for compositional concept generalization remain limited, while learning DisCoCat relation tensors on classical computers is costly.
Source: §I · p. 1Method
Mu-DisCoCat learns object representations from single-object image-caption pairs, freezes them, then optimizes relation parameters using variational quantum circuits.
Source: §I · p. 2Results
85.88% relational accuracy for MHE, while the classical CLIP baseline performs at 50%; hardware backends retain positive correlation with classical fidelities.
Source: §IV · p. 5, §V · p. 6Takeaways & Limitations
The authors present Mu-DisCoCat as a viable use case for executing CoCoGen on current quantum hardware.
Source: §VI · p. 8The quantum-model and CLIP comparison is not parameter-matched and does not establish computational or quantum advantage.
Source: §IV · p. 5
Abstract
from arXiv · showhide
Achieving compositional concept generalization (CoCoGen), the ability to understand novel situations by recombining learned primitives, remains a fundamental challenge in artificial intelligence. Compositional semantic models such as Compositional Distributional Semantics (DisCoCat) offer solutions by generalising vectors to tensors, but suffer from scaling bottlenecks when learning the tensors. Mapping DisCoCat onto Variational Quantum Circuits (VQCs) resolves this limitation for text, yet the methodology has not been expanded to multimodal situations such as the ones involved in CoCoGen. This paper introduces Mu-DisCoCat: a multimodal variational quantum learning framework for DisCoCat that achieves CoCoGen. The framework first learns stable object representations from single-object image-text pairs, then fixes these and uses them to learn the relations between them in multi-object situations. In classical simulations, the model used Uhlmann state fidelity to compute the overlap between the multimodal circuit representations and achieved higher relational OOD accuracy than the evaluated CLIP baseline. Its deployment was evaluated using the destructive SWAP test across noisy quantum emulators, including a range of IBM fake backends, IQM FakeAphrodite, and the IBM Marrakesh quantum processor. Despite real-world device noise, the hardware-executed models maintained a strong positive correlation with simulated fidelities, reliably distinguishing unseen similar and dissimilar pairs. Our work establishes a framework for executing CoCoGen on VQCs, demonstrating a viable use case for near-term quantum hardware.
I. INTRODUCTION
CoCoGen—the ability to understand novel situations by recombining learned concepts—remains an open AI challenge. Mu-DisCoCat adapts a multistage multimodal quantum approach to this challenge and evaluates it on noisy emulators and quantum hardware.
I. INTRODUCTION
- Humans achieve CoCoGen by recombining learned concepts, but achieving this capability in AI remains an open challenge.The introduction illustrates it with recognizing the danger of a yellow car after learning that cars crossing roads are dangerous.Source: §I · p. 1
- DisCoCat represents relations as tensors whose classical learning is costly, while multimodal CoCoGen extensions and prior VQC approaches had limited scope or suboptimal results.The cited multimodal examples focus on audio-text adjective–noun phrases or concepts seen during training; an earlier VQC approach jointly learned objects and relations in one stage.Source: §I · p. 1
- Earlier multimodal VQC work introduced a multistage strategy that learned object representations from single-object pairs, then froze them while optimizing relation parameters.The present work adapts that strategy for hardware execution.Source: §I · pp. 1–2
- Mu-DisCoCat adapts multistage training for near-term quantum hardware, using four-qubit image representations, discard instead of post-selection, and the destructive SWAP test for image-caption overlap.The framework was evaluated on IBM Aer, noisy IBM fake backends, the IBM Marrakesh processor, and the IQM FakeAphrodite emulator.Source: §I · p. 2
II. BACKGROUND
The background presents DisCoCat as a compositional semantic framework: grammatical types map to Hilbert spaces, and composition is represented through tensor operations. Variational circuits implement these representations.
II. BACKGROUND
- A pregroup is a partially ordered monoid with left and right adjoints; multiplication composes elements, and correctly ordered adjoint pairs cancel to the unit.Entities use generator types, while relations use multiplied adjoint types; for example, “left of” has type n^r · s · n^l.Source: §II · p. 2
- The pregroup-to-Hilbert-space mapping assigns finite-dimensional spaces to generators and maps products and adjoints to tensor products of spaces and their duals.The unit of multiplication maps to the field R.Source: §II · p. 2
- DisCoCat models semantic composition with tensor products and word interactions with tensor contraction, using inner products to define contractions.The paper refers readers to existing work for further details.Source: §II · p. 2
- The circuits compile Hilbert-space representations using an IQP ansatz for images and Sim4 for text, composing relational-caption word circuits by DisCoCat grammatical reduction.Table I compares Stage 1 Sim4 and IQP ansatz assignments on OOD validation and test splits, with results averaged over five seeds.Source: §II · p. 2, Table I · p. 3
A. Multistage Training
Mu-DisCoCat separates object learning from relational learning: it first learns object representations from single-object image-caption pairs, then fixes those parameters while learning relations. The section also describes image encodings and a CLIP-based comparison baseline.
A. Multistage Training
- Proof-of-concept images use five-qubit one-hot encoding for single objects and multi-hot encoding for two objects and their spatial relation.Each image type is represented by a binary vector mapped to a computational basis state.Source: A. Multistage Training · p. 3
- For real-world scalability tests, frozen 512-dimensional CLIP image embeddings use nine-qubit amplitude encoding or PCA-reduced angle encoding; Collage manually combines object circuits with entanglement.The CLIP text-projection baseline replaces the quantum text encoder with a CLIP text encoder while keeping the image encoder frozen; only text-side parameters are updated.Source: A. Multistage Training · p. 3
- Stage 1 learns shared grounded object representations from single-object image-caption pairs; Stage 2 freezes the learned shape parameters and optimizes relation parameters for image-caption alignment.The learned noun parameters transfer from single-object training to relational training.Source: A. Multistage Training · p. 3, Figure 1 · p. 3
B. From Simulation to Hardware
The section describes how caption–image similarity is calculated in simulation and estimated on hardware. It also outlines the circuit configurations used for single-object and relational alignment.
B. From Simulation to Hardware
- Because post-selection is prohibitively expensive on current hardware, the remaining caption qubits are discarded via a partial trace.The retained s-type output qubits represent sentence meaning.Source: B. From Simulation to Hardware · p. 3, Figure 3 · p. 4
- In simulation, caption–image similarity uses Uhlmann fidelity between a reduced caption density matrix and a pure image state.The pure-state image projector makes the fidelity Tr(ρCρI).Source: B. From Simulation to Hardware · p. 3
- Stage 1 uses 4-qubit Sim4 caption and IQP image circuits, requiring eight qubits together for the destructive SWAP test.The image circuit angle-encodes PCA-reduced CLIP image features.Source: B. From Simulation to Hardware · p. 3, Figure 2 · p. 4
- For Stage 2, a 20-qubit relational caption circuit supplies four retained output qubits, paired with four image qubits in a 24-qubit hardware circuit.The relational caption composes subject, relation, and object noun circuits according to DisCoCat grammar; the ancilla-free test uses Bell-basis measurements on corresponding output pairs.Source: B. From Simulation to Hardware · p. 4, Figure 3 · p. 4
IV. SIMULATION EVALUATION
Simulation on CoBi2 evaluates compositional generalization on unseen combinations of shapes and spatial relations. Multistage quantum models outperform the relational CLIP baseline, while a reduced 4-qubit model retains strong OOD accuracy for hardware validation.
IV. SIMULATION EVALUATION
- CoBi2 tests relational alignment on controlled synthetic images, with test combinations of shapes and relations unseen during training; results are averaged over five random seeds.Training observes the individual shapes and spatial relations, while validation and test use unseen combinations.Source: §IV · pp. 4–5
- The classical CLIP baseline performs at chance level (50%) on relational learning, while multistage quantum models score higher; MHE achieves 85.88% and Collage 72.32%.The multistage quantum models use a few hundred trainable parameters versus 151.5M for CLIP.Source: §IV · p. 5
- Figure 5 frames the encoding comparison around high within-class similarity and lower between-class similarity; the authors hypothesize that nonlinear angle rotations improve relational separability.The figure compares amplitude and angle encoding on test-dataset images.Source: Figure 5 · p. 5, §IV · p. 5
- The CLIP and quantum models are not parameter-matched, so their performance comparison does not establish computational or quantum advantage.Fewer trainable parameters do not necessarily mean lower current-hardware computational cost because circuit execution, sampling, and noise add overhead.Source: §IV · p. 5
- Multistage training outperforms corresponding non-multistage models while reducing trainable parameters from 792 to 480.The authors hypothesize that nonlinear angle-encoding rotations improve relational separability.Source: §IV · p. 5
- The reduced 4-qubit angle-encoding relational model achieves 72.69% OOD test accuracy with only 210 trainable parameters, providing a suitable basis for the hardware validation presented in Section V.Source: §IV · p. 5
V. HARDWARE VALIDATION
The hardware evaluation compares destructive SWAP test overlaps with simulated fidelities on held-out pairs. Similar and dissimilar pairs remain distinguishable across evaluated platforms, although agreement weakens for more noise-sensitive relational circuits.
V. HARDWARE VALIDATION
- Hardware validation uses angle encoding with PCA-reduced image representations, while noun parameters stay fixed and relation parameters are optimized during relational training.The 4-qubit angle-encoding model was used for hardware validation.Source: §V · pp. 5–6
- The destructive SWAP test estimates the Hilbert–Schmidt overlap from joint measurements, enabling hardware comparison with simulated fidelity without quantum state tomography.The hardware circuits were run after training on held-out benchmark pairs, and their overlaps were compared with simulated test fidelities.Source: §V · p. 6
- Across emulators and hardware, destructive-SWAP overlaps track simulated fidelities, with similar and dissimilar pairs remaining separated despite lower agreement on IBM Marrakesh.Relational agreement is excellent on IBM Aer and FakeMarrakesh emulators, while IQM FakeAphrodite shows lower overlaps and weaker correlation than IBM emulators; all backends retain positive correlation.Source: §V · p. 6
- For single-object validation, small mean absolute errors and modest centroid-distance reductions indicate that the learned similarity structure remains largely intact despite hardware imperfections.The reported strong agreement is attributed to the relatively shallow, compact 8-qubit destructive SWAP test circuit.Source: §V · p. 6
- Stage 2 uses a 24-qubit hardware circuit with average depth 365, compared with an 8-qubit circuit and depth 16 in Stage 1, increasing relational noise susceptibility.Increasing the number of shots improves agreement with classical fidelities; FakeMarrakesh's transpiled depth is 365 versus 346 for the other backend, with both using 24 active qubits.Source: §V · p. 6
VI. CONCLUSIONS AND FUTURE WORK
Mu-DisCoCat encodes compositional text and image meanings in variational quantum circuits and preserves learned similarity relationships on current quantum processors. The authors identify fixed visual representations as a shortcoming and point to image theory, larger datasets, and error mitigation as future work.
VI. CONCLUSIONS AND FUTURE WORK
- Mu-DisCoCat represents word and image meanings in variational quantum circuits and composes sentence meanings through tensor contraction encoded as Bell measurements.Image circuits encode image vectors directly or concatenate circuits for objects in an image; hardware implementation replaces post-selection with discard and state-overlap fidelity with the destructive SWAP test.Source: §VI · pp. 6, 8
- Despite reduced quantitative agreement with simulated fidelities under hardware noise, similar and dissimilar image-caption pairs remain distinguishable across evaluated platforms.The authors conclude that learned compositional similarity relationships can be preserved on current quantum processors.Source: §VI · p. 8
- The text component alone was trained while visual representations remained fixed; future work includes image compositional theory, larger real-world datasets, and quantum error mitigation.The authors also propose evaluation on future fault-tolerant quantum devices.Source: §VI · p. 8