Source-linked AI summary
Hybrid Quantum-Classical NLP Classification with Compact Semantic Representations: An Experimental Analysis of Representation Compression
Ali Hassan, Zijia Zhao, Maha A. Metawei
TL;DR
The paper addresses how to compress high-dimensional sentence embeddings into representations compatible with near-term quantum classifiers without unacceptable classification loss. It evaluates a hybrid pipeline using dimensionality reduction, quantum encoding, a variational circuit, and classical readout. Supervised reduction preserves task-relevant information more efficiently than PCA, with five-dimensional LDA and NCA approaching or exceeding the full-dimensional classical baseline under leakage-free validation.
Problem
The paper asks how much semantic information can be compressed into quantum-compatible representations without making classification performance unacceptable.
Method
The study combines pretrained sentence embeddings, PCA, NCA, or LDA reduction, quantum feature encoding, a variational quantum classifier, and classical readout under leakage-free cross-validation.
Results
LDA reaches 85.3% accuracy and NCA 83.1% at five dimensions, matching or exceeding the 85.1% full 384-dimensional classical baseline.
Takeaways & Limitations
Supervised dimensionality reduction can preserve task-relevant information more effectively than variance-based compression in compact hybrid quantum-classical NLP representations.
Takeaways & Limitations
The framework still requires matched classical baselines on identical compressed representations to determine whether the quantum circuit contributes predictive value beyond dimensionality reduction.
Abstract
from arXiv · showhide
Large language and sentence-embedding models provide rich semantic representations, but their high dimensionality poses a challenge for near-term quantum machine learning (QML), where quantum circuits can process only a limited number of input features. We investigate a hybrid quantum-classical pipeline that transforms high-dimensional sentence embeddings into compact representations for variational quantum classification. The workflow combines a pretrained sentence-embedding model, dimensionality reduction, angle encoding, a variational quantum circuit (VQC), and a classical decision layer. We systematically compare principal component analysis (PCA), neighborhood components analysis (NCA), and linear discriminant analysis (LDA), covering both unsupervised and supervised dimensionality reduction. Using the TREC question-classification dataset, we study the relationship between representation dimensionality, information retention, qubit count, and classification performance. Preliminary PCA experiments reveal a strong information bottleneck: reducing 768-dimensional embeddings to 3, 4, 5, and 8 dimensions retains about 8.2%, 10.2%, 11.9%, and 16.4% of the variance, with corresponding classification accuracies of 50.3%, 51.2%, 57.9%, and 63.4%. In contrast, supervised reduction is substantially more efficient. LDA reaches 85.3% accuracy and NCA reaches 83.1% using only 5 dimensions, under a leakage-free cross-validation protocol, compared with 85.1% for a full 384-dimensional classical baseline. These results indicate that supervised dimensionality reduction can preserve task-relevant information far more effectively than variance-based compression, making compact representations a promising route toward practical hybrid quantum-classical NLP models.
1 Introduction
The paper asks how much semantic information can be compressed into quantum-compatible representations without unacceptable classification loss. It investigates this question with a hybrid pipeline that separates classical compression effects from quantum-model effects.
- High-dimensional sentence embeddings are difficult to integrate with near-term QML because quantum circuits accept limited directly encoded features.The available feature count depends on the encoding strategy and circuit architecture.
- The central question is how much semantic information can be compressed without making classification performance unacceptable.
- The proposed pipeline combines sentence embeddings, dimensionality reduction, quantum feature encoding, and a variational quantum classifier.The study explicitly separates classical representation compression from the quantum model.
2 Related Work
Prior QNLP work spans compositional circuit models and embedding-based compression, but both approaches face scalability and representation-selection challenges. This paper motivates supervised projection as a way to compress dense semantic embeddings while retaining joint task-relevant structure.
- Compositional and sequential models: DisCoCat derives sentence meaning through structured tensor contraction, while DisCoCirc extends compositional modeling to discourse-level concept interactions.
- Compositional and sequential models: Compositional QNLP connects word representations through grammatical structure but requires linguistic preprocessing and can scale poorly with sentence length.Circuit size and tensor complexity increase with grammatical structure, creating challenges for larger datasets and realistic applications.
- Dimensionality reduction: PCA is widely used to fit representations to small quantum circuits, but it optimizes global variance rather than class discriminability.This can discard directions useful for downstream classification.
- Dimensionality reduction: Supervised LDA and NCA use label information during projection, directly contrasting with PCA within the same QNLP pipeline.
- Feature selection versus projection: Coordinate-wise VQFS is poorly matched to dense language embeddings because semantic meaning is distributed across co-dependent coordinates.LDA and NCA instead project the embeddings into a lower-dimensional subspace while retaining joint feature correlations.
3 Methodology
The methodology converts sentences into dense embeddings, compresses them with PCA, NCA, or LDA, and encodes the reduced features into a fixed-width variational quantum circuit. Quantum measurements then feed a classical six-class decision layer.
- Embedding and reduction: Each sentence is converted into a dense vector by a frozen pretrained sentence-transformer model.The preliminary experiments use all-MiniLM-L6-v2 to produce 384-dimensional embeddings.
- Embedding and reduction: The study evaluates PCA, NCA, and LDA as dimensionality-reduction methods for bridging high-dimensional embeddings and limited qubit capacity.PCA is unsupervised, whereas NCA and LDA use class information.
- Quantum feature encoding: Reduced 5-dimensional features are scaled to [−π, π] and encoded into quantum states with single-qubit Ry rotations.The normalized feature value determines each rotation angle.
- Variational quantum circuit: The quantum core uses a two-layer hardware-efficient ansatz with parameterized Ry and Rz rotations on five qubits and ring-entangling CNOT gates.The two layers contain 20 trainable parameters in total.
- Measurement and readout: Measurements of five single-qubit and five adjacent two-qubit Pauli-Z expectations produce a 10-feature vector for classical readout.The decision layer performs six-class classification over the TREC categories.
- Measurement and readout: The fixed five-qubit width makes efficient classical semantic compression the primary bottleneck rather than sentence-length-dependent quantum hardware scaling.
4 Experimental Results
Experiments compare compact sentence representations across restaurant sentiment and TREC question classification, showing that supervised reduction can preserve classification performance more effectively than PCA under tight dimensional constraints.
- Small-Scale Experiments: Restaurant Dataset: 90.0% classification accuracy with 3 qubits on the 70-sentence restaurant dataset accompanied 58.5% retained semantic variance.Scaling to 8 qubits retained 84.3% variance and achieved 94.3% accuracy, with accuracy peaking at 95.7% with 4 qubits.
- Preliminary and Fine-Grained PCA Evaluation: 8.2% retained variance and 50.3% classification accuracy resulted from reducing the TREC representation to 3 PCA dimensions.Across larger PCA representations, variance continued increasing while accuracy plateaued around 20–50 dimensions.
- Supervised Reduction vs. Unsupervised Compression: 85.3% accuracy for LDA and 83.1% for NCA at 5 dimensions matched or exceeded the 85.1% full 384-dimensional classical baseline.Both supervised methods outperformed PCA at 5 dimensions (57.9%) and 20 dimensions (71.5%).
5 Discussion and Limitations
The discussion argues that supervised compression is more consequential than raw representation size because it preserves class-relevant structure for compact quantum interfaces. It also emphasizes matched classical benchmarking and leakage-free validation as necessary for interpreting hybrid-model performance.
- PCA improves performance as dimensions increase but discards task-relevant semantic directions because it optimizes variance rather than class discriminability.
- Explained variance is not a reliable proxy for downstream accuracy because high variance does not guarantee class separability.
- Matched classical baselines on identical compressed representations are required to determine whether performance gains arise from dimensionality reduction or quantum expressibility.
- Physical qubit scaling introduces noise, deeper circuits, and greater optimization difficulty, motivating compact supervised representations for near-term devices.
- Supervised LDA and NCA compress 384-dimensional transformer embeddings to 5 dimensions while maintaining parity with uncompressed classical baselines.
- Leakage-free cross-validation fits the classical projection head within training folds, supporting generalizable and reproducible transferred representations.
6 Conclusion and Future Work
The conclusion presents a hybrid quantum–classical NLP framework and reports that supervised reduction mitigates the information bottleneck of aggressive compression. It motivates matched classical–quantum benchmarking and identifies larger representations, nonlinear compression, and hardware deployment as future directions.
- Supervised LDA and NCA match a full-dimensional classical baseline using only five dimensions, substantially mitigating the compression bottleneck.
- The proposed framework combines pretrained sentence embeddings, dimensionality reduction, quantum feature encoding, and variational quantum classification.
- Evaluating quantum and classical models on identical compressed representations is necessary to determine whether quantum circuits add predictive value beyond classical reduction.
- Future research will examine 12- and 16-qubit representations, nonlinear autoencoder compression, and physical hardware deployment with error mitigation.