Source-linked AI summary
Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognition
Chao-Han Huck Yang, Jun Qi, Samuel Yen-Chi Chen, Pin-Yu Chen, Sabato Marco Siniscalchi, Xiaoli Ma, Chin-Hui Lee
TL;DR
The paper addresses privacy-preserving speech recognition by decentralizing feature extraction in a federated QCNN architecture. Quantum servers encode Mel-spectrogram features with randomized circuits before local RNN recognition, achieving 95.12% accuracy on the Google Speech Commands dataset.
Problem
Speech recognition needs architectures that address acoustic-data privacy and parameter leakage while supporting decentralized processing.
Method
The method extracts Mel-spectrograms on a quantum server, encodes them with a QCNN using randomized quantum circuits, and sends the features to a local RNN acoustic model.
Results
95.12% accuracy was achieved for word recognition on the Google Speech Commands dataset by the decentralized QCNN-ASR framework.
Takeaways & Limitations
The QCNN models provide competitive spoken-term recognition with stable performance while enabling interpretable acoustic feature analysis through quantum convolution.
Takeaways & Limitations
Current NISQ devices are not fault-tolerant, and stringent hardware constraints limit applications with large circuit depth, many qubits, or high input dimensions.
Abstract
from arXiv · showhide
We propose a novel decentralized feature extraction approach in federated learning to address privacy-preservation issues for speech recognition. It is built upon a quantum convolutional neural network (QCNN) composed of a quantum circuit encoder for feature extraction, and a recurrent neural network (RNN) based end-to-end acoustic model (AM). To enhance model parameter protection in a decentralized architecture, an input speech is first up-streamed to a quantum computing server to extract Mel-spectrogram, and the corresponding convolutional features are encoded using a quantum circuit algorithm with random parameters. The encoded features are then down-streamed to the local RNN model for the final recognition. The proposed decentralized framework takes advantage of the quantum learning progress to secure models and to avoid privacy leakage attacks. Testing on the Google Speech Commands Dataset, the proposed QCNN encoder attains a competitive accuracy of 95.12% in a decentralized model, which is better than the previous architectures using centralized RNN models with convolutional features. We also conduct an in-depth study of different quantum circuit encoder architectures to provide insights into designing QCNN-based feature extractors. Neural saliency analyses demonstrate a correlation between the proposed QCNN features, class activation maps, and input spectrograms. We provide an implementation for future studies.
1. INTRODUCTION
The paper proposes a decentralized QCNN-based acoustic-modeling architecture that uses quantum convolution on NISQ servers and local speech-recognition models within vertical federated learning. The framework reports 95.12% word-recognition accuracy.
- Motivation: Vertical federated learning separates feature extraction from the ASR inference engine to support data protection.The motivation is increasing concern about acoustic-data privacy and privacy-preservation regulations such as GDPR.
- Architecture: The architecture combines a quantum convolution layer on NISQ servers or cloud APIs with a local speech-recognition model.This design separates quantum feature extraction from local recognition in a vertical federated learning setting.
- Contribution: The proposed QCNN combines variational quantum-circuit learning with a deep neural network for decentralized acoustic modeling.The authors describe this as the first work, to their knowledge, combining quantum circuits and DNNs for ASR.
- Result: 95.12% accuracy was achieved on word recognition by the proposed QCNN-ASR framework.The result is reported for the decentralized framework evaluated on the Google Speech Commands dataset.
2. RELATED WORK
The related work motivates hybrid classical-quantum learning through parameter protection and feature representation, while emphasizing the constraints of current NISQ hardware and prior speech-processing approaches.
- Quantum learning: Hybrid classical-quantum learning uses classical input signals and quantum algorithms during feature learning.The paper identifies quantum circuit learning as an accessible and reproducible QML approach for signal processing.
- NISQ constraints: NISQ devices are not error-corrected or fault-tolerant, limiting applications involving large circuit depth, many qubits, or high input dimensions.Variational quantum circuits are presented as a way to potentially absorb device noise into learned circuit parameters.
- Prior speech processing: Earlier quantum speech-recognition work used simulated quantum backpropagation rather than qubits in a real-world quantum device.The paper characterizes that approach as difficult to scale to large-scale ASR with parameter protection.
- Federated learning: Federated speech-processing architectures are presented as a setting where encryption and randomized encoding are relevant quantum advantages.The related work connects these properties with distributed ASR and cloud-based federated architectures.
3. DESIGNING QUANTUM CONVOLUTIONAL NEURAL NETWORKS FOR SPEECH RECOGNITION
The proposed QCNN uses cloud-side Mel-spectrogram processing and quantum convolution to produce encoded features for a local RNN acoustic model. Its quantum convolution filters encode patches, apply a circuit, measure the output, and support randomized parameter protection.
- Speech Processing under Vertical Federated Learning: The framework divides speech processing between a local user and a cloud server or API in a federated architecture.Speech is transformed into Mel-spectrogram features on the cloud side before quantum processing, then encoded features return to the local acoustic model.
- Quantum Convolutional Layer: Quantum convolution filters consist of encoding, quantum-circuit, and decoding operations that transform input patches into output features.The measured circuit outputs form the features used by the local ASR acoustic model.
- Quantum Convolutional Layer: A 2D Mel-spectrogram is divided into 2 × 2 patches, which are encoded into quantum states, processed by a circuit, and measured.The operation sequence is encoding, circuit transformation, and measurement-based decoding.
- Random Quantum Circuit: The design uses a randomly generated quantum circuit per QCNN model to support parameter protection.The example circuit applies Rx, Ry, Rz, and CNOT gates to encoded quantum states.
- Local Acoustic Model: The local acoustic models include RNNAtt and an advanced RNNUAtt model with a U-Net encoder for quantum- or neural-convolution features.RNNAtt uses bidirectional LSTM layers and self-attention, while RNNUAtt adds multi-scale convolution layers.
4. EXPERIMENTS
Experiments evaluate decentralized QCNN speech recognition on Google Speech Commands, comparing convolutional encoders, kernel sizes, and acoustic models. The strongest reported model reaches 95.12±0.18% accuracy, while saliency analyses examine learned acoustic patterns.
- Experimental Setup: The evaluation uses ten spoken-command classes from Google Speech Commands, with 11,165 training and 6,500 testing examples under background white noise.Mel-scale spectrograms are extracted from input speech using Librosa.
- Encoded Acoustic Features: Visualizations indicate that 2×2 QCNN features better reflect Mel-spectrogram energy patterns across high- and low-frequency regions than 3×3 encoded features.The neural convolution representation is similar but has lower intensity in high-frequency regions.
- Spoken-Word Recognition: 95.12±0.18% accuracy is achieved by RNNUAtt with an additional 2×2 quantum convolution layer, exceeding the reproduced RNNAtt baseline at 94.21±0.30%.The compared quantum and neural convolution layers use the same 2×2 kernel size.
- QCNN Architectures: 2×2 QCNN kernels show better accuracy and lower deviations than the tested 1×1 and 3×3 configurations, with 1.21% and 1.47% relative improvements over DS-CNN and ResNet baselines.The study constrains designs to 1×1, 2×2, and 3×3 kernels corresponding to 1, 4, and 9 qubits.
- Saliency Study: CAM analysis shows richer, more correlated acoustic features for QCNN than for neural-convolution and baseline RNN models when recognizing the utterance “on.”The activated hidden neurons identify related low-frequency patterns.
5. CONCLUSION
The paper presents QCNN as a decentralized feature-extraction approach for vertical federated speech processing. It reports competitive spoken-term recognition while identifying continuous ASR and broader privacy measurement as future directions.
- Conclusion: The proposed QCNN approach decentralizes speech feature extraction in vertical federated learning to support model parameter protection and interpretable acoustic feature learning.The architecture uses quantum convolution for decentralized speech processing.
- Conclusion: QCNN models show competitive spoken-term recognition and stable performance from quantum machines against classical DNN acoustic models using the same convolutional kernel size.The conclusion frames this comparison within the proposed decentralized approach.
- Future Work: Future work includes incorporating QCNN into continuous ASR and deploying more statistical privacy measurements from other privacy perspectives.The paper states that decentralization fulfills some data protection requirements but does not exhaust privacy evaluation.