Source-linked AI summary
Quaternion Convolutional Neural Networks
Xuanyu Zhu, Yi Xu, Hongteng Xu, Changjian Chen
TL;DR
Color-image processing with real-valued CNNs can miss cross-channel structure and risk overfitting. The paper proposes quaternion CNNs that represent color pixels as quaternions and redesign core layers, reporting better classification and denoising performance than comparable real-valued CNNs, with limited benefit on low-color-variety images.
Problem
Real-valued CNNs sum channel outputs, potentially losing color structure and retaining excessive kernel degrees of freedom; addressing these challenges remains under-investigated.
Method
QCNN represents each color pixel as a quaternion and uses quaternion convolution and fully connected layers to model cross-channel color relationships with fewer parameter degrees of freedom.
Results
QCNN outperforms traditional real-valued CNNs in both color image classification and color image denoising.
Takeaways & Limitations
Quaternion operations preserve more color information and provide physically meaningful color-space transformations, supporting high- and low-level color vision tasks.
Takeaways & Limitations
For images with little color variety, QCNN becomes similar to a real-valued CNN and achieves comparable or slightly worse denoising results.
Abstract
from arXiv · showhide
Neural networks in the real domain have been studied for a long time and achieved promising results in many vision tasks for recent years. However, the extensions of the neural network models in other number fields and their potential applications are not fully-investigated yet. Focusing on color images, which can be naturally represented as quaternion matrices, we propose a quaternion convolutional neural network (QCNN) model to obtain more representative features. In particular, we redesign the basic modules like convolution layer and fully-connected layer in the quaternion domain, which can be used to establish fully-quaternion convolutional neural networks. Moreover, these modules are compatible with almost all deep learning techniques and can be plugged into traditional CNNs easily. We test our QCNN models in both color image classification and denoising tasks. Experimental results show that they outperform the real-valued CNNs with same structures.
1 Introduction
Traditional CNNs merge color channels by summation, which can discard inter-channel structure and increase overfitting risk. QCNNs represent color images with quaternion operations that preserve color structure, reduce parameter freedom, and improve results across classification and denoising.
- CNN convolution kernels merge input channels by summing their convolution outputs into one channel per kernel.
- Summing channels can ignore complicated inter-channel relationships, lose color structure, and leave convolution kernels vulnerable to overfitting.
- QCNN represents each color pixel as a quaternion and the image as a quaternion matrix rather than three independent real-valued matrices.
- Quaternion convolution applies both scaling and rotation in color space, providing a more structural representation of color information than real-valued convolution.
- QCNN imposes an implicit architectural regularizer that models cross-channel relationships while reducing parameter freedom during training.
- QCNN benefits both color image classification and denoising, while preserving more color information than real-valued CNNs.
2 Related works
Quaternion methods extend neural-network and image-processing techniques beyond real numbers, using hypercomplex operations suited to rotations and structured color representations. Prior work includes quaternion transforms and neural networks, while CNNs remain dominant across vision tasks.
- 2.1 Quaternion-based color image processing: Quaternions are hypercomplex numbers represented by one real and three imaginary components whose units obey specific multiplication rules.
- 2.1 Quaternion-based color image processing: Quaternion vectors support addition, scalar multiplication, and element multiplication, providing operations for quaternion-based models.
- 2.1 Quaternion-based color image processing: Quaternion operations can represent rotations of three-dimensional vectors, supporting physically meaningful transformations in color-related processing.
- 2.1 Quaternion-based color image processing: Quaternion-based harmonic-analysis methods include quaternion Fourier, wavelet, and Kalman transforms, with reported advantages for 3D objects over real-valued methods.
- 2.2 Real-valued CNNs and their extensions: CNNs have achieved strong results in classification, super-resolution, denoising, and inpainting, motivating their extension to additional number fields.
- 2.2 Real-valued CNNs and their extensions: Complex-valued and quaternion neural networks extend real-valued models, with complex CNNs especially suited to audio signals represented as complex numbers.
3 Proposed Quaternion CNNs
The proposed QCNN represents color images as quaternion matrices and redesigns convolutional, fully connected, pooling, and activation modules in the quaternion domain. These modules preserve cross-channel relationships, constrain transformations, and remain compatible with real-valued CNN components.
- Each color pixel is represented as a pure quaternion, so the image becomes a 2D quaternion matrix rather than three independent real-valued matrices.
- 3.1 Quaternion convolution layers: Quaternion convolution applies channel-coupled rotations and scalings, using a shared rotation axis with operation-specific angle and scaling parameters.
- 3.1 Quaternion convolution layers: Restricting convolution to rotations and scalings acts as an implicit regularizer and preserves inter-channel information without the input-layer order reduction of real-valued CNNs.
- 3.1 Quaternion convolution layers: For grayscale images, quaternion convolution reduces to the same function as real-valued convolution because color vectors are parallel to the gray axis.
- 3.2 Quaternion fully-connected layers: Quaternion convolution and fully connected layers can be stacked or reformulated as matrix multiplications, while outputs can connect to real-valued convolutional and fully connected layers.
- Quaternion pooling averages imaginary parts or selects elements by criteria such as magnitude, and quaternion activations apply channel-wise nonlinearities with color-space correction for invalid values.
4 Learning Quaternion CNNs
The paper trains quaternion CNNs using parameter initialization and backpropagation adapted to quaternion operations. Differentiability is maintained component-wise, while task-specific real-valued output layers can remain in hybrid networks.
- Initialization treats scaling as controlling transformed-vector magnitude and rotation angle as an additional parameter that rotates the input vector.
- 4.2 Backpropagation: Quaternion operations and their gradients can be represented through matrix multiplications, enabling chain-rule backpropagation for parameter updates.
- 4.2 Backpropagation: The backward process corresponds to a rotation with the same axis as the forward process but with a reverse angle.
- Functions differentiable with respect to each quaternion component satisfy the quaternion chain rule and can serve as loss or activation functions in fully-quaternion CNNs.
5 Experiments
Experiments evaluate QCNNs against real-valued CNNs on color-image classification and denoising. QCNNs generally converge faster and achieve better results, with advantages strongest for colorful images and limited on near-grayscale images.
- Color image classification: QCNN classification models consistently converged faster, reached smaller training loss, and achieved higher test accuracy than real-valued CNNs.The advantage remained even when QCNNs used fewer parameters than same-size real-valued CNNs.
- Color image denoising: QCNN denoising models consistently achieved higher PSNR values than real-valued CNNs on both evaluated datasets.Their loss converged faster and to a smaller value, while test-image PSNR became higher after 100 epochs.
- Color image denoising: QCNN denoising preserved more detailed image structures, including the pattern on a plate, than the real-valued CNN.The paper attributes this to avoiding color-information loss and learning more texture features in lower layers.
- Advantages and limitations: QCNN advantages were strongest for colorful images, whereas performance was comparable or slightly worse for images with little color variation.Mean saturation and average angle from the grayscale axis were both positively correlated with the PSNR difference.
6 Conclusions and Future Work
The paper concludes that QCNNs improve color image classification and denoising over traditional real-valued CNNs while offering quaternion-based layers compatible with existing networks. Future work targets more efficient training and extensions to larger-scale data and applications.
- QCNNs achieve better performance than traditional real-valued CNNs on both color image classification and color image denoising.
- The proposed quaternion convolution operation represents color information structurally, while quaternion-based layers remain compatible with existing real-valued networks.
- Future Work: Future work will investigate more efficient QCNN learning algorithms, including reducing backpropagation computation through reverse rotations of color vectors.
- Future Work: The authors also plan to extend QCNNs to large-scale data and additional applications.