PaperlayerView exact revision on arXiv

Source-linked AI summary

Emotion Collider: Dual Hyperbolic Mirror Manifolds for Sentiment Recovery via Anti Emotion Reflection

Rong Fu, Ziming Wang, Shuo Yin, Kun Liu, Xianda Li, Simon Fong

arXiv:2602.16161v5cs.MMcs.CLcs.LG

TL;DR

Multimodal affect models must combine complementary inputs while addressing modality-specific reconstruction needs. EC-Net pairs hyperbolic representations with hypergraph fusion and modality-aware reconstruction; the paper reports strong benchmark performance and robustness to missing or corrupted inputs.

  • Problem

    Multimodal affect models must combine complementary signals despite heterogeneous modalities, while many reconstruction methods use a shared latent distribution that overlooks modality-specific global statistics.

    Source: §1 · pp. 1–2

  • Method

    EC-Net combines Poincaré-ball modality hierarchies, bidirectional hypergraph fusion, radial-angular contrastive objectives, and modality-aware reconstruction.

    Source: §1 · p. 2

  • Results

    EC-Net achieves state-of-the-art performance on standard multimodal sentiment benchmarks and consistent robustness to missing modalities, synthetic corruption, and moderate hyperparameter perturbations.

    Source: §4.13 · p. 16

  • Takeaways & Limitations

    The paper concludes that modality-aware reconstruction and fusion produce stable, semantically meaningful representations under realistic noise patterns.

    Source: §1 · p. 2

    Both Poincaré-ball curvatures are fixed hyperparameters and remain constant during training.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

Emotional expression underpins natural communication and effective human-computer interaction. We present Emotion Collider (EC-Net), a hyperbolic hypergraph framework for multimodal emotion and sentiment modeling. EC-Net represents modality hierarchies using Poincare-ball embeddings and performs fusion through a hypergraph mechanism that passes messages bidirectionally between nodes and hyperedges. To sharpen class separation, contrastive learning is formulated in hyperbolic space with decoupled radial and angular objectives. High-order semantic relations across time steps and modalities are preserved via adaptive hyperedge construction. Empirical results on standard multimodal emotion benchmarks show that EC-Net produces robust, semantically coherent representations and consistently improves accuracy, particularly when modalities are partially available or contaminated by noise. These findings indicate that explicit hierarchical geometry combined with hypergraph fusion is effective for resilient multimodal affect understanding.

1 Introduction

Multimodal affect modeling must combine complementary signals despite differences in representation, alignment, and noise, while existing approaches face limits in higher-order relations, hierarchical geometry, and missing-modality recovery. EC-Net combines hyperbolic modality representations, hypergraph fusion, and modality-aware reconstruction with contrastive objectives.

1 Introduction

  • Multimodal emotion inference draws on complementary text, audio, and visual cues, but their representations, alignment, and noise characteristics differ.Prior work reports more faithful inference from combined modalities than from a single source [3].Source: §1 · p. 1
  • Sequence models capture temporal context, whereas graph models represent interactions; conventional pairwise Euclidean graphs limit higher-order relations and hierarchical semantic structure [4] [7].Hypergraphs can connect multiple nodes, but existing approaches often treat modalities symmetrically and retain Euclidean representations.Source: §1 · p. 1
  • Missing or corrupted modalities challenge models that assume complete inputs, motivating reconstruction, distillation, and modality-aware training strategies.Prior solutions often focus only on inference or use a shared latent distribution, overlooking modality-specific global statistics and reducing reconstruction fidelity [8, 9, 10].Source: §1 · pp. 1–2
  • EC-Net combines modality-specific hierarchical embeddings, bidirectional hypergraph aggregation, and property-aware reconstruction with hyperbolic contrastive objectives for incomplete or noisy views.Its contributions also specify radial and angular contrastive components and adaptive hyperedges across modalities and time steps.Source: §1 · p. 2
  • EC-Net’s four contributions are Poincaré-ball embeddings, adaptive bidirectional hypergraph fusion, radial-angular contrastive learning, and modality-aware reconstruction and fusion.The stated aim is stable, semantically meaningful representations under realistic noise patterns.Source: §1 · p. 2

2 Related Work

Related work motivates EC-Net through research on hyperbolic affect representations, missing-modality learning, and contrastive methods, alongside geometric and robustness studies. The paper positions its approach at the intersection of these streams.

2.1 Hyperbolic representations for emotion and sentiment

  • Hyperbolic emotion models report gains over Euclidean architectures, while hierarchical encoders and multimodal fusion methods use geometry to capture affective structure [11] [7].These lines of work motivate geometric priors for multimodal affective representations.Source: §2.1 · p. 2

2.2 Multimodal fusion and learning with missing modalities

  • Missing-modality research spans reconstruction, imputation, partial-observability architectures, and training with complete and incomplete examples [15] [8] [16].Prior work also uses graph and hypergraph representations for inter-modal relations.Source: §2.2 · p. 2

2.3 Contrastive and representation learning in multimodal affective tasks

  • Intra- and inter-modal contrastive objectives support modality-aware, semantically consistent embeddings, with prototype and curriculum schemes refining separation between affective labels [3] [7].The cited work connects these techniques to reducing modality gaps and improving robustness in low-data regimes.Source: §2.3 · p. 2

2.4 Geometric and manifold-based methods

  • Manifold distributions, diffusion, geometry-adapted attention, and theoretical studies extend hyperbolic learning toward uncertainty, generative modeling, scale, and generalization [19] [20] [21] [22] [23] [24].Figure 1 depicts EC-Net’s paired Poincaré-ball manifolds and mirror-space recovery pipeline.Source: §2.4 · p. 2, Figure 1 · p. 3

2.5 Robustness, corruption and deception-related cues

  • Robustness benchmarks probe degradation under noise, occlusion, or adversarial conditions, while deception research uses nonverbal cues and cross-modal inconsistencies [25] [8].The paper relates geometric asymmetry to auxiliary detection objectives.Source: §2.5 · p. 3

2.6 Theoretical advances and disentanglement

  • Work on disentanglement, task editing, and orthogonal parameterizations motivates structural constraints for modularity, transferability, and reduced task interference [27] [29].These principles inform EC-Net’s factor decomposition and controlled component interactions.Source: §2.6 · p. 3

2.7 EC-Net position

  • EC-Net combines hyperbolic priors, missing-modality strategies, and contrastive or orthogonality-driven learning with a mirror-space involution and learned property pathway.The paper frames this combination as addressing practical robustness requirements documented in prior studies.Source: §2.7 · p. 3

3 Methodology

EC-Net pairs emotion and anti-emotion Poincaré-manifold embeddings with a learnable mirror mapping and recovery pathway. Its method combines geometric safeguards, property decomposition, hypergraph fusion, and normalized training objectives.

3.1 Notation and manifold assumptions

  • Curvature-ratio tests found gradient L2-norm dynamic range within a factor of 3 when cE/cA was at most 2, motivating a practical ratio range of [0.5, 2].The emotion and anti-emotion manifolds are modeled as fixed-curvature Poincaré balls.Source: §3.1 · p. 4

3.2 Numerical safeguards for exponential map and clipping statistics

  • With εbnd = 0.05, median radial clipping was 0.08% and the 95% upper bound was 0.15% across three seeds.Radial clipping is applied to prevent numerical instability near the Poincaré-ball boundary.Source: §3.2 · p. 4

3.3 Paired hyperbolic embeddings

  • Modality features are projected into tangent spaces and mapped to both emotion and anti-emotion manifolds using trainable linear projections.The projections use fixed manifold curvatures; their outputs are the modality-specific manifold embeddings.Source: §3.3 · p. 4

3.4 Differentiable Mirror Layer implemented as a learnable involution

  • The Differentiable Mirror Layer uses maps in both directions between manifolds and regularizes them for approximate involutive behavior.Riemannian importance re-weighting lowered cycle loss from 0.241 ± 0.007 to 0.204 ± 0.005; the paired t-test reported p < 0.001 over three runs.Source: §3.4 · p. 5

3.5 Emotion vector field and implicit mirror-space score matching

  • EC-Net models increasing sentiment as a local vector field and applies a curl penalty when its empirical curl proxy exceeds 0.01.After the penalty, final curl was (2.1 ± 0.3) × 10^-3 across three seeds; implicit mirror-space score matching avoids explicit Jacobian determinants of gϕ and supports reverse-diffusion recovery.Source: §3.5 · p. 5

3.6 Property embedding, orthogonal decomposition and EMA specifics

  • A shared property embedding and decomposition network separate sample-specific from sample-invariant components, with alignment and orthogonality constraints.Property embeddings are updated by exponential moving average every 100 steps; the reported principal-angle distribution has mean 3.8° and maximum 8.2°.Source: §3.6 · p. 5

3.7 Fusion, prediction and asymmetry cue with baseline comparisons

  • A permutation-invariant fusion network combines available manifold embeddings and recovered vectors, representing missing modalities with a learnable mask token.The asymmetry score correlated with human deception labels at Spearman ρ = 0.42 (p < 0.001, N = 2,560), compared with approximately 0.00 for random and 0.18 for logistic regression.Source: §3.7 · p. 6

3.8 Full optimization objective and dynamic loss normalization; 3.9 Training procedure, mask schedule, gradient clipping, hyperparameter search and implementation constants

  • The weighted training objective normalizes losses by running standard deviations, while training uses random modality masking and anneals the minimum mask rate from 0.5 to 0.1 every 10 epochs.The per-loss statistics use an exponential moving average updated every 50 batches; gradients are clipped to global L2 norm 1.0.Source: §3.8 · p. 6, §3.9 · p. 6

4 Experiments

Experiments evaluate EC-Net on CMU-MOSI, CMU-MOSEI, and IEMOCAP, reporting results across full-modality, missing-modality, corruption, and other robustness settings. The paper reports state-of-the-art benchmark performance and consistent robustness, alongside analyses of component contributions, geometric behavior, and computational overhead.

4.4 Main results with full modalities

  • EC-Net attains the strongest aggregated-metric performance on the full-modality CMU-MOSI, CMU-MOSEI, and IEMOCAP benchmarks.Tables 1 and 2 compare multiple reference methods; results are means and standard deviations over three runs.Source: §4.4 · p. 7, Table 1 · p. 9, Table 2 · p. 11

4.7 Ablation study

  • Component-removal experiments measure Acc2, F1, and Acc7 degradation in both fixed-missing and missing-rate regimes.The ablation analysis identifies the components contributing most to end performance.Source: §4.7 · p. 13, Table 5 · p. 14

4.5 Robustness to fixed missing patterns

  • EC-Net retains a consistent advantage across evaluated fixed modality-availability patterns and maintains high accuracy as the global missing rate increases.The fixed-pattern comparison reports Acc2, F1, and Acc7; the varying-rate analysis reports Acc2.Source: §4.5 · p. 13, §4.6 · p. 13, Table 3 · p. 14

4.8 Robustness to synthetic corruption

  • Under controlled corruption of 10% of test samples, EC-Net shows only minor performance degradation.Perturbations include visual blur and salt-and-pepper noise, audio background noise, and text misspellings or token reordering.Source: §4.8 · p. 15, Table 6 · p. 15

4.9 Hyperparameter sensitivity and stress tests

  • Sensitivity scans place the default configuration within a broad plateau, while extreme curvature-ratio and orthogonality-penalty tests expose failure modes.The visualizations also report stable loss trajectories, near-orthogonality, and low-variance convergence.Source: §4.3 · p. 7, §4.9 · p. 15, Table 7 · p. 15

4.10 Asymmetry cue and downstream correlation

  • EC-Net’s asymmetry cue shows stronger correlation with human deception labels than simple baselines, and high scores align with cross-modal discordance in inspected samples.Among the 100 highest-scoring utterances, 73 were sarcastic, 21 had blended emotional content with low human-rating agreement, and 6 were annotation artefacts.Source: §4.10 · p. 15, Table 9 · p. 16, §4.12 · p. 16

4.11 Complexity Analysis

  • The dual-hyperbolic architecture introduces only modest overhead versus similar-capacity Euclidean baselines while delivering higher accuracy under full- and partial-modality conditions.The complexity profile includes parameter count, FLOPs, throughput, memory footprint, and training time.Source: §4.11 · p. 16, Table 10 · p. 16

4.13 Summary

  • EC-Net achieves state-of-the-art benchmark performance and consistent robustness to missing modalities, synthetic corruption, and moderate hyperparameter perturbations.Ablations attribute most gains to the property pathway and reconstruction module; stress tests identify failure modes that informed training safeguards.Source: §4.13 · p. 16

5 Conclusion

Emotion Collider (EC-Net) combines Poincaré-ball modality embeddings with hypergraph fusion to model hierarchical and high-order cross-modal relations. Its contrastive and reconstruction components address semantic discrimination and incomplete or noisy inputs, and experiments report stable representations and improved robustness.

5 Conclusion

  • EC-Net combines modality-specific Poincaré-ball hierarchies with adaptive hyperedges and bidirectional aggregation to model high-order temporal and cross-modal interactions.The framework integrates hyperbolic embeddings and hypergraph-based fusion for multimodal emotion and sentiment analysis.Source: §5 · p. 16
  • A contrastive scheme using radial and angular embedding components is intended to enhance semantic consistency and discriminative power.Source: §5 · p. 16
  • Modality-aware reconstruction is incorporated into fusion to preserve sample-specific information and mitigate information loss from incomplete or noisy channels.Source: §5 · p. 16
  • Experiments report more stable, interpretable representations and improved robustness under modality degradation; future work includes larger multilingual datasets and adaptive curvature learning.Source: §5 · p. 16

A.1 Radial scaling is an inter-curvature diffeomorphism

The proposition establishes a smooth bijection between Poincaré balls with different curvature parameters. Its radial map preserves directions, and its smooth inverse is obtained by swapping the curvatures.

A.1 Radial scaling is an inter-curvature diffeomorphism

  • The radial map Φ is a diffeomorphism between the two Poincaré balls, with an inverse obtained by swapping c1 and c2.The result applies for positive curvature parameters c1 and c2.Source: §A.1 · p. 20
  • The scalar radius map is smooth and strictly increasing, yielding a smooth bijection between the balls’ radial intervals.Its inverse is the corresponding scalar map with c1 and c2 exchanged.Source: §A.1 · p. 20
  • For nonzero x, Φ changes its radius while preserving its direction; defining Φ(0)=0 gives smoothness at the origin.The proof uses the finite limit of R(r)/r as r approaches zero.Source: §A.1 · p. 20

A.2 Volume correction weight cancels Poincaré volume inflation

The section derives an importance weight from the reciprocal of the Poincaré volume density. In normalized expectations, this weight cancels geometric volume distortion up to a global constant.

A.2 Volume correction weight cancels Poincaré volume inflation

  • The volume element is expressed in Euclidean coordinates using dx, with n denoting the ambient dimension.The proposition concerns a Poincaré ball with curvature parameter c > 0.Source: §A.2 · p. 20
  • The weight w(x) ∝ (1 − c∥x∥2)^n is proportional to the reciprocal Poincaré density factor and cancels volume distortion in normalized expectations.The proportionality ignores a global constant that cancels under normalization.Source: §A.2 · p. 20

A.3 Tangent-space mirror-layer parameterization is well defined

A bounded residual mapping is sent from the tangent space at the origin through the Riemannian exponential map. Finite residual bounds keep its image strictly inside the Poincaré ball.

A.3 Tangent-space mirror-layer parameterization is well defined

  • For finite residual bound εbnd, the exponential image lies strictly inside the ball, with an explicit interior margin.The exponential-map image norm is strictly less than 1/√cE for finite tangent-vector norm.Source: §A.3 · p. 21
  • Clipping Rϕ to radius εbnd enforces the bound and guarantees the mirror-layer parameterization is numerically stable and well defined.The residual vectors belong to the tangent space at the origin, identified with Rn.Source: §A.3 · p. 21

A.4 Monotone association between hyperbolic residual and prediction-label disagreement

The section relates hyperbolic reconstruction residual magnitude to prediction-label disagreement using ordered residual strata. When disagreement proportions increase across strata, a Cochran-Armitage trend statistic has positive expectation.

A.4 Monotone association between hyperbolic residual and prediction-label disagreement

  • The analysis stratifies samples by nondecreasing reconstruction residual and counts agreement and disagreement outcomes within each stratum.The residual is defined using the Poincaré distance between the sample representation and its reconstruction.Source: §A.4 · p. 21
  • If disagreement proportions are nondecreasing across residual-ordered strata, the Cochran-Armitage trend statistic with nondecreasing scores has positive expectation.This provides formal evidence for an increasing association between residual magnitude and disagreement probability.Source: §A.4 · p. 21
  • A positive standardized trend statistic yields a small one-sided p-value for an increasing trend under standard regularity conditions.The expectation is strictly positive under a nontrivial deviation from the global disagreement mean.Source: §A.4 · p. 21

A.5 Rademacher complexity of the hyperbolic mirror-layer hypothesis class

The appendix analyzes the mirror-layer class's Rademacher complexity, gives a convergence rate for manifold-aware Adam, and derives a high-probability generalization bound under explicit geometric and smoothness conditions.

A.5 Rademacher complexity of the hyperbolic mirror-layer hypothesis class

  • With bounded residuals and γ < 1/√c, mirror-layer outputs stay within radius γ, enabling vector contraction and Poincaré-distance Lipschitzness to bound empirical Rademacher complexity.The distance's Lipschitz constant is bounded on the compact radius-γ region.Source: §A.5 · p. 22

A.6 Convergence rate of manifold-aware Adam

  • Under geodesic smoothness, a curvature lower bound, and bounded stochastic-gradient second moment, manifold-aware Adam has a stated convergence rate.Its curvature factor reflects metric scaling near the ball boundary, and the step-size schedule compensates for that effect.Source: §A.6 · pp. 22–23

A.7 Generalisation error bound explicit in sample size N

  • With γ < 1/√c, the theorem gives a high-probability uniform risk bound for every f, with terms defined by sample size N and dimension n.The bound holds with probability at least 1 − δ.Source: §A.7 · pp. 23, 25
Loading 2602.16161v5…