Source-linked AI summary
Label-Aware Hyperbolic Embeddings for Fine-grained Emotion Classification
Chih-Yao Chen, Tun-Min Hung, Yi-Li Hsu, Lun-Wei Ku
TL;DR
Fine-grained emotion classification must distinguish subtle, semantically similar labels, while common Euclidean fine-tuning may struggle with confusing pairs. HypEmo learns hierarchical label embeddings in hyperbolic space, projects RoBERTa text representations there, and uses text-label distance to weight cross-entropy. HypEmo achieves state-of-the-art weighted F1 on GE and ED and outperforms LCL with much higher efficiency.
Problem
Fine-grained emotion classification must distinguish subtle, semantically similar labels, while common Euclidean fine-tuning may struggle with confusing pairs.
Source: §1 · pp. 1–2Method
HypEmo learns hierarchical label embeddings in hyperbolic space, projects RoBERTa text representations there, and uses text-label distance to weight cross-entropy.
Source: §1 · p. 2, §3.4 · p. 4Results
HypEmo achieves state-of-the-art weighted F1 on GE and ED and outperforms LCL with much higher efficiency.
Source: §4.3 · p. 6, §5 · p. 9Takeaways & Limitations
HypEmo performs especially well when label structure is more complex or inter-class relationships are ambiguous.
Source: §1 · p. 2Evaluation is limited to English datasets, and with flat, independent labels the method may back off to a conventional classification model.
Source: Limitations · p. 9
Abstract
from arXiv · showhide
Fine-grained emotion classification (FEC) is a challenging task. Specifically, FEC needs to handle subtle nuance between labels, which can be complex and confusing. Most existing models only address text classification problem in the euclidean space, which we believe may not be the optimal solution as labels of close semantic (e.g., afraid and terrified) may not be differentiated in such space, which harms the performance. In this paper, we propose HypEmo, a novel framework that can integrate hyperbolic embeddings to improve the FEC task. First, we learn label embeddings in the hyperbolic space to better capture their hierarchical structure, and then our model projects contextualized representations to the hyperbolic space to compute the distance between samples and labels. Experimental results show that incorporating such distance to weight cross entropy loss substantially improves the performance with significantly higher efficiency. We evaluate our proposed model on two benchmark datasets and found 4.8% relative improvement compared to the previous state of the art with 43.2% fewer parameters and 76.9% less training time. Code is available at https: //github.com/dinobby/HypEmo.
1 Introduction
Fine-grained emotion classification must distinguish subtle, similar labels, a challenge beyond simply handling more classes. HypEmo combines hyperbolic label representations with Euclidean RoBERTa and reports stronger results on GoEmotions and EmpatheticDialogs.
1 Introduction
- Fine-grained emotion classification is difficult because it combines more classes with greater similarity between labels, such as furious and angry.The cited datasets contain up to 27 and 32 emotion classes, respectively.Source: §1 · p. 1
- Euclidean fine-tuning may force nearly synonymous emotion labels apart in latent space, risking overfitting, while emotion labels can have a hierarchy.The passage illustrates the hierarchy with Joy, Contentment, and Pleasure at primary, secondary, and tertiary levels.Source: §1 · p. 2
- On GoEmotions and EmpatheticDialogs, HypEmo outperforms existing methods and performs best with complex label structures or ambiguous inter-class relationships.The authors also report performance comparable to systems using external knowledge or data augmentation.Source: §1 · p. 2
- HypEmo learns hierarchical label embeddings in hyperbolic space, projects RoBERTa representations there, and uses text-label distance to weight cross-entropy loss.The paper’s contributions include a framework and objective function for fine-grained emotion classification.Source: §1 · p. 2
2 Related Work
The related work motivates hyperbolic representations as a way to model hierarchical labels and address the limits of Euclidean approaches to fine-grained classification. It also introduces the Poincaré ball as a differentiable hyperbolic representation model.
2.1 Fine-grained Classification
- Prior fine-grained classification work primarily uses Euclidean space, where the authors argue that emotions with close semantics may not be separable.The cited approaches include lexicon-based features, parse-based representations, generative models, and label-aware contrastive loss.Source: §2.1 · p. 2
2.2 Hyperbolic Geometry
- Hyperbolic geometry’s expanding space suits tree-like label hierarchies, motivating joint representation learning in Euclidean and hyperbolic spaces.The passage notes that tree nodes grow exponentially with depth.Source: §2.2 · pp. 2–3
2.3 Poincaré Embeddings
- The Poincaré ball is commonly used for hyperbolic representation learning because it provides a differentiable distance function.Its Riemannian metric defines geometric quantities including distances, angles, and curve length.Source: §2.3 · p. 3
3 Methodology
HypEmo combines Euclidean text representations with hyperbolic projections and label embeddings. It uses the distance between text and label representations to weight cross-entropy loss.
3.1 Euclidean Sequence Representations
- RoBERTa encodes each input sequence, using the [CLS] hidden state as its contextualized representation; the encoder can be model-agnostic.The paper uses RoBERTa-base and notes that transformer-like models or an encoder trained directly in hyperbolic space could also be used.Source: §3.1 · p. 3
3.2 Hyperbolic Projection
- The Poincaré-ball exponential and logarithmic maps project representations between Euclidean and hyperbolic spaces, allowing the framework to use both.The paper motivates this combination by the strength of Euclidean-pretrained language models and the ability of hyperbolic space to model label inventories.Source: §3.2 · p. 3
3.3 Hyperbolic Label Embeddings
- Label embeddings are trained in hyperbolic space from parent–child relations, using Parrott’s emotion hierarchy when a dataset lacks such relations.The objective maximizes distances between unrelated labels; Riemannian Adam is used for optimization.Source: §3.3 · pp. 3–4
3.4 Label-Aware Hyperbolic Weighted Loss
- The model projects text representations into hyperbolic space and uses their distance to the corresponding label embedding as a weight on cross-entropy loss.The distance penalizes text–label pairs farther apart in hyperbolic space, while the framework jointly learns representations across Euclidean and hyperbolic spaces.Source: §3.4 · pp. 4–5
4 Experiment
Across GoEmotions and Empathetic Dialogues, HypEmo is evaluated against language-model, label-aware, and hyperbolic baselines. Results cover overall performance and efficiency, difficult label subsets, hierarchy choices, and learned representations.
4.1 Datasets
- The experiments use GoEmotions and Empathetic Dialogues, containing 27 and 32 emotion labels, respectively, with dataset-specific single-label inputs and splits.GoEmotions excludes neutral and multi-labeled samples; Empathetic Dialogues uses the situation as input.Source: §4.1 · p. 5
4.2 Experiment Settings and Baselines
- Baselines include pretrained language models, label-aware and hierarchy-aware classifiers, and hyperbolic classifiers; evaluation uses accuracy and weighted F1.The experiments use RoBERTabase as the backbone, with Euclidean Adam for the encoder and Riemannian Adam for label embeddings.Source: §4.2 · pp. 5–6
4.3 Main Results
- 2.8% and 2.5% absolute F1 gains over LCL on ED and GE, respectively, accompany HypEmo’s use of one encoder and no data augmentation.LCL doubles data through synonym replacement and uses two encoders; HypEmo also outperforms LabelEmb and hierarchy-aware HiAGM.Source: §4.3 · p. 6, Table 1 · p. 7
- HypEmo improves weighted F1 across tested encoders; on BERTbase, it raises ED performance by 5.9 percentage points.The authors report improvement on RoBERTabase and ELECTRAbase across both datasets as well.Source: §4.3 · p. 6, Table 2 · p. 7
4.4 Case Study
- On difficult ED label subsets, HypEmo beats LCL on subset (a) (63.6 vs 54.9) and outperforms CE and SupCon on subsets a, b, and c.It remains on par with other models on the simpler subset d and outperforms augmented LCL as well.Source: §4.4 · p. 7, Figure 3 · p. 7
4.5 Effect of Label Hierarchy
- The expert-defined Parrott hierarchy performs best, while randomly shuffling the hierarchy performs worst.The authors use Parrott’s emotion model to form the label hierarchy.Source: §4.5 · p. 8, Figure 5 · p. 8
4.6 Visualization of Representations
- In t-SNE visualizations, the hyperbolic weighted loss separates afraid and terrified more clearly than CE or SupCon.The authors report larger inter-class distances and more dispersed clusters for the proposed loss.Source: §4.6 · p. 8, Figure 4 · p. 8
5 Conclusion
The conclusion presents HypEmo as a joint Euclidean–hyperbolic framework using a label-aware weighted loss for FEC. It reports state-of-the-art weighted F1 on two benchmarks and stronger performance when labels are difficult to distinguish.
5 Conclusion
- HypEmo combines Euclidean and hyperbolic representations through a label-aware hyperbolic weighted loss for FEC.Source: §5 · p. 8
- HypEmo achieves state-of-the-art weighted F1 on GoEmotions and Empathetic Dialogues and outperforms data-augmentation methods more efficiently.Source: §5 · p. 9
- HypEmo performs better when emotion labels are difficult to differentiate.Source: §5 · p. 9
Limitations
The authors report English-only results because they know of no fine-grained emotion datasets in other languages. The method works best with hierarchical, semantically close labels and may revert to conventional classification for flat, independent labels.
Limitations
- Results are English-only; the authors report no fine-grained emotion datasets in other languages.Source: Limitations · p. 9
- HypEmo works best with hierarchical labels, especially when some labels are semantically close and difficult to distinguish.Source: Limitations · p. 9
- With flat, independent labels, the method may fall back to conventional classification.Source: Limitations · p. 9