PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative Models Enhanced by Sequence Labelling and Aspect-Code Switching Improve Cross-lingual Aspect-Based Sentiment Analysis

Jakub Šmíd, Pavel Přibáň, Pavel Král

arXiv:2608.30425v1cs.CL

TL;DR

Cross-lingual ABSA seeks fine-grained sentiment analysis in target languages without annotated data, but language-specific expressions and complex tasks remain underexplored. The paper combines SEQLAB with aspect-code switching and reports new state-of-the-art results across broad multilingual evaluations, while identifying aspect-term and sequence-labelling limitations.

  • Problem

    Zero-shot cross-lingual ABSA must transfer knowledge from annotated source languages to unannotated targets, while complex TASD and sequence-to-sequence models remain underexplored.

  • Method

    The framework combines a sequence-to-sequence model with auxiliary encoder sequence labelling and aspect-code switching that swaps aspect terms between source and translated sentences.

  • Results

    The method achieves new state-of-the-art results on cross-lingual E2E-ABSA and TASD across eleven languages, three domains, two backbone models, and varied source-target pairs.

  • Takeaways & Limitations

    SEQLAB and aspect-code switching provide a framework for evaluating and improving cross-lingual ABSA across languages, domains, tasks, backbones, and source-target configurations.

  • Takeaways & Limitations

    The labelling scheme omits aspect categories and operates on individual words rather than multi-word spans, limiting its expressiveness for aspect terms.

Abstract

from arXiv · show

Cross-lingual aspect-based sentiment analysis (ABSA) transfers knowledge from a source language with annotated data to a target language, enabling fine-grained sentiment analysis without annotated target-language data. While monolingual ABSA has seen significant progress, cross-lingual ABSA remains underexplored, especially for complex tasks involving multiple sentiment elements like target-aspect-sentiment detection (TASD). In this paper, we propose a novel SeqLab framework that enhances cross-lingual ABSA using a sequence-to-sequence model with an auxiliary sequence-labelling task performed by the encoder, enhancing aspect term recognition and sentiment predictions. Additionally, we incorporate aspect-code switching (ACS), a translation-based technique that swaps aspect terms between source and translated sentences, generating additional training data to enhance the model's cross-lingual understanding. We evaluate our approach across eleven languages, three domains, and two backbone models, surpassing previous state-of-the-art results for the commonly studied E2E-ABSA task. Unlike most prior work that relies solely on English as the source language, we systematically assess different source-target language pairs and extend our evaluation to the more challenging, yet underexplored TASD task in cross-lingual settings. Finally, we provide a detailed error analysis highlighting key challenges and limitations.

1 Introduction

The paper targets zero-shot cross-lingual ABSA, where knowledge must transfer from an annotated source language to unannotated target languages. It introduces SEQLAB with aspect-code switching to address language-specific challenges and extend evaluation beyond E2E-ABSA to TASD.

  • Zero-shot cross-lingual ABSA transfers knowledge from a labelled source language to target languages without annotated target-language data.
  • Language-specific aspect terms, informal expressions, and limited low-resource-language representation in multilingual models make zero-shot transfer challenging.
  • TASD simultaneously extracts aspect terms, aspect categories, and sentiment polarity, but complex tasks and sequence-to-sequence models remain largely unexplored cross-lingually.
  • SEQLAB combines sequence-to-sequence modelling with auxiliary encoder sequence labelling to improve aspect recognition and refine generated sentiment tuples.
  • Aspect-code switching swaps aspect terms between source and machine-translated sentences, augmenting target-language data and capturing language-specific variation.
  • The framework is evaluated across eleven languages, three domains, two backbone models, varied source-target pairs, and the underexplored TASD task, achieving new state-of-the-art results.

2 Related Work

Cross-lingual ABSA research has progressed from translation and embedding-based methods toward multilingual models and LLM-based approaches. However, prior work has focused mainly on E2E-ABSA, with fewer studies addressing TASD and generative models.

  • Early methods used translation-based annotation with word alignment or cross-lingual embeddings for aspect extraction and sentiment classification.
  • Recent approaches mainly address E2E-ABSA with multilingual encoder-only Transformers, often combined with machine translation.
  • TRANSLATION-TA fine-tunes on translated data, while BILINGUAL-TA adds original source data and ACS introduces alignment-free projection.
  • LLM-based methods explore pseudo-labelling, prompting, and cross-lingual fine-tuning, while some recent studies extend evaluation to TASD.

3 Methodology

The framework combines generative sequence-to-sequence ABSA with auxiliary encoder sequence labelling and aspect-code switching. Sequence-labelling signals assist training and inference, while translated and code-switched data expand the training pool.

  • Framework: SEQLAB uses a sequence-to-sequence model with auxiliary encoder sequence labelling and combined inference to improve generated sentiment tuples.The framework is designed for TASD and can support E2E-ABSA by omitting aspect categories.
  • Generative Framework: The generative model encodes the input contextually and decodes the output token by token using prior output tokens and the encoded representation.Special tokens represent aspects, categories, polarities, tuple separators, and sentences without sentiment tuples.
  • Sequence Labelling Assistance: The auxiliary labelling task assigns T-POS, T-NEG, T-NEU, or O labels to input tokens and maps encoder hidden vectors to label probabilities.Labels identify aspect-term words together with their sentiment polarity, while only the first token of each word is labelled.
  • Sequence Labelling Assistance: The sequence-labelling scheme does not fully represent multi-word aspect terms or aspect categories, because it focuses on individual words as an auxiliary signal.The authors describe this choice as simpler and more efficient than representing entire aspect phrases.
  • Inference: Generated tuples below a word-probability threshold are retained only when their aspect terms overlap with sequence-labelling predictions; otherwise, they are discarded.Tuples above the threshold are retained without further checks, while subset agreement between generated and labelled aspect terms permits lower-confidence tuples.
  • Aspect-Code Switching: Aspect-code switching creates source-aspect and target-aspect variants from each source sentence and translation, producing four uniformly sampled dataset subsets.The subsets are source, translated, and two code-switched datasets merged into one training pool.

4 Experimental Setup

The experiments evaluate E2E-ABSA and TASD across multilingual restaurant, hotel, and laptop data. They vary languages, source-target directions, backbone models, and comparison systems under a micro-F1 evaluation protocol.

  • Datasets and Tasks: Experiments cover E2E-ABSA and TASD using restaurant, hotel, and laptop datasets across eleven languages.The languages include English, Spanish, French, Dutch, Russian, Turkish, Japanese, Korean, Thai, Vietnamese, and Chinese.
  • Datasets and Tasks: The hotel and laptop datasets are parallel, with original reviews only in English and translations in the other languages.The restaurant dataset contains real user reviews in six languages.
  • Models and Protocol: The study uses base and large mT5 models alongside large mBART, with translations produced through the Google Translate API.Source-language validation sets are used for model selection in the unsupervised setting.
  • Evaluation: Micro-F1 counts a predicted tuple as correct only when all its sentiment elements are accurate, and scores are averaged across five random-seed runs.This evaluation is consistent with related cross-lingual ABSA work.
  • Baselines and Comparisons: SEQLAB is compared with a ZERO-SHOT source-only baseline and several prior methods, while differences in pretrained models and task definitions are acknowledged.Implicit aspect terms are excluded to maintain a uniform setup and enable direct comparison.

5 Results

Across languages and domains, combining SEQLAB with ACS delivers the strongest cross-lingual ABSA results, including on the underexplored TASD task. The gains persist against smaller-model and LLM baselines, while ablations and error analysis clarify component effects and remaining challenges.

  • Backbones and zero-shot transfer: Nearly 2%: SEQLAB improves average zero-shot results, while mT5 surpasses XLM-R by almost 2% on average.mT5 gains +7.43% for Dutch and +3.84% for Russian, but drops nearly 5% for Spanish.
  • Ablation and augmentation: Over 7% and 4%: ACS improves results over ZERO-SHOT and ZERO-SHOT + SEQLAB, respectively; SEQLAB adds approximately 1% over ACS alone.These results support the value of incorporating translated and aspect-code-switched target-language data.
  • Restaurant results: Over 3%: combining SEQLAB and ACS surpasses the previous best method on average in the restaurant domain.The combination achieves the best overall result for each language.
  • TASD and cross-lingual performance: Only 5% weaker: the best cross-lingual results are on average only 5% below monolingual performance, with TASD trends mirroring E2E-ABSA.SEQLAB consistently improves performance, while ACS substantially enhances cross-lingual transfer.
  • Ablation study: More than 2%: VANILLA lags behind SEQLAB in the zero-shot ablation, while the full generative model outperforms encoder-only prediction by 5–8%.Sequence labelling contributes more than the enhanced inference stage; sentiment information adds almost 1%.
  • Error analysis: Aspect-term prediction is the main challenge, involving missing, incorrect, or partial spans and occasional English outputs in target-language predictions.French has the lowest overall restaurant performance, while Dutch and Spanish have the highest; polarity is easiest but neutral remains difficult.

6 Conclusion

The paper proposes a sequence-to-sequence framework augmented with sequence labelling and aspect-code switching for cross-lingual ABSA. It reports state-of-the-art results across tasks, languages, domains, backbone models, and source-target combinations, while analyzing remaining challenges.

  • The framework combines an auxiliary encoder sequence-labelling task with aspect-code switching to improve cross-lingual ABSA.The auxiliary task supports aspect-term recognition and sentiment prediction, while aspect-code switching generates additional training examples by swapping aspect terms between source and target sentences.
  • The method achieves new state-of-the-art results on both E2E-ABSA and TASD.TASD extends the evaluation to a more complex cross-lingual task involving multiple sentiment elements.
  • The evaluation covers eleven languages, three domains, two backbone models, and systematically varied source-target language pairs.
  • The error analysis identifies key challenges and provides insights for future research.

Limitations

The framework has limitations in task scope, architecture, sequence-labelling expressiveness, and model-family coverage. Its sequence-labelling component operates on individual words and omits aspect-category information, while decoder-only models are not evaluated.

  • The experiments focus on cross-lingual ABSA, leaving possible extensions to tasks such as named entity recognition unexplored.
  • The added encoder linear layer introduces a small overhead compared with traditional sequence-to-sequence models.
  • The simple sequence-labelling approach omits aspect categories and represents individual words rather than multi-word aspect spans.A more expressive labelling scheme could potentially improve performance.
  • The study evaluates encoder-decoder models but does not explore decoder-only alternatives.The paper notes that decoder-only models may be less suitable for sequence labelling because of unidirectional processing and limited fine-grained token-level supervision.

Ethics Statement

The study uses established benchmark datasets to support fair and transparent comparisons and reports no involvement of harm to individuals. It also acknowledges that pretrained models may contain unintended biases from large-scale web training data.

  • The experiments use widely adopted benchmark datasets from prior research to enable fair and transparent result comparison.
  • The study was conducted ethically and does not involve harm to individuals.
  • The pretrained models may reflect unintended race- or gender-related biases inherited from large-scale web data.

B Experimental Details

The experimental setup uses benchmark datasets across languages and domains, specified optimization and decoding settings, multiple pretrained backbones, translation-based augmentation, and prompt-based task descriptions. Additional experiments examine source-target combinations, model variants, and task definitions.

  • The experiments use models from HuggingFace Transformers, AdamW optimization, batch size 16, greedy decoding, threshold 0.999, and sequence-labelling loss weight α=0.1.Training runs search over as many as 25 epochs on a single NVIDIA L40 GPU with 48 GB memory.
  • The evaluation includes restaurant, hotel, and laptop domains with dataset statistics reported separately for the language and domain collections.The hotel and laptop datasets contain identical numbers of sentences and aspects across languages because they are parallel.
  • The main experiments use base mT5, with large mBART and large mT5 included to assess robustness.The reported learning rates are 1e-4 for base mT5, 1e-5 for large mBART, and 1e-4 for large mT5.
  • Google Translate API is used for translation, with approximately 6% of cases discarded when special symbols are lost.Existing ACS data lacked aspect-category annotations for TASD, so the original categories are recovered and new data are generated.
  • The prompt defines the sentiment-analysis task, sentiment elements, and expected output, with TASD prompts adaptable to E2E-ABSA by omitting aspect categories.The aspect-term and aspect-category definitions and category inventory are specified for the task.
  • Additional experiments report results across source-target language combinations, backbone models, tasks, and domains.Table 9 provides complete results for different methods with English as the source language.

C.1 Detailed Domain Results

Domain and language-pair results show that ACS and SEQLAB are generally effective, but the strongest training strategy depends on target-domain data quality and language pair. TASD remains harder than E2E-ABSA, while combining ACS with SEQLAB improves performance across tasks, models, and settings.

  • Domain Results: Multilingual training performs best in restaurants, whereas ACS + SEQLAB often outperforms monolingual models in hotel and laptop domains.TR-ONLY is notably weaker than multilingual training in restaurants.
  • Domain Results: 99.87% BERTScore and 99.28% SBERT similarity indicate that translated and gold target-language data are nearly identical in hotel and laptop domains.This similarity helps explain why multilingual and TR-ONLY results are nearly identical there.
  • Domain Results: High-quality target-language data favors multilingual or monolingual training, while ACS + SEQLAB is most effective when such data are unavailable or lower quality.The reported domain differences are attributed to variation in whether target-language data are human-authored or translation-derived.
  • Training Strategies: ACS consistently outperforms TR-ONLY, and adding SEQLAB generally produces further gains across training strategies.ZERO-SHOT is the weakest strategy in the reported comparisons.
  • Cross-lingual Tasks: SEQLAB consistently improves E2E-ABSA and TASD, and combining it with ACS further enhances results across source-target language pairs.TASD is more challenging because it predicts sentiment-element triplets rather than simpler tuples.
  • Source–Target Combinations: Spanish-to-French transfer performs exceptionally well, whereas Turkish is consistently the weakest source and target language.The passage associates these patterns with shared Romance-language characteristics and Turkish structural differences.
  • Model Comparisons: ACS + SEQLAB outperforms GPT-4o mini, with gains exceeding 20% in most cross-lingual TASD settings.The comparison is reported against GPT-4o mini's zero-shot performance.
  • Model Comparisons: Across mT5 and mBART, larger models perform better, and each model achieves its best results when ACS and SEQLAB are combined.mT5 large achieves the best overall performance in the reported model comparison.

D Qualitative Error Analysis

The qualitative analysis categorizes representative ACS + SEQLAB errors on the restaurant dataset into four aspect-extraction failure types. These errors concern incomplete, excessive, unrelated, or unsupported aspect predictions.

  • Error Taxonomy: The analysis examines representative ACS + SEQLAB errors on the restaurant dataset and organizes them into four categories.The examples use the mT5-base model, with Cyrillic text transliterated for readability.
  • Error Types: Partial aspect extraction occurs when the model predicts only part of a multi-word aspect.This error reflects incomplete span identification.
  • Error Types: Over-predicted aspect errors include extra tokens beyond the gold aspect.The predicted span is longer than the reference aspect.
  • Error Types: Wrong aspect extraction occurs when the model predicts an unrelated aspect.The predicted aspect does not correspond to the intended gold aspect.
  • Error Types: False positives occur when the model predicts an aspect even though none is present.The error is an unsupported aspect prediction.
Loading 2608.30425v1…