Arabic aspect based sentiment analysis using bidirectional GRU based models

1. Introduction

• Sentiment analysis determines an author's attitude toward a subject and classifies document polarity as positive, negative, or neutral.The introduction identifies corpus-based, lexicon-based, and hybrid-based approaches for Arabic sentiment analysis.Source: §1 · p. 1
• The study targets T2 opinion-target extraction and T3 aspect-polarity detection, two ABSA tasks that identify opinions associated with particular aspects.Document- and sentence-level sentiment analysis cannot precisely identify what people liked or disliked when reviews contain multiple or conflicting aspect opinions.Source: §1 · p. 2
• The BGRU-CNN-CRF model combines word and character representations to extract opinion targets, while IAN-BGRU classifies sentiment polarity toward those extracted aspects.The workflow first processes reviews sentence by sentence, then tokenizes, removes stopwords, and vectorizes each sentence before applying the models.Source: §1 · pp. 3–4
• 39.7% enhancement in F1-score for opinion target extraction (T2) and 7.58% in accuracy for aspect-based sentiment polarity classification (T3) over baseline research.The reported scores are 70.67% for T2 and 83.98% for T3.Source: §1 · p. 4
• The proposed models do not rely on handcrafted features or external resources such as lexicons, which are not widely available for Arabic content analysis.The paper notes that collecting such resources requires considerable effort.Source: §1 · p. 4

2. Related work

• The review is organized around English and Arabic ABSA studies covering opinion-target extraction and aspect-polarity tasks.These are the two principal areas treated in the section.Source: §2 · p. 4

2.1.1. Opinion Target Extraction (OTE)

• Deep recurrent networks, CNNs, and Bi-LSTM-CRF have been applied to opinion-target extraction, with reported gains from POS and domain-specific embeddings in one CNN approach.The cited studies include deep recurrent networks instead of CRF [Irsoy and Cardie, 2014], RNN variations, CNN [Da’u and Salim, 2019], and Bi-LSTM-CRF [Chen et al., 2017b].Source: §2.1.1 · p. 4

2.1.2. aspect sentiment polarity classification

• Aspect-polarity research ranges from target-dependent and attention-based LSTMs to memory and multi-attention models; BERT-based networks were reported to outperform non-BERT complex models.Prior work also describes target-context interactions, recurrent attention, and multi-hop attention for representing aspect-relevant context.Source: §2.1.2 · pp. 4–6

2.2. Arabic ABSA

• The paper describes Arabic ABSA research as limited despite the large number of Arabic speakers and the language's rich morphology.The section introduces the available studies in its following subsection.Source: §2.2 · p. 6

2.2.1. OTE and aspect sentiment polarity classification

• Prior Arabic hotel-review studies used handcrafted features or LSTM-based models; the paper investigates BGRU-CNN-CRF for extraction and IAN-BGRU for polarity.Earlier work includes supervised classifiers with morphological, syntactic, and semantic features, SVM and RNN, and Bi-LSTM-CRF and AB-LSTM-PC. The paper reports competitive results for its models and high OTE results with FastText embeddings.Source: §2.2.1 · p. 6

3. Proposed methods

• Two GRU-based models are proposed: BGRU-CNN-CRF for opinion-target extraction (T2) and IAN-BGRU for aspect-polarity classification (T3).The section introduces them as the models addressing the study's research tasks.Source: §3 · p. 7
• GRU is presented as a solution to vanishing and exploding gradient problems that make recurrent neural networks harder to train.The section notes that recurrent hidden-unit activations depend on the previous step and that RNNs can process inputs of any length.Source: §3 · p. 7
• RNNs process sequential data using recurrent hidden units whose activations depend on the preceding step.The section describes their ability to process inputs of any length and remember information over time.Source: §3 · p. 7

3.1.1. Gated Recurrent Unit (GRU)

• A GRU updates its state by interpolating between the previous activation and a candidate activation, using reset and update gates.The update gate controls how much past information is carried forward and how much new memory is added; the reset gate controls how much past information is forgotten.Source: §3.1.1 · p. 7

3.1. Background

• GRU is presented as a recurrent neural network component with fewer parameters and lower memory use than LSTM.The paper states that this simpler architecture makes GRU faster than LSTM during training.Source: §3.1.1 · p. 7, §3.1 · p. 7

3.1.2. Convolutional neural network (CNN)

• CNNs provide character-level word features, including prefixes and suffixes, which are combined with other representations for the OTE model.Character embeddings are processed with convolution filters and max pooling; dropout is applied before the CNN.Source: §3.1.2 · p. 8

3.1.3. Bidirectional GRU (BGRU)

• A bidirectional GRU combines forward and reverse sequence processing to provide context from both directions.The two GRUs process the input sequence in normal and reverse time order, and their outputs are combined.Source: §3.1.3 · p. 8

3.1.4. Conditional random field (CRF)

• The CRF layer decodes the most likely sequence of aspect labels, using a transition matrix to model dependencies between adjacent tags.The paper represents aspect spans with B-Aspect, I-Aspect, and O labels.Source: §3.1.4 · p. 8

3.2.1. IAN-BGRU for polarity classification

• IAN-BGRU uses bidirectional GRU representations for a target and its context, applies attention in both directions, then concatenates the attended representations for polarity classification.The target representation guides context attention, while the context representation guides target attention; the concatenated vectors are fed to softmax.Source: §3.2.1 · p. 9

3.2.2. BGRU-CNN-CRF for aspect term extraction

• BGRU-CNN-CRF concatenates character-level CNN features with word embeddings, models context with a bidirectional GRU, and decodes label sequences with a CRF.Dropout is applied before the CNN and to the BGRU output vectors.Source: §3.2.2 · p. 10

• The study focuses on sentence-level tasks, although the dataset provides both text-level and sentence-level annotations.The dataset was prepared for SemEval-2016 Task 5, a multilingual ABSA task spanning eight languages and seven domains.Source: §4 · pp. 10–11
• The dataset is supplemented with SVM and N-gram baseline research, whose results serve as task-specific baselines.Table 1 is described as presenting dataset size and distribution across the research tasks.Source: §4 · p. 11, Table 1 · p. 11

• 70% of the dataset was used for training, 10% for validation, and 20% for testing.Source: §5 · p. 11
• The models were trained and tested on the Arabic hotel reviews dataset using PyTorch.Source: §5 · p. 11
• Computations for each model were performed separately on a GeForce GTX 1080 Ti GPU.Source: §5 · p. 11

5.1. opinion target expression extraction task (T2)

• BGRU-CNN-CRF extracts opinion targets using word and character embeddings.Source: §5.1 · p. 11

5.1.1. Evaluation method

• F1 is adopted to evaluate OTE, with the score computed from precision and recall.Source: §5.1.1 · p. 11

5.1.2. Hyperparameters Setting

• Word embeddings use fastText or AraVec Word2Vec, with 100-dimensional word embeddings and GRU states; character embeddings are 25-dimensional.Training uses batch size 16, momentum 0.9, dropout 0.5, and early stopping; the best parameters appeared at about 60 epochs.Source: §5.1.2 · pp. 11–12

5.1.3. Results

• The model replaces character-level BLSTM with CNN and the baseline LSTM with GRU, citing fewer parameters for both choices.The authors describe GRU as easier to train and learn.Source: §5.1.3 · p. 12
• With fastText embeddings, the model outperformed Bi-LSTM-CRF and achieved results close to Al-Dabet et al. (2020).The comparison is reported for the Arabic hotel reviews dataset.Source: §5.1.3 · p. 12

5.2.1. Evaluation method

• Accuracy was adopted to evaluate aspect sentiment polarity identification, measuring the percentage of samples predicted correctly.The passage defines T and N as the correctly predicted samples and the total samples, respectively.Source: §5.2.1 · pp. 12–13

5.2.2. Hyperparameters Setting

• The IAN-BGRU used AraVec CBOW-unigram Twitter embeddings and was trained with Adam for 12 epochs, with batch size 64, dropout 0.3, and learning rate 3e−5.The reported word-embedding, attention-vector, and hidden-state dimensions were each 300; the L2-regularization weight was 2e−5.Source: §5.2.2 · pp. 13–14

5.2.3. Comparison Models

• The comparison set included an n-gram SVM baseline and systems whose reported results were adopted from prior research rather than reimplemented.The comparison used Arabic hotel reviews; INSIGHT-1 used CNN with concatenated aspect and word embeddings, while TD-LSTM modeled left and right context separately.Source: §5.2.3 · p. 14
• The comparison also included target-aware attention systems: AB-LSTM-PC integrates aspect embeddings, while IAN variants interactively model target and context using LSTM or GRU encoders.IAN-BGRU uses bidirectional GRU to model the aspect term and its context.Source: §5.2.3 · p. 14

5.2.4. Discussions

• The discussion reports that target-aware TD-LSTM outperformed LSTM, and AB-LSTM-PC stably outperformed TD-LSTM; it also links these differences to target modeling and attention.The authors describe IAN as the best-performing baseline family and report that IAN-GRU exceeded IAN-LSTM and IAN-BLSTM by about 0.5% and 0.25%, respectively.Source: §5.2.4 · pp. 14–15
• IAN-BGRU achieved the best results among the compared methods on T3, with Table 3 comparing accuracy for the proposed model, baseline, and other Arabic models.The discussion attributes IAN-BGRU's performance to two related attention networks that model context and target interactively.Source: §5.2.4 · p. 15, Table 3 · p. 15, §5.2.5 · p. 15
• MEMNET's multiple attention layers did not improve performance; it slightly exceeded AB-LSTM-PC but scored below all IAN variations.The authors suggest model complexity and the lack of a mechanism to stop attention when no useful information remains as possible reasons.Source: §5.2.4 · p. 15

5.2.5. Case study

• The case study describes IAN-BGRU correctly assigning positive sentiment to location and negative sentiment to rooms in one review.The authors also report that their model can classify polarity for a location when the review contains the shifting word “not,” without an external lexicon.Source: §5.2.5 · pp. 15–16

6. Conclusion and future work

• 39.7% enhancement in F1-score for opinion target extraction (T2) and 7.58% in accuracy for aspect-based sentiment polarity classification (T3) were reported over baseline research.The reported results were an F1 score of 70.67% for T2 and accuracy of 83.98% for T3.Source: §6 · p. 16
• The BGRU-CNN-CRF model extracts opinion targets, while IAN-BGRU identifies sentiment polarity toward the extracted aspects.The first combines word and character representations with bidirectional GRU, CNN, and CRF; the second is an interactive attention network based on bidirectional GRU.Source: §6 · p. 16
• The authors intend to apply transformer-based models to ABSA tasks in future work.Source: §6 · p. 16