PaperlayerView exact revision on arXiv

Source-linked AI summary

A Hybrid Persian Sentiment Analysis Framework: Integrating Dependency Grammar Based Rules and Deep Neural Networks

Kia Dashtipour, Mandar Gogate, Jingpeng Li, Fengling Jiang, Bin Kong, Amir Hussain

arXiv:1909.13568v1cs.CLcs.LG

TL;DR

The daily bytes of user-feedback data cannot be handled manually. The paper proposes a Persian hybrid framework that combines dependency-based rules with deep neural networks. The authors report significant performance improvement over state-of-the-art SVM, logistic regression, and advanced DNN approaches.

  • Problem

    The daily bytes of user-feedback data cannot be handled manually.

    Source: §1 · p. 2

  • Method

    The paper proposes a Persian hybrid framework that combines dependency-based rules with deep neural networks.

    Source: §6 · p. 26

  • Results

    The authors report significant performance improvement over state-of-the-art SVM, logistic regression, and advanced DNN approaches.

    Source: §6 · p. 26

  • Takeaways & Limitations

    Dependency-based rules let sentiment flow from words to concepts through symbolic dependency relations, while DNNs classify when no rule pattern is triggered.

    Source: §1 · p. 3

    The authors state that the model has only been evaluated in the settings reported, leaving its performance in other settings unestablished.

    Source: §6 · p. 26

Abstract

from arXiv · show

Social media hold valuable, vast and unstructured information on public opinion that can be utilized to improve products and services. The automatic analysis of such data, however, requires a deep understanding of natural language. Current sentiment analysis approaches are mainly based on word co-occurrence frequencies, which are inadequate in most practical cases. In this work, we propose a novel hybrid framework for concept-level sentiment analysis in Persian language, that integrates linguistic rules and deep learning to optimize polarity detection. When a pattern is triggered, the framework allows sentiments to flow from words to concepts based on symbolic dependency relations. When no pattern is triggered, the framework switches to its subsymbolic counterpart and leverages deep neural networks (DNN) to perform the classification. The proposed framework outperforms state-of-the-art approaches (including support vector machine, and logistic regression) and DNN classifiers (long short-term memory, and Convolutional Neural Networks) with a margin of 10-15% and 3-4% respectively, using benchmark Persian product and hotel reviews corpora.

1. Introduction

The introduction argues that word-frequency methods miss word order and dependency relations needed to identify sentiment, while Persian analysis also faces noisy text and scarce resources. It presents dependency-based rules and a hybrid rules–DNN framework, evaluated against conventional and neural classifiers.

1. Introduction

  • Persian word co-occurrence approaches overlook word order and dependency relations, which can change a sentence’s overall polarity.The introduction also identifies noisy text and scarce Persian NLP resources as challenges for reliable sentiment analysis.Source: §1 · pp. 2–3
  • The proposed rules use dependency patterns to propagate sentiment from words to concepts, accounting for hierarchical relations, word order, and individual polarities.The paper describes the rule-based classifier as assigning Persian sentence polarity without supervised training.Source: §1 · pp. 3–4
  • Dependency-based rules outperform SVM, LR, and fastText by 10–15% and perform comparably with DNN classifiers.The introduction describes experiments on benchmark Persian product and hotel reviews.Source: §1 · p. 4
  • The hybrid framework combines the dependency-rule classifier with DNN architectures to address sentences that the rules leave unclassified.The introduction names CNN and LSTM among the DNN architectures used in the framework.Source: §1 · pp. 4–5
  • The paper also reports an ablation study of individual dependency rules within the complete framework.Its experiments compare the proposed approach with SVM, logistic regression, and DNN models on product and hotel review corpora.Source: §1 · p. 5

2. Related Work

The related work surveys symbolic, statistical, and neural sentiment methods, then identifies gaps in Persian approaches involving word order, dependency structure, and language resources. It motivates combining dependency-based rules with deep neural networks for Persian sentiment analysis.

2.1. English Sentiment Analysis Approaches

  • English sentiment methods include lexicons and rules as well as supervised, semi-supervised, and unsupervised learning based on word co-occurrence.Prior work also uses dependency trees to propagate polarity and dependency-based semantic parsing to construct SVM features.Source: §2.1 · pp. 5–6

2.2. Multilingual Sentiment Analysis Approaches

  • Translation-based multilingual sentiment methods depend on translation quality and may fail on mixed-language input or lose subjective context and real-life writing styles.The related work describes these as limitations of translation-based approaches for scarce-resource languages.Source: §2.2 · p. 6

2.3. Persian Sentiment Analysis Approaches

  • Persian studies include n-gram and neural feature-extraction approaches, but many rely on word co-occurrence and lexicons without exploiting hierarchical relations or word order.The cited fourgram method outperformed unigram, bigram, and trigram features; a deep-autoencoder approach outperformed CNN and MLP for Persian polarity detection.Source: §2.3 · p. 7
  • The paper notes that lexical negation rules from other languages cannot be directly applied to Persian, while DNNs have shown strong performance with large supervised corpora.It proposes a Persian framework combining dependency-based rules and DNNs to improve polarity detection in noisy data.Source: §2.3 · p. 7

3. Methodology

The methodology contrasts word-frequency polarity counting with dependency-based rules. The rules use syntactic relations and dependent words to determine how sentiment flows through a sentence and affects its overall polarity.

3. Methodology

  • Dependency-based rules use hierarchical dependency relations to determine sentence sentiment beyond word co-occurrence frequencies.The methodology illustrates that counting negative words can misclassify a sentence whose dependent words alter its polarity.Source: §3 · p. 8
  • The rules use syntactic relations between words to establish a logical flow of sentiment and determine overall sentence polarity.The examples describe “not” and “but” as dependent words that change the sentence’s polarity or the polarity of its second component.Source: §3 · p. 8
  • In the example, “old” does not change overall polarity, while “not” changes the polarity associated with “bad” and “but” affects the second component.The section points to dependency trees as the representation of syntactic relations used by the rules.Source: §3 · p. 8

3.1. Persian Dependency-based Rules

The section presents Persian dependency-based sentiment rules, adopted from and, that use syntactic patterns to determine polarity. The rules cover polarity inversion, complement clauses, and adversatives.

3.1. Persian Dependency-based Rules

  • The section presents novel Persian dependency-based sentiment rules adopted from and.Source: §3.1 · p. 9

3.1.1. Polarity Inversion

  • Negation reverses a token’s polarity: positive tokens become negative, while negative tokens become positive.Source: §3.1.1 · p. 9

3.1.2. Complement Clause

  • With a complement clause introduced by “that,” the sentence is split and the polarity of the first part determines overall polarity.Source: §3.1.2 · pp. 9–10

3.1.3. Adversative

  • For adversatives such as “but,” “although,” or “however,” the sentence is split and the second part determines overall polarity.Source: §3.1.3 · p. 11

3.1.4. Adverbial Clause

These rules use clause and dependency structure to focus sentiment assignment on relevant parts of a sentence. They cover adverbial and adjective clauses, related noun–adjective pairs, and prepositions.

3.1.4. Adverbial Clause

  • An adverbial clause triggers the rule only when it contains a subject and verb; “whereas” splits the sentence, with the second part determining sentiment.Source: §3.1.4 · p. 11

3.1.5. Adjective Clause

  • A sentence containing an adjective-clause pronoun such as “which” is split, and the polarity of the second part is considered.Source: §3.1.5 · p. 12

3.1.6. Joint Noun and Adjective

  • When a noun is related to an adjective, both are extracted and the adjective’s polarity is considered.Source: §3.1.6 · p. 12

3.1.7. Preposition

  • A negative preposition modifier changes sentence polarity: negative activity becomes positive, while positive activity becomes negative.Source: §3.1.7 · p. 12

3.1.8. Additional rule

The additional rules use cues such as “This,” “which,” prepositions, and emoji to select a sentence part or assign polarity. The section also summarizes the dependency-based rules in Table 1.

3.1.8. Additional rule

  • When “This” appears mid-sentence, the sentence is split and the first part determines polarity; the example’s sentiment after “this” is not considered.Source: §3.1.8 · p. 13
  • The rule overview summarizes adversative handling as using the second part’s polarity and preposition handling as changing polarity.Source: Table 1 · p. 13, §3.1.8 · p. 14
  • For “which,” the rule splits the sentence and considers the second part; related noun–adjective pairs are extracted for lexicon-based polarity assignment.Source: §3.1.8 · p. 14
  • Positive prepositions before adjectives can make sentence polarity positive, negative prepositions can make it negative, and emoji polarity can determine sentence polarity.Source: §3.1.8 · p. 14

3.2. Preposition Sub-rule

The preposition sub-rule assigns polarity using positive or negative preposition words before adjectives. The text also identifies a word that the negation rule fails to detect.

3.2. Preposition Sub-rule

  • Positive preposition words such as “with,” “happy,” and “enjoy” before an adjective can make concept polarity positive.Source: §3.2 · pp. 14–15
  • Negative preposition words such as “without,” “anti,” “Not,” “poison,” and “No” change sentence polarity to negative.Source: §3.2 · p. 15
  • The negation rule does not detect the word A K, so it cannot be triggered by that word.Source: §3.2 · p. 15

3.3. Emoji Sub-rule

Emojis are treated as expressive components that generally reinforce sentence polarity, except in sarcastic sentences. The sub-rule assigns positive or negative sentence polarity according to whether a positive or negative emoji appears.

3.3. Emoji Sub-rule

  • Emojis generally reinforce the sentence’s expressed polarity, except in sarcastic sentences.Source: §3.3 · p. 15
  • A positive emoji in a sentence assigns positive polarity to the sentence.Source: §3.3 · p. 15
  • A negative emoji in a sentence assigns negative polarity to the sentence.Source: §3.3 · p. 15

4. Hybrid Framework

The framework combines dependency-based rules with CNN and LSTM classifiers, using the neural models when the rules cannot classify a sentence. It uses concatenated fastText embeddings for DNN input and describes the CNN and LSTM architectures used.

4. Hybrid Framework

  • The proposed hybrid framework integrates dependency-based rules with DNN classifiers, including CNN and LSTM.Source: §4 · p. 15

4.1. Framework Overview

  • Sentences first pass through dependency-based rules; when no rule applies or lexicon coverage is unavailable, concatenated fastText embeddings go to a DNN for polarity classification.The overview also describes preprocessing and using the PerSent lexicon to assign polarity to extracted words.Source: §4.1 · pp. 15–16

4.2. Deep Neural Networks DNN architecture

  • DNN input concatenates fastText embeddings; the paper reports that embeddings learned from a small supervised corpus perform worse than unsupervised fastText embeddings.Source: §4.2 · p. 17

4.3. Convolutional Neural Network (CNN)

  • The CNN uses convolutional, max-pooling, and fully connected hidden layers; the best experimental results used a 9-layer architecture.The architecture is illustrated in Fig. 6 and referenced with Table 2.Source: §4.3 · p. 17, Figure 6 · p. 17

4.4. Long Short Term Memory (LSTM)

  • The LSTM consists of two stacked bidirectional layers with 128 and 64 cells, followed by dropout at 0.2 and a two-neuron softmax output layer.Source: §4.4 · p. 17

5. Performance Evaluation

This section evaluates the hybrid approach on product and hotel review benchmarks, comparing its performance with other classifiers and presenting examples of classified product-review test sentences.

5.1. Dataset

  • Product reviews comprised 1,500 positive and 1,500 negative manually annotated reviews; hotel reviews comprised 1,800 positive and 1,800 negative reviews.The corpora were used to evaluate the approach, with hotel reviews providing a new-domain comparison.Source: §5.1 · pp. 18–19

5.2. Data Preprocessing

  • The corpus was preprocessed using tokenization and normalization, punctuation was removed, PerSent assigned polarity to tokenized words, and Hazm parsed dependency trees with 90.86% accuracy.The dependency tree and assigned polarities were fed to the proposed dependency-based rules classifier.Source: §5.2 · p. 19

5.3. Experimental Setup

  • The dependency-based rules require no training; data were split into train (60%), validation (10%), and test (30%), with rule-based performance evaluated on test and compared against SVM, Logistic Regression, and fastText baselines.Source: §5.3 · pp. 19–20

5.4. Experimental Results

  • The hybrid approach achieved better accuracy than CNN, LSTM, SVM, and logistic regression on product and hotel reviews.The product and hotel comparisons are reported in Tables 3 and 4, respectively.Source: Table 3 · p. 20, §5.4 · p. 21, Table 4 · p. 22
  • Rules failed to classify 5% of product-test reviews and 16% of hotel-test reviews; hybrid CNN outperformed hybrid LSTM on product reviews, while hybrid LSTM led on hotel reviews.The unclassified shares were 2% positive and 3% negative for product reviews, and 8% positive and 8% negative for hotel reviews.Source: §5.4 · p. 21
  • In experiments comparing dependency-based rules across lexicons, SentiFarsNet produced lower accuracy than PerSent, attributed to SentiFarsNet’s smaller size: 2500 words with polarity.Source: §5.4 · p. 21
  • The rules are limited by missing word-sense and multiword-expression coverage, informal language and sarcasm, poor handling of long sentences, and undetected spelling mistakes.The paper identifies contextual spell-checking as a needed addition and proposes expanding PerSent to include currently uncovered words.Source: §5.4 · pp. 22–23
  • Joint-noun and adjective rules had better accuracy than the other rules in both corpora, while complement clauses performed worst.The ablation results are reported for product and hotel reviews in Tables 5 and 6.Source: §5.4 · pp. 24–25, Table 5 · p. 24
  • Table 7 presents sentences classified by the hybrid approach from the product-review test set.Source: §5.4 · p. 25

6. Conclusions

The hybrid framework combines Persian dependency-based rules with deep neural networks and significantly improves performance over conventional approaches and DNN models on product and hotel review corpora. Its evaluation is limited to two review domains, with broader generalization left for future study.

  • Persian sentiment approaches based on word co-occurrence frequencies fail to account for word order and hierarchical dependency relations relevant to sentiment.Source: §6 · pp. 25–26
  • The proposed framework combines Persian dependency-based rules with deep neural networks to address sentences left unclassified by the rules.The rules consider dependency relations, word order, and individual word polarities.Source: §6 · p. 26
  • The hybrid framework significantly improves performance over SVM, Logistic Regression, and advanced DNN models on benchmark product and hotel review corpora.The conclusion reports evaluation on review corpora from two domains.Source: §6 · p. 26
  • Evaluation covers review corpora from two domains; the authors identify generalization to more challenging corpora across applications as future work.Source: §6 · p. 26
  • Future work includes extending the lexicon, extracting multilingual concepts from mixed Persian-English sentences, and using Graph Neural Networks to learn dependency rules.The authors also plan to investigate subjectivity and sentiment detection in those sentences.Source: §6 · p. 26
Loading 1909.13568v1…