Distributional sentiment modeling and anomaly detection for consumer complaint assessment

1.1.1 Sentiment analysis and complaint narratives

• Complaint narratives contain heterogeneous accounts of dissatisfaction and financial harm, making them useful both for classification and for measuring dissatisfaction intensity.Source: §1 · p. 2

1 Introduction

• Continuous negative sentiment scores preserve distributional information that discrete polarity labels discard, allowing complaint intensity to be examined across outcomes.The paper focuses on the score’s mean, variance, shape, and tail behavior.Source: §1 · pp. 2–3

1.1.2 Operational risk and financial text analytics

• Operational-risk research uses unstructured text to detect emerging risks, but complaint analytics largely emphasizes point predictions or case flags rather than sentiment-score distributions.Source: §1.1.2 · p. 5

1.1.3 Bounded sentiment scores, Beta models, and topic representations

• Because negative sentiment scores lie on (0, 1), Beta models can summarize their bounded shape and dispersion; the paper distinguishes this single-score role from topic models of issue proportions.Source: §1.1.3 · p. 5

1.1.4 Distributional distances and divergences in sentiment analysis

• KL divergence and squared Hellinger distance compare fitted outcome-group distributions, shifting the question from a narrative’s polarity to whether full sentiment distributions differ.Source: §1.1.4 · p. 6

1.1.5 Anomaly detection in sentiment analysis

• Record-level diagnostics compare each complaint with fitted distributions to flag tail scores or outcomes inconsistent with the sentiment distribution, rather than merely searching for extremes.Source: §1.1.5 · p. 7

• The study covers CFPB narratives, group-level Beta fitting and distributional comparisons, plus diagnostics for outcome inconsistency, tail behavior, and high negative sentiment without relief.Source: §1.2 · p. 7

• The analysis constructs continuous negative sentiment scores for cleaned narratives and applies a boundary correction for Beta modeling.Source: §1.3 · p. 7
• A Beta distribution is fitted to the full cleaned sample and separately to meritorious and non-meritorious complaints.Source: §1.3 · p. 8
• KL divergence and squared Hellinger distance measure separation between the two fitted distributions and test group information beyond pointwise classification.Source: §1.3 · p. 8
• Fitted-distribution diagnostics identify outcome–sentiment inconsistency, tail behavior, and high negative sentiment cases without relief.Source: §1.3 · p. 8

• Negative sentiment is treated as a continuous random variable on (0, 1), rather than as a polarity label, so its whole distribution can be studied.Source: §1.4 · p. 8
• Beta-family models provide interpretable representations of bounded support, asymmetry, and dispersion when characterizing meritorious and non-meritorious complaints.Source: §1.4 · p. 8
• KL divergence and squared Hellinger distance compare the fitted outcome-group distributions as a complement to classification metrics.Source: §1.4 · p. 8
• Anomaly diagnostics connect sentiment analysis to operational-risk monitoring by treating outcome–sentiment inconsistency, tail observations, and high-sentiment cases without relief as screening signals.Source: §1.4 · p. 8
• The diagnostic procedure is presented as a transparent layer for regulators, financial institutions, and consumer-protection agencies to support complaint interpretation, classification, and risk assessment.Source: §1.4 · pp. 8–9

2 Data

• The sample covers complaints filed from 1 September 2023 to 30 June 2026, beginning after the CFPB revised its product and issue options.The date boundary keeps product and issue labels consistent after the form change.Source: §2 · p. 9

2.1 Preprocessing and visualization

• Records were retained when narratives included a dollar amount averaging from $1 to under $10,000 and a meritorious or non-meritorious response label.Averages of $10,000 or more were excluded because they often reflect business accounts, institutional disputes, or unusual cases.Source: §2.1 · p. 10
• The cleaned sample contains 181,316 narratives: 85,501 meritorious and 95,815 non-meritorious complaints, with an imbalance ratio of 1.12.Meritorious complaints received monetary or non-monetary relief; non-meritorious complaints closed with an explanation.Source: §2.1 · p. 10, Table 2.1 · p. 10
• Both outcome groups cluster at low average dollar amounts and share a similar right tail, while non-meritorious complaints are slightly more common in some low bins.The figure's x-axis shows average dollar amount and its y-axis shows narrative counts by bin.Source: §2.1 · p. 10, Figure 2.1 · p. 11

3 Distributional sentiment modeling

• The workflow converts each narrative into a corrected negative-sentiment probability and models scores for the full sample and outcome groups with Beta distributions.Extracted dollar amounts provide monetary context, and the sentiment score represents textual intensity.Source: §3 · p. 11
• The fitted outcome-group distributions are compared using KL divergence and squared Hellinger distance.Source: §3 · p. 11
• Record-level diagnostics flag outcome-sentiment inconsistency, tail probabilities, and high-sentiment cases without relief.Source: §3 · p. 11

3.1 Continuous sentiment score construction

• The cleaned corpus contains Nclean complaint narratives, each represented by narrative ci, dollar amount Ai, and outcome Yi, where Yi = 1 marks a meritorious complaint.The algorithm applies the classifier, corrects scores, fits Beta distributions, compares groups, and computes record-level diagnostics.Source: §3.1 · p. 12
• A transformer classifier supplies negative, neutral, and positive probabilities, and the paper uses FinBERT to score complaint narratives.FinBERT is selected as a financial-domain-adapted model for narratives containing financial products, credit reporting terminology, disputes, and regulatory language.Source: §3.1 · pp. 12–13
• The negative-class probability is the continuous sentiment score because it carries the most direct signal of complaint severity and operational risk.Higher values indicate stronger negative sentiment, preserving classifier output rather than reducing it to a polarity label.Source: §3.1 · p. 13
• A boundary correction maps sentiment scores into (0, 1) while preserving observation ordering for Beta modeling.The correction addresses the Beta distribution's open support when softmax probabilities approach 0 or 1.Source: §3.1 · p. 13

3.2 Beta models for continuous sentiment intensity

• Corrected scores are treated as bounded continuous variables on (0, 1) and modeled with a Beta distribution.The fitted distribution for the full cleaned sample compactly describes its overall negative-sentiment landscape.Source: §3.2 · pp. 13–14
• Beta parameters for any sample subset are estimated by maximum likelihood.Source: §3.2 · p. 14
• The cleaned corpus is partitioned by outcome, and the resulting group-specific Beta distributions represent differences in sentiment intensity, dispersion, and shape.These fitted distributions feed the separation metrics and anomaly diagnostics.Source: §3.2 · p. 14

3.3 Distributional separation metrics

• The metrics assess differences in the full sentiment distributions rather than relying only on mean sentiment scores.Source: §3.3 · pp. 14–15

3.3.1 Kullback–Leibler divergence

• KL divergence is asymmetric and measures the information loss from using the non-meritorious distribution to approximate the meritorious distribution.A larger value indicates greater directional separation.Source: §3.3.1 · p. 15

3.3.2 Squared Hellinger distance

• Squared Hellinger distance is symmetric and bounded from 0 to 1, with 0 indicating identical distributions and values near 1 indicating strong separation.Source: §3.3.2 · pp. 15–16

3.4.1 Outcome sentiment inconsistency

• The primary screen flags complaints whose sentiment score is more plausible under the opposite outcome distribution; its baseline threshold is η = 0.This is a post-classification reliability check based on disagreement between outcome information and distributional sentiment evidence.Source: §3.4.1 · pp. 17–18

3.4.2 Tail based diagnostic anomalies

• The secondary tail-based screen identifies unusual sentiment intensity within a complaint’s observed outcome group, using τ = 0.05 as its baseline.Sensitivity analysis uses τ values of 0.01, 0.05, and 0.10.Source: §3.4.2 · p. 18

3.4.3 High sentiment no relief cases

• The most targeted screen combines non-meritorious complaints, an upper-5% sentiment cutoff, and dollar amounts above the cleaned-sample upper quartile for operational screening.These cases are not classified as incorrect company responses, and the diagnostic framework requires no temporal window structure.Source: §3.4.3 · p. 19

4.1 Negative sentiment signals and Beta fits

• For narratives exceeding 512 tokens, the study scores consecutive chunks and T5-compressed text, then averages the two similar scores for each document.The resulting score stays in [0, 1] and feeds boundary correction and Beta fitting.Source: §4.1 · pp. 19–20
• The non-meritorious group has a slightly higher fitted mean and larger variance; the full-sample mean lies between groups and nearer the larger non-meritorious group.Most scores fall roughly between 0.10 and 0.50, with both groups’ right tails reaching toward 1.Source: §4.1 · pp. 20–21
• KS gaps are 0.058 (full sample), 0.056 (meritorious), and 0.055 (non-meritorious); the misfit is small in the center and concentrated in the upper tail.These are effect sizes rather than test decisions: large samples reject the exact Beta null for small deviations, but the model remains a useful first approximation.Source: §4.1 · pp. 21–22, Figure 4.2 · p. 22

4.2 Distributional separation between outcome groups

• The squared Hellinger distance is 0.0046, near zero on its [0, 1] scale, and symmetric average KL divergence is 0.0185, confirming heavy overlap; negative sentiment is not an outcome separator on its own.The non-meritorious density is flatter and stays higher over much of the upper range.Source: §4.2 · p. 23

4.3 Anomaly diagnostics from fitted sentiment distributions

• η = 0 flags 87,326 complaints (48.16%); η = 0.10 lowers this to 51,883 (28.61%), while τ = 0.05 flags 8,973 (4.95%).At η = 0, rates are 69.67% for non-meritorious and 24.06% for meritorious complaints; tail-rule rates stay near τ, ranging from 1.13% at τ = 0.01 to 9.61% at τ = 0.10.Source: Table 4.4 · p. 24, §4.3 · pp. 24–25
• 1,629 complaints (0.90% of the cleaned sample) meet the high-negative-sentiment/no-relief screen: s*i > 0.814 and Ai > 2,861.54.They are not necessarily mislabeled; they flag unusually severe narratives with sizable amounts despite explanation-only closure.Source: §4.3 · p. 25

5 Conclusion

• Meritorious and non-meritorious fitted distributions overlap substantially; non-meritorious complaints have a slightly higher fitted mean, larger variance, and heavier upper tail.Group comparison alone cannot separate the outcomes.Source: §5 · p. 26
• The framework’s main diagnostic use is case-level: it flags highly negative complaints that received no relief for closer review alongside dollar amounts and other attributes.The paper presents this as operational risk monitoring, not a sharp group-separation rule.Source: §5 · p. 26

5.1 Limitations

• The study covers one CFPB product category and a sample period beginning after August 2023, so findings should be read within that setting.Other categories may differ in narrative style, company response patterns, dollar amounts, and sentiment structure.Source: §5.1 · p. 26
• Sentiment scores may reflect model calibration, domain transfer, text compression, and chunk-combination choices; Beta fits summarize rather than fully represent narrative evidence.The score also depends on the classifier used.Source: §5.1 · pp. 26–27
• The authors frame anomaly diagnostics as screening signals, not proof that a company response is wrong.Source: §5.1 · p. 27

5.2 Future work

• Future work proposes broader category, period, and subgroup coverage; joint severity and outcome modeling; human review of flagged complaints; and comparisons of sentiment models and distribution families.Source: §5.2 · p. 27