Source-linked AI summary
Latent Dirichlet Allocation (LDA) for Topic Modeling of the CFPB Consumer Complaints
Kaveh Bastani, Hamed Namavari, Jeffry Shaffer
TL;DR
The paper addresses the time-consuming, bias-prone labeling of CFPB complaints by proposing an LDA-based decision support system that identifies mixtures of topics. Its time-trend findings indicate successful enforcement for some topics, while regulations were not adequately effective for others.
Problem
Manual complaint analysis is time-consuming and often subject to human biases, motivating a text-mining approach that assigns multiple topics to consumer complaints.
Method
The paper proposes an LDA-based decision support system that summarizes consumer complaints as mixtures of topics.
Results
Decreasing time trends indicated successfully enforced regulations for Harassment, Mortgage/Loan Modification and Foreclosure, and Loan/Student Loan, while regulations were not adequately effective for Fraudulent/Unauthorized Inquiry and Dispute.
Takeaways & Limitations
The proposed system is designed to reveal useful complaint topics while supporting existing human analysts rather than replacing them.
Takeaways & Limitations
The CFPB labeling convention allows only one label, and the proposed tool is intended to support rather than replace human analysts.
Abstract
from arXiv · showhide
A text mining approach is proposed based on latent Dirichlet allocation (LDA) to analyze the Consumer Financial Protection Bureau (CFPB) consumer complaints. The proposed approach aims to extract latent topics in the CFPB complaint narratives, and explores their associated trends over time. The time trends will then be used to evaluate the effectiveness of the CFPB regulations and expectations on financial institutions in creating a consumer oriented culture that treats consumers fairly and prioritizes consumer protection in their decision making processes. The proposed approach can be easily operationalized as a decision support system to automate detection of emerging topics in consumer complaints. Hence, the technology-human partnership between the proposed approach and the CFPB team could certainly improve consumer protections from unfair, deceptive or abusive practices in the financial markets by providing more efficient and effective investigations of consumer complaint narratives.
1. Introduction
The CFPB complaint database contains growing narrative data whose manual review is time-consuming and bias-prone, while existing analyses largely omit narrative content. The paper proposes LDA-based text mining to extract multiple latent topics and examine their trends over time.
- Research gap: Existing CFPB complaint studies analyze company, demographic, or government-agency patterns but do not formally analyze complaint narratives.The narratives provide contextual information from which further insights could be built.
- Research gap: Increasing complaint-narrative volume creates a need for efficient review procedures because manual analysis is time-consuming and subject to human biases.The CFPB database is publicly available and updated regularly.
- Research gap: The CFPB labeling convention can mislabel complaints, omit issues absent from the menu, or force complaints involving multiple issues into one label.Consumers select predetermined labels and can select only one label.
- Contribution: The paper presents text mining as a way to extract non-trivial patterns from unstructured, noisy complaint documents and reports no prior published CFPB narrative study using text mining.It positions the work as demonstrating text mining’s utility for CFPB consumer complaint narratives.
- Proposed approach: LDA is proposed to extract latent topics from CFPB complaint narratives and assign narratives a mixture of topics.This probabilistic approach addresses the need to represent narratives that may share multiple topics.
- Proposed approach: The approach aims to reveal topic trends over time for evaluating CFPB regulations and financial companies’ responses to complaint issues.The paper presents the LDA-based analysis of CFPB consumer complaints and discusses its results.
2. LDA based Topic Modeling to the CFPB Complaint Narratives
The paper prepares publicly available CFPB complaint narratives for text mining and applies LDA to represent documents as mixtures of latent topics. LDA is selected because it supports both topic modeling and predictive modeling, unlike the limitations described for LSA and pLSA.
- Data Preprocessing: The preprocessing pipeline lowercases documents, removes special characters and numbers, tokenizes terms, removes stop words, stems words, and constructs a term-document matrix.The matrix records term frequencies by document, with terms as rows and documents as columns.
- Data Preprocessing: Domain-specific preprocessing removes company names, state names, and CFPB masking tokens because they are frequent but uninformative for topic modeling.Generic stop words are also removed, while stemming maps variants such as “finance,” “financial,” “finances,” and “financing” to “financ.”
- Latent Dirichlet Allocation (LDA): LDA is used instead of LSA or pLSA because it provides a generative probabilistic model and can infer topic representations for future complaint documents.The algorithm requires the number of topics K as an input.
- Latent Dirichlet Allocation (LDA): LDA models each document as a mixture of latent topics, with each topic represented as a distribution over words and topic importance varying across documents.The model estimates hidden topic distributions and per-word topic assignments from observed complaint words.
3. Topic Modeling for CFPB Consumer Complaints
The paper applies LDA to CFPB complaint narratives to extract topics, assign topic proportions to complaints, and examine topic popularity over time. Human judgment labels the extracted topics, while visualizations support exploration of assignments, semantic similarities, company-specific issues, and temporal trends.
- LDA setup: LDA preprocessing produces a term-document matrix that is analyzed in Python using inputs α, η, and the topic count K.α smooths topic distributions, η smooths word distributions, and values below one yield more concentrated distributions.
- LDA setup: The analysis selects K=40 because trial-and-error comparisons judged this setting to produce more meaningful topics.The authors prefer LDA to HDP because HDP did not perform as well in generating semantically meaningful topics.
- Topic representation: LDA outputs topics, per-document topic assignments, and topic proportions that represent the topics associated with each complaint and their relevance.The paper analyzes 86,803 consumer complaints and illustrates assignments and proportions with individual narratives.
- Topic representation: Human examination is required to assess topic coherence and meaningfulness and to assign interpretable labels to the unsupervised topics.The paper presents the resulting topic labels and their highest-probability words in Table 1.
- Topic representation: For one complaint, Auto Loan/Dealership receives the highest topic proportion at 0.325, followed by Collection Agency, Communication, Credit Reporting, and Customer Service.The narrative contains more words associated with Auto Loan/Dealership, such as vehicle, Volvo, trade in, dealer, and car.
- Semantic similarities and visualization: Topic proportions support semantic-similarity analysis because complaints with higher proportions for a topic are more semantically similar in that topic.The paper provides Tableau visualizations for topic assignments, proportions, semantic similarities, and company-specific complaint topics.
- Topic popularity over time: From March 2015 to July 2016, Credit Reporting, Rewards and Promotions, Credit Score, and CFPB generally increase, while Loan/Student Loan, Harassment, and Mortgage/Loan Modification and Foreclosure decrease.Fraudulent/Unauthorized Inquiry, Fund and Deposit, Debt Collector, and Dispute show high variability and spikes.
4. Discussion
The discussion finds that LDA produces semantically meaningful, human-interpretable topics that address limitations of CFPB’s single-label convention and reveal issues absent from its predetermined labels. Topic popularity trends also provide institution- and topic-level signals for monitoring complaints, regulations, and emerging issues.
- Topic extraction: LDA created coherent and semantically meaningful topics that generally correspond one-to-one with CFPB’s predetermined labels.A few labels correspond to combinations of topics, while the extracted topics remain semantically cohesive.
- Interpretability: The extracted topics summarize large collections of complaint narratives into a human-interpretable decomposition and help practitioners discover new complaint content.The discussion presents this as a benefit of topic modeling for CFPB complaints.
- Topic extraction: LDA captured topics such as “CFPB”, “Family”, and “Divorce and Ex-spouse” that had no corresponding CFPB labels.These topics represent issues that may have been neglected by the CFPB labeling convention.
- Labeling limitations: The approach addresses CFPB labeling limitations by assigning multiple topics to complaints that may concern multiple issues, unlike the single-label portal convention.The CFPB issue field contains 90 predetermined labels, but consumers can select only one label during submission.
- Trend analysis: Topic popularity over time can serve as a quantitative metric for evaluating CFPB regulation effectiveness and monitoring topic popularity by financial institution.The discussion links these trends to investigating institutional compliance and consumer-oriented decision making.
5. Conclusion
The paper proposes an LDA-based decision support system to extract meaningful topics from CFPB complaints, analyze their time trends, and support more efficient investigations. The analysis identifies topics with decreasing, increasing, or highly variable trends and uses these patterns to assess regulation effectiveness and detect emerging issues.
- Approach: The proposed LDA-based decision support system extracts semantically meaningful topic mixtures from CFPB consumer complaint narratives.This approach avoids relying solely on human annotation of large volumes of difficult-to-interpret narrative data.
- Trend analysis: Topic popularity and time-trend analysis can support evaluation of CFPB regulations by characterizing complaint topics as increasing, decreasing, or variable.The paper uses these trends to assess how complaint issues change over time.
- Regulatory assessment: Decreasing trends for “Harassment”, “Mortgage/Loan Modification and Foreclosure”, and “Loan/Student Loan” were interpreted as evidence of successfully enforced CFPB regulations.The conclusion specifically associates these topics with successful enforcement.
- Regulatory assessment: Increasing or highly variable trends for “Fraudulent/Unauthorized Inquiry” and “Dispute” were interpreted as indicating that CFPB regulations were not adequately effective for those issues.The cited conclusion links this assessment to the topics’ increasing or highly variable trends.
- Operational use: The approach can be operationalized through preprocessing, LDA topic modeling, and topic-popularity analysis, running monthly or daily to monitor emerging complaint topics.Recent increases in topics such as “Rewards and Promotions” and “Account Management” are given as examples of emerging-topic signals.
- Operational use: The system is designed to support rather than replace human analysts, helping CFPB staff investigate complaints more efficiently and improve consumer protection from unfair, deceptive, or abusive practices.The paper frames the proposed technology-human partnership as a support tool for existing analysts.
Appendix A
Appendix A presents a time-trend analysis for topics not included in Figure 6.
- Figure A.1 analyzes time trends for the remaining topics excluded from Figure 6.
Appendix B
Appendix B lists the CFPB dataset’s top 60 issue labels and their correspondence with LDA-extracted topics, which together cover more than 99% of complaints.
- Table B.1 maps the CFPB dataset’s top 60 issue labels to topics extracted using LDA.
- The listed issue labels include identity theft, balance transfer, loan repayment, credit-report use, unauthorized loans, account charges, and account cancellation.
- Additional labels cover debit or ATM cards, payment-crediting problems, credit decisions, credit monitoring, identity protection, and bankruptcy.
- The top 60 issues account for labels covering more than 99% of the consumer complaints in the dataset.