Source-linked AI summary
Machine learning approach for text and document mining
Vishwanath Bijalwan, Pinki Kumari, Jordan Pascual, Vijay Bhaskar Semwal
TL;DR
Text categorization assigns documents to predefined categories, but comparing classification approaches is important for selecting a suitable method. The paper applies KNN, Naive Bayes, and term-graph methods after preprocessing and term-association mining, then evaluates classification and retrieves relevant documents. It reports that KNN achieves the highest accuracy, despite higher time complexity.
Problem
Text categorization must assign documents to predefined categories using machine-learning and information-retrieval methods.
Method
The paper preprocesses documents, mines frequent term associations for term graphs, applies KNN, Naive Bayes, and term-graph classification, and builds a retrieval application.
Results
KNN achieves the highest accuracy compared with Naive Bayes and Term-Graph classification on the evaluated Reuters 21578 articles.
Takeaways & Limitations
KNN is reported as more suitable for the paper’s text and document mining task, while the system returns relevant documents for entered queries.
Abstract
from arXiv · showhide
Text Categorization (TC), also known as Text Classification, is the task of automatically classifying a set of text documents into different categories from a predefined set. If a document belongs to exactly one of the categories, it is a single-label classification task; otherwise, it is a multi-label classification task. TC uses several tools from Information Retrieval (IR) and Machine Learning (ML) and has received much attention in the last years from both researchers in the academia and industry developers. In this paper, we first categorize the documents using KNN based machine learning approach and then return the most relevant documents.
1. Introduction
The paper examines text categorization and document retrieval using machine-learning and information-retrieval methods, focusing on KNN alongside Naive Bayes and term-graph approaches. It proposes preprocessing documents, mining frequent term associations, constructing graphs, classifying documents, and returning relevant results for user queries.
- Motivation: The paper surveys six classification families, including Rocchio, KNN, regression, Naive Bayes, decision trees, and decision rules.
- Objectives: The proposed application classifies uploaded news articles and retrieves the most relevant document for user-entered keywords.
- Motivation: Text categorization predicts a new document’s category from training documents with known categories.
- Naive Bayes: Naive Bayes estimates category probabilities from term occurrences while assuming terms occur independently.
- Term Graph Model: The term-graph model preprocesses documents, mines frequent co-occurring terms, and preserves their associations in a weighted graph.
- KNN and Retrieval: KNN classifies queries using the categories of nearby documents, while the system also computes keyword similarities and distances for retrieval.
3. Experimental Results
The experiment preprocesses Reuters-21578 documents, compares KNN, Term Graph, and Naïve Bayes classifiers, and evaluates them using accuracy before deploying the highest-accuracy approach for document retrieval.
- Dataset: Reuters-21578 provides 9,603 training documents, 3,299 test documents, and 8,676 unused documents across five selected categories.
- Preprocessing: Documents were preprocessed with bag-of-words representation, stop-word removal, TF-IDF, case folding, and normalization.
- Evaluation: KNN, Term Graph, and Naïve Bayes classified the training documents, then test accuracy was calculated against the provided answers.
- Evaluation: Accuracy measures the percentage of correctly classified documents and is generally used for single-label text-categorization tasks.
- Application: The application uses the higher-accuracy algorithm to return relevant documents for user-entered keywords.
- Results: KNN achieved the best accuracy among Naïve Bayes, Term Graph, and KNN on the Reuters-21578 articles.
4. Conclusion
The paper concludes that KNN provides the highest accuracy, while noting its high time complexity and describing a hybrid Term-Graph implementation and a retrieval application.
- KNN achieves maximum accuracy compared with Naïve Bayes and Term-Graph methods.
- KNN has high time complexity but delivers better accuracy than the other evaluated methods.
- The authors implemented Term-Graph with methods beyond the traditional AFOPT combination and report that the hybrid performs better than that traditional combination.
- An information-retrieval application uses the Vector Space Model to return relevant documents for client queries.
- Future work will target reduced complexity, increased accuracy, and text summarization.
Authors
Vishwanath Bijalwan is an assistant professor whose research interests include wireless sensor networks, WiMax, Wi-Fi, machine learning, and information retrieval.
- Vishwanath Bijalwan is an assistant professor at Uttarakhand Technical University Dehradun.
- His research interests include wireless sensor networks, WiMax, Wi-Fi, machine learning, and information retrieval.
- He has five years of research experience and has published four research papers.