An Evolutionary Feature-Selection Algorithm for High-Dimensional Document Mining

International Journal of Computer Science and Technology Vol. 2, No. 1, January 2000 | pp. 1–8

DOI: 10.46882/2000/IJCST/000013

Research Article

Title: An Evolutionary Feature-Selection Algorithm for High-Dimensional Document Mining

Names of Authors: Hiroshi Tanaka¹, Elena Petrova²

Authors’ Affiliations: ¹Graduate School of Information Sciences, Tohoku University, Sendai, Japan; ²Department of Computational Linguistics, Sofia University, Sofia, Bulgaria.

Abstract:
High-dimensional text repositories present significant computational challenges for automated document classification due to the presence of irrelevant, redundant, and noisy linguistic features. Traditional vector space representations result in massive, sparse matrices that drastically degrade classifier accuracy and inflate training time. This paper introduces an optimized, evolutionary feature-selection algorithm designed to extract highly informative text tokens from massive corpus architectures. The proposed framework modifies standard genetic operators by incorporating an information-theoretic gain filter that accelerates population convergence. We tested the performance of the algorithm across a standardized dataset consisting of 100,000 multi-topic documents with an initial vocabulary space exceeding 50,000 distinct terms. The empirical results demonstrate that our evolutionary algorithm reduces the active feature space by 84.3% while simultaneously boosting classification accuracy by 6.5% when integrated with conventional Support Vector Machines and Naive Bayes classifiers. The processing speed during the feature extraction stage was reduced by a factor of 4, ensuring rapid throughput for text mining models. This architecture offers a scalable methodology for enterprise content search tools requiring immediate structural indexing.

Keywords: Text Mining, Feature Selection, Genetic Algorithms, High-Dimensional Data, Information Theory, Document Classification.

Manuscript Timeline: Received: October 10, 1999; Revised: November 15, 1999; Accepted: December 04, 1999; Published: January 20, 2000.

Citation: Tanaka, H., & Petrova, E. (2000). An Evolutionary Feature-Selection Algorithm for High-Dimensional Document Mining. International Journal of Computer Science and Technology, 2(1), 1–8.

Subscribe to read the full article: https://www.internationalscholarsjournals.org/subscribe-to-read