Chinese Short Text Categorization Based on Semi-Supervised Learning

Abstract

Most of the text on the Internet is unlabelled with the rapid development of the Internet, and it is difficult for us to classify the unlabelled text accurately under the condition of insufficient labelled samples. Sei-supervised learning is a method, which combines the labelled samples with the unlabelled samples, can solve the problem in a better way. AdaBoost is one of the most representative algorithm of boosting algorithms, and this paper used the improved decision tree to be weak classifiers of the AdaBoost. Based on this, this paper devised a boosting algorithm which was based on semi-supervised learning and the improved decision tree. The algorithm is devoted to solving the problem of the Chinese short text categorization under the condition of insufficient labelled samples. Experiments show that the algorithm can effectively improve the performance of the Chinese short text categorization on balanced and imbalanced data sets.