Using String Kernels for Classification of Slovenian Web Documents
- 1.6k Downloads
In this paper we present an approach for classifying web pages obtained from the Slovenian Internet directory where the web sites covering different topics are organized into a topic ontology. We tested two different methods for representing text documents, both in combination with the linear SVM classification algorithm. The first representation used is a standard bag-of-words approach with TFIDF weights and cosine distance used as similarity measure. We compared this to String kernels where text documents are compared not by words but by substrings. This removes the need for stemming or lemmatisation which can be an important issue when documents are in other languages than English and tools for stemming or lemmatisation are unavailable or are expensive to make or learn. In highly inflected natural languages, such as Slovene language, the same word can have many different forms, thus String kernels have an advantage here over the bag-of-words. In this paper we show that in classification of documents written in highly inflected natural language the situation is opposite and String Kernels significantly outperform the standard bag-of-words representation. Our experiments also show that the advantage of String kernels is more evident for domains with unbalanced class distribution.
Unable to display preview. Download preview PDF.
- B.E. BOSER, I.M. GUYON, and V.N. VAPNIK (1992): Proc. 5th Annual ACM Workshop on Computational Learning Theory, 144–152. Pittsburgh, PA, July 1992. ACM PressGoogle Scholar
- J. BRANK, M. GROBELNIK, N. MILIC-FRAYLING, D. MLADENIC (2003): Training text classifiers with SVM on very few positive examples, Technical repot, MSR-TR-2003-34.Google Scholar
- T. JOACHIMS (1999): Making large-scale svm learning practical. In: B. Scholkopf, C. Burges, and A. Smola (eds.): Advances in Kernel Methods-Support Vector Learning. MIT-Press.Google Scholar
- D. MLADENIC and M. GROBELNIK (2003): Feature selection on hierarchy of web documents. Journal of Decision Support Systems, 35(1): 45–87.Google Scholar
- K. MORIK, P. BROCKHAUSEN, and T. JOACHIMS (1999): Combining statistical learning with a knowledge-based approach — A case study in intensive care monitoring. Int. Conf. Machine LearningGoogle Scholar
- J. PLISSON, N. LAVRAC, and D. MLADENIC (2004): A rule based approach to word lemmatization. Proc. 7th Int. Conf. Information Society IS-2004, 83–86. Ljubljana: Institut Jozef Stefan.Google Scholar
- C. SAUNDERS, H. TSCHACH, and J. SHAWE-TAYLOR (2002): Syllables and Other String Kernel extensions. Proc. 19th Int. Conf. Machine LearningGoogle Scholar