Multivariate Predictive Clustering Trees for Classification

  • Tomaž StepišnikEmail author
  • Dragi Kocev
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12117)


Decision trees are well established machine learning models that combined in ensembles produce state-of-the-art predictive performance. Predictive clustering trees are a generalization of standard classification and regression trees towards structured output prediction and semi-supervised learning. Most of the research attention is on univariate decision trees, whereas multivariate decision trees, in which multiple attributes can appear in a test, are less widely used. In this paper, we present a multivariate variant of predictive clustering trees, and experimentally evaluate it on 12 classification tasks. Our method shows good predictive performance and computational efficiency, and we illustrate its potential for performing feature ranking.


Predictive clustering trees Multivariate decision trees Classification Multi-label classification 


  1. 1.
    Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)zbMATHGoogle Scholar
  2. 2.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)zbMATHCrossRefGoogle Scholar
  3. 3.
    Breiman, L., Friedman, J., Olshen, R., Stone, C.J.: Classification and Regression Trees. Chapman & Hall/CRC, San Francisco (1984)zbMATHGoogle Scholar
  4. 4.
    Debeljak, M., Squire, G.R., Kocev, D., Hawes, C., Young, M.W., Džeroski, S.: Analysis of time series data on agroecosystem vegetation using predictive clustering trees. Ecol. Model. 222(14), 2524–2529 (2011)CrossRefGoogle Scholar
  5. 5.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014)Google Scholar
  6. 6.
    Kocev, D., Vens, C., Struyf, J., Džeroski, S.: Tree ensembles for predicting structured outputs. Pattern Recogn. 46(3), 817–833 (2013)CrossRefGoogle Scholar
  7. 7.
    Menze, B.H., Kelm, B.M., Splitthoff, D.N., Koethe, U., Hamprecht, F.A.: On oblique random forests. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011. LNCS (LNAI), vol. 6912, pp. 453–469. Springer, Heidelberg (2011). Scholar
  8. 8.
    Murthy, S.K., Kasif, S., Salzberg, S.: A system for induction of oblique decision trees. J. Artif. Intell. Res. 2, 1–32 (1994)zbMATHCrossRefGoogle Scholar
  9. 9.
    Prabhu, Y., Varma, M.: FastXML: a fast, accurate and stable tree-classifier for extreme multi-label learning. In: ACM-Association for Computing Machinery (2014)Google Scholar
  10. 10.
    Slavkov, I., Gjorgjioski, V., Struyf, J., Džeroski, S.: Finding explained groups of time-course gene expression profiles with predictive clustering trees. Mol. BioSyst. 6(4), 729–740 (2010)CrossRefGoogle Scholar
  11. 11.
    Struyf, J., Džeroski, S., Blockeel, H., Clare, A.: Hierarchical multi-classification with predictive clustering trees in functional genomics. In: Bento, C., Cardoso, A., Dias, G. (eds.) EPIA 2005. LNCS (LNAI), vol. 3808, pp. 272–283. Springer, Heidelberg (2005). Scholar
  12. 12.
    Tsoumakas, G., Katakis, I., Vlahavas, I.: Mining multi-label data. In: Maimon, O., Rokach, L. (eds.) Data Mining and Knowledge Discovery Handbook, pp. 667–685. Springer, Boston (2010). Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Jožef Stefan InstituteLjubljanaSlovenia
  2. 2.Jožef Stefan International Postgraduate SchoolLjubljanaSlovenia
  3. 3.Bias Variance Labs, d.o.o.LjubljanaSlovenia

Personalised recommendations