Bias Management of Bayesian Network Classifiers

  • Gladys Castillo
  • João Gama
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3735)


The purpose of this paper is to describe an adaptive algorithm for improving the performance of Bayesian Network Classifiers (BNCs) in an on-line learning framework. Instead of choosing a priori a particular model class of BNCs, our adaptive algorithm scales up the model’s complexity by gradually increasing the number of allowable dependencies among features. Starting with the simple Naïve Bayes structure, it uses simple decision rules based on qualitative information about the performance’s dynamics to decide when it makes sense to do the next move in the spectrum of feature dependencies and to start searching for a more complex classifier. Results in conducted experiments using the class of Dependence Bayesian Classifiers on three large datasets show that our algorithm is able to select a model with the appropriate complexity for the current amount of training data, thus balancing the computational cost of updating a model with the benefits of increasing in accuracy.


Bias Management Bayesian Classifiers Machine Learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bouckaert, R.: Bayesian Network Classifiers in Weka (2004), Technical Report 14/2004. Computer Science Department. University of Waikato (2004)Google Scholar
  2. 2.
    Brian, D., Webb, G.: The need for Low Bias Algorithms in Classification Learning from Large Data Sets. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) PKDD 2002. LNCS (LNAI), vol. 2431, pp. 62–73. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  3. 3.
    Buntine, W.: Theory Refinement on Bayesian Networks. In: Proceedings of the 7th Conference on Uncertainty in Artificial Intelligence, pp. 52–60 (1991)Google Scholar
  4. 4.
    Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian Network Classifiers. Machine Learning 4, 131–161 (1997)CrossRefGoogle Scholar
  5. 5.
    Gama, J.: Iterative Bayes. Intelligent Data Analysis 4, 475–488 (2000)zbMATHGoogle Scholar
  6. 6.
    Kohavi, R., Wolpert, D.: Bias Plus Variance Decomposition for Zero-One Loss Functions. In: Proceedings of the 13th International Conference on Machine Learning (ICML 1996). Morgan Kaufmann Publishers, San Francisco (1999)Google Scholar
  7. 7.
    Kontkanen, P., Myllymaki, P., Silander, T., Tirr, H.: On Supervised Selection of Bayesian Networks. In: Proceedings of the Fifteenth International Conference on Uncertainty in Artificial Intelligence (UAI 1999), pp. 334–342. Morgan Kaufmann Publishers, San Francisco (1999)Google Scholar
  8. 8.
    Quinlan, R.: C4.5 Programs for Machine Learning. Morgan Kaufmann Publishers, San Francisco (1993)Google Scholar
  9. 9.
    Ripley, B.: Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge (1996)zbMATHGoogle Scholar
  10. 10.
    Roure, J., Sangüesa, R.: Incremental Methods for Bayesian Network Learning. Research Report LSI-99-42-R. Software Department. Technical University of Catalonia (1999)Google Scholar
  11. 11.
    Sahami, M.: Learning Limited Dependence Bayesian Classifiers. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, pp. 335–338. AAAI Press, Portland (1996)Google Scholar
  12. 12.
    Sen, P.K.: Estimates of the regression coefficient based on Kendall’s tau. Journal of the American Statistical Association 63, 1379–1389 (1968)zbMATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Gladys Castillo
    • 1
    • 2
  • João Gama
    • 1
    • 3
  1. 1.LIACCUniversity of PortoPortugal
  2. 2.Department of MathematicsUniversity of AveiroPortugal
  3. 3.FEPUniversity of PortoPortugal

Personalised recommendations