Forecasting Software Reliability

  • Bev Littlewood
Part of the Topics in Safety, Reliability and Quality book series (TSRX, volume 1)


Computer software fails because of the presence of intellectual faults, ranging from simple coding faults to fundamental design faults. In principle, such faults can be permanently removed when they are detected by failure of the software. Then the software will exhibit reliability growth. The problem considered here is the one of forecasting this growth: it includes the estimation of the current reliability of the program from the previous failure data. We begin with a brief description of the software failure process: a non-stationary stochastic process. Several of the best-known software reliability growth models are described, and examples given of their performance on real software failure data. They shown marked disagreement and thus reveal a need for methods of comparing and evaluating software reliability forecasts. Several simple techniques for conducting this evaluation are described and illustrated using several different models on real data sets. Finally, it is shown how in certain circumstances it is possible to improve the predictive accuracy of software reliability models by a re-calibration technique.


Prediction System Software Reliability Adaptive Procedure Predictive Quality Reliability Growth 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abdel Chaly, A.A. (1986), Ph D Thesis, City University, London.Google Scholar
  2. Adams, E.N. (1984), Optimizing preventive service of software products, IBM Journal of Research and Development, 28 (No 1).Google Scholar
  3. Aitchison, J. & Dunsmore, I.R. (1975), Statistical Prediction Analysis, Cambridge University Press, Cambridge.zbMATHCrossRefGoogle Scholar
  4. Akaike, H. (1982), Prediction and Entropy, MRC Technical Summary Report, Mathematics Research Center, University of Wisconsin-Madison.Google Scholar
  5. Amman, P.E. & Knight, J.C. (1987), Data diversity: an approach to software fault tolerance, Digest FTCS-17 (17th International Symposium on Fault-Tolerant Computing), 122–126.Google Scholar
  6. Ascher, H. & Feingold, H. (1984), Repairable Systems Reliability, Marcel Dekker, New York.zbMATHGoogle Scholar
  7. Braun, H. & Paine, J.M. (1977), A comparative study of models for reliability growth, Technical Report No 126, Series 2, Department of Statistics, Princeton University.Google Scholar
  8. Brocklehurst, S. (1987), On the effectiveness of adaptive software reliability modelling, CSR Technical Report, City University, London.Google Scholar
  9. Chan, P.Y., Littlewood, B. & Snell, J. (1985), Parametric spline approach to adaptive reliability modelling, CSR Technical Report, City University, London.Google Scholar
  10. Cox, D.R. & Lewis, P.A.W. (1966), The Statistical Analysis of Series of Events, Methuen, London.zbMATHGoogle Scholar
  11. Crow, L.H. (1977), Confidence interval procedures for reliability growth analysis, Technical Report No197, US Army Material Systems Analysis Activity, Aberdeen, Md.Google Scholar
  12. Dale, C.J. (1982), Software Reliability Evaluation Methods, British Aerospace Dynamics Group, ST-26750.Google Scholar
  13. Dawid, A.P. (1982), The well-calibrated Bayesian, (with discussion), Journal of the American Statistical Association, 77, 605–613.MathSciNetzbMATHCrossRefGoogle Scholar
  14. Dawid, A.P. (1984a), Calibration-based empirical probability, Research Report36, Department of Statistical Science, University College, London.Google Scholar
  15. Dawid, A.P. (1984b), Statistical theory: the prequential approach, Journal of the Royal Statistical Society, A, 147, 278–292.MathSciNetzbMATHCrossRefGoogle Scholar
  16. Dawid, A.P. (1989), Probability Forecasting. In: S. Kotz, N. L. Johnson and C. B. Read (Eds), Encyclopedia of Statistical Sciences, Vol 6, Wiley, New York.Google Scholar
  17. Duane, J.T. (1964), Learning curve approach to reliability monitoring, IEEE Transactions on Aerospace, 2, 563–566.CrossRefGoogle Scholar
  18. Forman, E.H. & Singpurwalla, N.D. (1977), An empirical stopping rule for debugging and testing computer software, Journal of the American Statistical Association, 72, 750–757.Google Scholar
  19. Coel, A.L. & Okumoto, K. (1979), Time-dependent error-detection rate model for software reliability and other performance measures, IEEE Transactions on Reliability, 28, 206–211.CrossRefGoogle Scholar
  20. Goudie, I. (1984), private communication.Google Scholar
  21. Jelinski, Z. & Moranda, P.B. (1972), Software reliability research. In:W. Freiberger (Ed), Statistical Computer Performance Evaluation, Academic Press, New York, 465–484.Google Scholar
  22. Joe, H. & Reid, N. (1985), Estimating the number of faults in a system, Journal of the American Statistical Association, 80, 222–226.MathSciNetCrossRefGoogle Scholar
  23. Keiller, P.A., Littlewood, B., Miller, D.R. & Sofer, A. (1983a), On the quality of software reliability predictions, Proc. NATO ASI on Electronic Systems Effectiveness and Life Cycle Costing (Norwich, UK, 1982 ), Springer, Berlin, 441–460.Google Scholar
  24. Keiller, P.A., Littlewood, B., Miller, D.R. & Sofer, A. (1983b), Comparison of software reliability predictions, Digest FTCS 13 (13th International Symposium on Fault-Tolerant Computing), 128–134.Google Scholar
  25. Kendall, M.G. & Stuart, A. (1961), The Advanced Theory of Statistics, Griffin, London.Google Scholar
  26. Langberg, N. & Singpurwalla, N.D. (1981), A unification of some software reliability models via the Bayesian approach, Technical Report, TM-66571, The George Washington University, Washington DC.Google Scholar
  27. Laprie, J.C. (1984), Dependability evaluation of software systems in operation, IEEE Transactions on Software Engineering, 10.Google Scholar
  28. Littlewood, B. (1979), How to measure software reliability and how not to, IEEE Transactions on Reliability, 28, 103–110.CrossRefGoogle Scholar
  29. Littlewood, B. (1981), Stochastic reliability growth: a model for fault-removal in computer programs and hardware designs, IEEE Transactions on Reliability, 30, 313–320.CrossRefGoogle Scholar
  30. Littlewood, B. & Keiller, P.A. (1984), Adaptive software reliability modelling, Digest FTCS-14 (14th International Conference on Fault-Tolerant Computing), 108–113.Google Scholar
  31. Littlewoon, B. & Sofer, A. (1985), A Bayesian modification to the Jelinski-Moranda software reliability growth model, CSR Technical Report. Google Scholar
  32. Littlewood, B. & Verrall, J.L. (1973), A Bayesian reliability growth model for computer software, (Applied Statistics), 22, 332–346.MathSciNetCrossRefGoogle Scholar
  33. littlewood, B. & Verrall, J.L. (1981), On the likelihood function of a debugging model for computer software reliability, IEEE Transactions on Reliability, 30, 145–148.zbMATHCrossRefGoogle Scholar
  34. Miller, D.R. (1983), private communication.Google Scholar
  35. Miller, D.R. (1986), Exponential order statistic models of software reliability growth, IEEE Transactions on Software Engineering, 12, 12–24.zbMATHCrossRefGoogle Scholar
  36. Musa, J.D. (1975), A theory of software reliability and its application, IEEE Transactions on Software Engineering, 1, 312–327.CrossRefGoogle Scholar
  37. Musa, J.D. (1979), Software reliability data, report available from Data and Analysis Center for Software, Rome Air Development Center, Rome, NY.Google Scholar
  38. Nagel, P.M. & Skrivan, J.A. (1981), Software reliability: repetitive run experimentation and modelling, BCS-40399(Dec.), Boeing Computer Services Company, Seattle, Washington.Google Scholar
  39. Rosenblatt, M. (1952), Remarks on a multivariate transformation, Annals of Mathematical Statistics, 23, 470–472.MathSciNetzbMATHCrossRefGoogle Scholar
  40. Shooman, M. (1973), Operational testing and software reliability during program development, in Record. 1973 IEEE Symp. Computer Software Reliability (New York, 1973, April 30-May 2), 51–57.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 1991

Authors and Affiliations

  • Bev Littlewood
    • 1
  1. 1.Centre for Software ReliabilityCity UniversityLondonUK

Personalised recommendations