Efficient Parameter Optimization Based on Combination of Direct Global and Local Search Methods

  • Michael Syrjakow
  • Helena Szczerbicka
Part of the The IMA Volumes in Mathematics and its Applications book series (IMA, volume 111)


In this paper we focus on direct optimization methods which require nothing but goal function values for orientation (blind search). On the one hand this property ensures robustness and universal applicability. On the other hand direct optimization usually requires a lot of computational effort (goal function evaluations) to ensure optimization success (convergence towards a globally-optimal region of the search space) and an acceptable quality of the optimization result (small approximation error). These fundamental drawbacks of direct optimization are due to the fact that no auxiliary information like derivatives or other problem specific knowledge is exploited to accelerate the optimization process. In order to substantially improve the performance of direct optimization we propose a combination of probabilistic global and deterministic local optimization methods. The resulting combined 2-phase optimization strategy has been proven to be both powerful and efficient. The excellent heuristic properties of this hybrid method allows to use it as the basic component of a multiple-stage optimization strategy. The goal of multiple-stage optimization is to perform a systematic analysis of the most important extreme points of a given optimization problem. This paper presents the structure of our developed optimization algorithms. Beyond that, results of an extensive optimization experiment are presented showing the potentialities but also the limits of our developed methods.


Goal Function Problem Dimension Direct Optimization Optimization Stage Optimization Success 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    E. Aants, J. Korst, Simulated Annealing and Boltzmann Machines, Wiley, 1990.Google Scholar
  2. [2]
    D.E. Goldberg Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, 1989.Google Scholar
  3. [3]
    R.A. Hooks, T.A. Jeeves, Direct Search Solutionfor Numerical and Statistical Problems, Journal ACM 8, pp. 212–221, 1961.CrossRefGoogle Scholar
  4. [4]
    K.-P. Huber, M.R. Berthold, Building Precise Classifiers with Automatic Rule Extraction, Proceedings of the IEEE International Conference on Neural Networks ICNN’95, Perth, Western Australia, Vol. 3, pp. 1263–1268, 1995.Google Scholar
  5. [5]
    Z. Michalewicz, Genetic Algorithms d- Data Structures = Evolution Programs, Springer, 1992.Google Scholar
  6. [6]
    J.D. Schaffer, R.A. Caruana, L.J. Eshelman, R. Das, A Study of Control Parameters Affecting Online Performance of Genetic Algorithms for Function Optimization, Proceedings of the third International Conference on Genetic Algorithms, June 4–7, George Mason University, pp. 51–60, 1989.Google Scholar
  7. [7]
    H.-P. Schwefel, Numerical Optimization of Computer Models,Wiley, 1981.zbMATHGoogle Scholar
  8. [8]
    J. Shekel, Test Functions for Multimodal Search Techniques, in Fifth Annual Princeton Conference on Information Science and Systems, 1971.Google Scholar
  9. [9]
    M. Syrjakow, H. Szczekbrka, Optimization of Simulation Models with REMO, Proceedings of the European Simulation Multiconference ESM’94, Barcelona, Spain, June 1–3, pp. 274–281, 1994.Google Scholar
  10. [10]
    M. Syrjakow, H. Szczerbicka, Simulation and Optimization of Complex Technical Systems, Proceedings of the 1995 Summer Computer Simulation Conference SCSC’95, Ottawa, Ontario, Canada, July 24–26, pp. 86–95, 1995.Google Scholar
  11. [11]
    M. Sykjnkow, H. Szczerbicka, Combination of Direct Global and Local Optimization Methods, Proceedings of the International Conference on Evolutionary Computing ICEC’95, Perth, Western Australia, November 29 - December 1, pp. 326–333, 1995.Google Scholar
  12. [12]
    M. Syrjakow, H. Szozerbroka, M.R. Berthold, K.-P. Huger, Acceleration of Direct Model Optimization Methods by Function Approximation,Proceedings of the 8th European Simulation Symposium ESS’96, Genoa, Italy, October 2426, Volume II, pp. 181–186, 1996.Google Scholar

Copyright information

© Springer Science+Business Media New York 1999

Authors and Affiliations

  • Michael Syrjakow
    • 1
  • Helena Szczerbicka
    • 2
  1. 1.Institute for Computer Design and Fault Tolerance (Prof. D. Schmid)University of KarlsruheKarlsruheGermany
  2. 2.Department of Computer Science (Fb3 Informatik)University of BremenBremenGermany

Personalised recommendations