Gaussian Process assisted Particle Swarm Optimization

Abstract

Real-world optimization problems often are non-convex, non-differentiable and highly multimodal, which is why stochastic, population-based metaheuristics are frequently applied. If the optimization problem is also computationally very expensive, only relatively few function evaluations can be afforded. We develop a model-assisted optimization approach as a coupling of Gaussian Process modeling, a regression technique from machine learning, with the Particle Swarm Optimization metaheuristic. It uses earlier function evaluations to predict areas of improvement and exploits the model information in the heuristic search. Under the assumption of a costly target function, it is shown that model-assistance improves the performance across a set of standard benchmark functions. In return, it is possible to reduce the number of target function evaluations to reach a certain fitness level to speed up the search.

BibTeX

@inproceedings{Kron10GPPSO,
  author = {Kronfeld, Marcel and Zell, Andreas},
  title = {Gaussian Process assisted Particle Swarm Optimization},
  booktitle = {Learning and Intelligent Optimization Conference (LION IV)},
  year = {2010},
  editor = {Blum, Christian and Battiti, Roberto},
  number = {6073},
  series = {Lecture Notes in Computer Science, LNCS},
  pages = {139-153},
  address = {Venice, Italy},
  month = jan,
  publisher = {Springer Verlag},
  abstract = {Real-world optimization problems often are non-convex, non-differentiable
	and highly multimodal, which is why stochastic, population-based
	metaheuristics are frequently applied. If the optimization problem
	is also computationally very expensive, only relatively few function
	evaluations can be afforded. We develop a model-assisted optimization
	approach as a coupling of Gaussian Process modeling, a regression
	technique from machine learning, with the Particle Swarm Optimization
	metaheuristic. It uses earlier function evaluations to predict areas
	of improvement and exploits the model information in the heuristic
	search. Under the assumption of a costly target function, it is shown
	that model-assistance improves the performance across a set of standard
	benchmark functions. In return, it is possible to reduce the number
	of target function evaluations to reach a certain fitness level to
	speed up the search.},
}