BHO-MA: Bayesian Hyperparameter Optimization with Multi-objective Acquisition
Loading...
Files
Accepted Version
Date
2024-02-01
Authors
Dogan, Vedat
Prestwich, Steven
Journal Title
Journal ISSN
Volume Title
Publisher
Springer Nature
Published Version
Abstract
Good hyperparameter values are crucial for the performance of machine learning models. In particular, poorly chosen values can cause under- or overfitting in regression and classification. A common approach to hyperparameter tuning is grid search, but this is crude and computationally expensive, and the literature contains several more efficient automatic methods such as Bayesian optimization. In this work, we develop a Bayesian hyperparameter optimization technique with more robust performance, by combining several acquisition functions and applying a multi-objective approach. We evaluated our method using both classification and regression tasks. We selected four data sets from the literature and compared the performance with eight popular methods. The results show that the proposed method achieved better results than all others.
Description
Keywords
Bayesian optimization , Multi-objective optimization , Hyperparameter tuning
Citation
Dogan, V. and Prestwich, S. (2024) 'BHO-MA: Bayesian Hyperparameter Optimization with Multi-objective Acquisition', in Pereira, A. I., Mendes, A., Fernandes, F. P., Pacheco, M. F., Coelho, J. P. and Lima, J. (eds) Optimization, Learning Algorithms and Applications. OL2A 2023. Communications in Computer and Information Science, vol 1981, pp 391-408. Springer, Cham. https://doi.org/10.1007/978-3-031-53025-8_27