Multifidelity Bayesian optimization for hyperparameter tuning of deep reinforcement learning algorithms

E.C. Garrido-Merchán, M. Molina, G. Martínez

Computing and Artificial Intelligence Vol. 3, nº. 2, pp. 2923-1 - 2923-13

Summary:

This research focuses on comparing standard Bayesian optimization and multifidelity Bayesian optimization in the hyperparameter search to improve the performance of reinforcement learning algorithms in environments such as OpenAI LunarLander and CartPole. The primary goal is to determine whether multifidelity Bayesian optimization provides significant improvements in solution quality compared to standard Bayesian optimization. To address this question, several Python implementations were developed, evaluating the solution quality using the mean of the total rewards obtained as the objective function. Various experiments were conducted for each environment and version using different seeds, ensuring that the results were not merely due to the inherent randomness of reinforcement learning algorithms. The results demonstrate that multifidelity Bayesian optimization outperforms standard Bayesian optimization in several key aspects. In the LunarLander environment, multifidelity optimization achieved better convergence and more stable performance, yielding a higher average reward compared to the standard version. In the CartPole environment, although both methods quickly reached the maximum reward, multifidelity did so with greater consistency and in less time. These findings highlight the ability of multifidelity optimization to optimize hyperparameters more efficiently, using fewer resources and less time while achieving superior performance.

Spanish layman's summary:

Este estudio muestra que la optimización bayesiana multifidelidad mejora el ajuste de hiperparámetros en algoritmos de refuerzo, logrando mejores resultados con menos recursos que la versión estándar.

English layman's summary:

This study shows that multifidelity Bayesian optimization improves hyperparameter tuning in reinforcement learning, achieving better results with fewer resources than the standard approach.

Keywords: deep reinforcement learning; bayesian optimization; meta learning

DOI reference: https://doi.org/10.59400/cai2923

Published on paper: 2025.

Published on-line: May 2025.

Citation:
E.C. Garrido-Merchán, M. Molina, G. Martínez, Multifidelity Bayesian optimization for hyperparameter tuning of deep reinforcement learning algorithms. Computing and Artificial Intelligence. Vol. 3, nº. 2, pp. 2923-1 - 2923-13, 2025. [Online: May 2025]