Forecasting workload in cloud computing: towards uncertainty-aware predictions and transfer learning
| dc.contributor.author | Rossi, Andrea | en |
| dc.contributor.author | Visentin, Andrea | en |
| dc.contributor.author | Carraro, Diego | en |
| dc.contributor.author | Prestwich, Steven D. | en |
| dc.contributor.author | Brown, Kenneth N. | en |
| dc.contributor.funder | Science Foundation Ireland | en |
| dc.contributor.funder | European Commission | en |
| dc.contributor.funder | Google Cloud | en |
| dc.date.accessioned | 2025-10-24T14:23:04Z | |
| dc.date.available | 2025-10-24T14:23:04Z | |
| dc.date.issued | 2025-02-25 | en |
| dc.description.abstract | Accurately forecasting workload demand in cloud computing environments is essential for optimizing resource allocation, minimizing costs, and ensuring reliable service quality. As cloud computing scales to meet the needs of diverse applications - from AI and machine learning to data-intensive analytics - predictive models play a critical role in dynamically managing multiple resources. Traditional models provide limited guidance for decision-makers since they are typically univariate models, ignoring the prediction of the interplay between multiple resources, and do not account for the uncertainty of their predictions, preventing resource management from acting promptly according to such uncertainty to ensure specific target service level requirements. To address these limitations, we introduce univariate and bivariate Bayesian deep learning models that predict future workload demand of one and multiple resources respectively, while quantifying the uncertainty of their predictions. In particular, our approach leverages Hybrid Bayesian Neural Networks and probabilistic Long Short-Term Memory models, enhanced with architecture modifications to handle complex, multivariate cloud workload patterns. Moreover, we investigate fine-tuning-based transfer learning methods to enhance their adaptability in real-world cloud scenarios where new data centres with different workload characteristics operate. We validate our models on extensive datasets from Google and Alibaba cloud clusters. Results show that modelling the uncertainty of predictions positively impacts performance, especially on service level metrics, because uncertainty quantification can be tailored to desired target service levels that are critical in cloud applications. Moreover, transfer learning benefits performance in scenarios where models are built on data from the same provider. | en |
| dc.description.sponsorship | Google Cloud (Research Credits program GCP203677602) | en |
| dc.description.status | Peer reviewed | en |
| dc.description.version | Published Version | en |
| dc.format.mimetype | application/pdf | en |
| dc.identifier.articleid | 258 | en |
| dc.identifier.citation | Rossi, A., Visentin, A., Carraro, D., Prestwich, S. and Brown, K.N. (2025) 'Forecasting workload in cloud computing: towards uncertainty-aware predictions and transfer learning', Cluster Computing, 28(4), 258 (20pp). https://doi.org/10.1007/s10586-024-04933-2 | en |
| dc.identifier.doi | 10.1007/s10586-024-04933-2 | en |
| dc.identifier.eissn | 1573-7543 | en |
| dc.identifier.endpage | 20 | en |
| dc.identifier.issn | 1386-7857 | en |
| dc.identifier.issued | 4 | en |
| dc.identifier.journaltitle | Cluster Computing | en |
| dc.identifier.startpage | 1 | en |
| dc.identifier.uri | https://hdl.handle.net/10468/18076 | |
| dc.identifier.volume | 28 | en |
| dc.language.iso | en | en |
| dc.publisher | Springer Nature | en |
| dc.relation.project | info:eu-repo/grantAgreement/SFI/Centres for Research Training (CRT) Programme/18/CRT/6223/IE/SFI Centre for Research Training in Artificial Intelligence/ | en |
| dc.relation.project | info:eu-repo/grantAgreement/SFI/Research Centres Programme::Phase 2/12/RC/2289_P2/IE/INSIGHT_Phase 2 / | en |
| dc.relation.project | info:eu-repo/grantAgreement/EC/HE::HORIZON-RIA/101070141/EU/Green responsibLe privACy preservIng dAta operaTIONs/GLACIATION | en |
| dc.relation.project | info:eu-repo/grantAgreement/EC/H2020::RIA/952215/EU/Foundations of Trustworthy AI - Integrating Reasoning, Learning and Optimization/TAILOR | en |
| dc.rights | © 2024, the Authors. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons. org/licenses/by/4.0/. | en |
| dc.rights.uri | https://creativecommons.org/licenses/by/4.0/ | en |
| dc.subject | Bayesian neural networks | en |
| dc.subject | Cloud computing | en |
| dc.subject | Workload prediction | en |
| dc.subject | Uncertainty | en |
| dc.subject | Deep learning | en |
| dc.subject | Transfer learning | en |
| dc.title | Forecasting workload in cloud computing: towards uncertainty-aware predictions and transfer learning | en |
| dc.type | Article (peer-reviewed) | en |
