Modelling uncertainty in cloud workload forecasting: a Hybrid Bayesian Neural Network approach

Simple item page
dc.contributor.advisorBrown, Kenneth
dc.contributor.advisorPrestwich, Steve
dc.contributor.advisorVisentin, Andrea
dc.contributor.authorRossi, Andreaen
dc.contributor.funderScience Foundation Ireland
dc.contributor.funderHorizon 2020
dc.date.accessioned2025-02-05T09:53:37Z
dc.date.available2025-02-05T09:53:37Z
dc.date.issued2024
dc.date.submitted2024
dc.description.abstractThe exponential growth of cloud services utilisation has rendered robust and accurate workload forecasting paramount. Predicting future computational demands is essential for efficient resource management, cost optimisation, and guaranteeing service level agreement (SLA). At the same time, the increase in cloud resource demand poses new challenges related to data centres' carbon emissions and their environmental impact. However, these predictions are characterised by inherent uncertainty stemming from both fundamental limitations in data and the dynamic nature of cloud usage. This dissertation delves into workload forecasting in cloud computing. Our primary focus is quantifying predictions' inherent uncertainty, an important but often overlooked aspect. Traditionally, forecasting models provide point estimates, leaving cloud providers blind to the potential range of future demands. This lack of uncertainty awareness can lead to suboptimal resource allocation, unnecessary costs, and SLA violations. In particular, our work focuses on quantifying the uncertainty of different natures. The epistemic uncertainty arises from limitations in our knowledge or model, while the aleatoric uncertainty stems from the variability in the data itself, reflecting the inherent randomness of cloud workloads. By capturing both these forms of uncertainty, we gain a deeper understanding of the prediction's confidence interval, empowering informed decision-making. This work introduces a novel Hybrid Bayesian Neural Network (HBNN) model designed to capture both types of uncertainty in workload forecasting by incorporating a Bayesian layer at the network's end. The HBNN surpasses conventional approaches by estimating the future probability distribution to compute the associated confidence intervals. This quantifiable uncertainty empowers the model for more accurate prediction. The thesis expands upon the HBNN model by exploring its application in bivariate forecasting, simultaneously predicting processing units and memory demands. This approach proves good accuracy at the cost of more training data required. Furthermore, this work investigates the impact of various factors on forecast accuracy, including training data size, unseen data generalisation, and the potential of transfer learning across different cloud environments. Furthermore, recognising the potential computational hurdles associated with Bayesian neural networks, this work introduces a clustering-based preprocessing technique. This technique intelligently selects training data points, significantly reducing computational cost while maintaining forecast accuracy. This development enables the HBNN model to be deployed in cloud environments at scale. The thesis comprehensively evaluates the HBNN model through extensive experiments on real-world cloud workload datasets. The results demonstrate the model's superior performance in quantifying uncertainty, achieving higher accuracy than baseline models and showcasing improved generalisation on unseen data. By quantifying inherent uncertainty, HBNN empowers cloud providers to manage cloud resources more efficiently, reliably and cost-effectively.en
dc.description.statusNot peer revieweden
dc.description.versionAccepted Versionen
dc.format.mimetypeapplication/pdfen
dc.identifier.citationRossi, A. 2024. Modelling uncertainty in cloud workload forecasting: a Hybrid Bayesian Neural Network approach. PhD Thesis, University College Cork.
dc.identifier.endpage177
dc.identifier.urihttps://hdl.handle.net/10468/16964
dc.language.isoenen
dc.publisherUniversity College Corken
dc.relation.projectinfo:eu-repo/grantAgreement/SFI/SFI Centres for Research Training Programme::Data and ICT Skills for the Future/18/CRT/6223/IE/SFI Centre for Research Training in Artificial Intelligence/
dc.relation.projectinfo:eu-repo/grantAgreement/SFI/SFI-NSF Student Mobility Programme/18/CRT/6223 (S1)/IE/18/CRT/6223 Supplement/
dc.relation.projectinfo:eu-repo/grantAgreement/SFI/SFI Research Centres Programme::Phase 1/16/RC/3918/IE/Confirm Centre for Smart Manufacturing/
dc.relation.projectinfo:eu-repo/grantAgreement/SFI/SFI Maternity/Adoptive Leave Allowance/12/RC/2289-P2s/IE/INSIGHT Phase 2/
dc.relation.projectinfo:eu-repo/grantAgreement/EC/HE::HORIZON-RIA/101070141/EU/Green responsibLe privACy preservIng dAta operaTIONs/GLACIATION
dc.rights© 2024, Andrea Rossi.
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectTime series forecasting
dc.subjectCloud computing
dc.subjectUncertainty quantification
dc.subjectDeep learning
dc.subjectTime series snalysis
dc.subjectDeep learning
dc.subjectWorkload prediction
dc.subjectWorkload forecasting
dc.titleModelling uncertainty in cloud workload forecasting: a Hybrid Bayesian Neural Network approach
dc.typeDoctoral thesisen
dc.type.qualificationlevelDoctoralen
dc.type.qualificationnamePhD - Doctor of Philosophyen
Files
Original bundle
Now showing 1 - 2 of 2
Thumbnail Image
Name:
RossiA_PhD2024.pdf
Size:
4.43 MB
Format:
Adobe Portable Document Format
Description:
Full Text E-thesis
Download
Thumbnail Image
Name:
RossiA_PhD2024_Submission for examination form.pdf
Size:
798.24 KB
Format:
Adobe Portable Document Format
License bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
license.txt
Size:
5.2 KB
Format:
Item-specific license agreed upon to submission
Description:
Download
Collections
Research Theses
College of Science, Engineering and Food Science - Doctoral Theses
Computer Science - Doctoral Theses