A cloud reservation system for big data applications

Show simple item record

dc.contributor.author Marinescu, Dan C.
dc.contributor.author Paya, Ashkan
dc.contributor.author Morrison, John P.
dc.date.accessioned 2019-08-29T14:19:09Z
dc.date.available 2019-08-29T14:19:09Z
dc.date.issued 2017-03
dc.identifier.citation Marinescu, D. C., Paya, A. and Morrison, J. P. (2017) 'A Cloud Reservation System for Big Data Applications', IEEE Transactions on Parallel and Distributed Systems, 28(3), pp. 606-618. DOI: 10.1109/TPDS.2016.2594783 en
dc.identifier.volume 28 en
dc.identifier.issued 3 en
dc.identifier.startpage 606 en
dc.identifier.endpage 618 en
dc.identifier.issn 1045-9219
dc.identifier.uri http://hdl.handle.net/10468/8414
dc.identifier.doi 10.1109/TPDS.2016.2594783 en
dc.description.abstract Emerging Big Data applications increasingly require resources beyond those available from a single server and may be expressed as a complex workflow of many components and dependency relationships-each component potentially requiring its own specific, and perhaps specialized, resources for its execution. Efficiently supporting this type of Big Data application is a challenging resource management problem for existing cloud environments. In response, we propose a two-stage protocol for solving this resource management problem. We exploit spatial locality in the first stage by dynamically forming rack-level coalitions of servers to execute a workflow component. These coalitions only exist for the duration of the execution of their assigned component and are subsequently disbanded, allowing their resources to take part in future coalitions. The second stage creates a package of these coalitions, designed to support all the components in the complete workflow. To minimize the communication and housekeeping overhead needed to form this package of coalitions, the technique of combinatorial auctions is adapted from market-based resource allocation. This technique has a considerably lower overhead for resource aggregation than the traditional hierarchically organized models. We analyze two strategies for coalition formation: the first, history-based uses information from past auctions to pre-form coalitions in anticipation of predicted demand; the second one is a just-in-time-that builds coalitions only when support for specific workflow components is requested. en
dc.description.sponsorship Enterprise Ireland (Irish Centre for Cloud Computing and Commerce); IDA Ireland (Irish Centre for Cloud Computing and Commerce) en
dc.format.mimetype application/pdf en
dc.language.iso en en
dc.publisher IEEE en
dc.relation.uri https://ieeexplore.ieee.org/document/7523396
dc.rights © 2017, IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. en
dc.subject Big Data en
dc.subject Cloud computing en
dc.subject Combinatorial mathematics en
dc.subject Just-in-time en
dc.subject Network servers en
dc.subject Protocols en
dc.subject Resource allocation en
dc.subject Cloud reservation system en
dc.subject Big Data applications en
dc.subject Component-dependency relationships en
dc.subject Resource management en
dc.subject Cloud environments en
dc.subject Two-stage protocol en
dc.subject Dynamically forming rack-level server coalitions en
dc.subject Workflow component execution en
dc.subject Combinatorial auctions en
dc.subject Market-based resource allocation en
dc.subject Servers en
dc.subject Organizations en
dc.subject Computers en
dc.subject Google en
dc.subject Cloud resource management en
dc.subject Hierarchical organization en
dc.title A cloud reservation system for big data applications en
dc.type Article (peer-reviewed) en
dc.internal.authorcontactother John Morrison, Computer Science, University College Cork, Cork, Ireland. +353-21-490-3000 Email: j.morrison@cs.ucc.ie en
dc.internal.availability Full text available en
dc.date.updated 2019-08-29T14:08:37Z
dc.description.version Accepted Version en
dc.internal.rssid 428006706
dc.contributor.funder National Science Foundation en
dc.contributor.funder European Commission en
dc.contributor.funder IDA Ireland en
dc.contributor.funder Enterprise Ireland en
dc.description.status Peer reviewed en
dc.identifier.journaltitle IEEE Transactions On Parallel and Distributed Systems en
dc.internal.copyrightchecked Yes
dc.internal.licenseacceptance Yes en
dc.internal.IRISemailaddress j.morrison@cs.ucc.ie en
dc.relation.project info:eu-repo/grantAgreement/NSF//1525943/US/AF: Small: Is the Simulation of Quantum Many-Body Systems Feasible on the Cloud?/ en
dc.relation.project info:eu-repo/grantAgreement/EC/H2020::RIA/643946/EU/Self-Organising, Self-Managing Heterogeneous Cloud/CloudLightning en

Files in this item

This item appears in the following Collection(s)

Show simple item record

This website uses cookies. By using this website, you consent to the use of cookies in accordance with the UCC Privacy and Cookies Statement. For more information about cookies and how you can disable them, visit our Privacy and Cookies statement