TY - GEN
T1 - Resource allocation for distributed streaming applications
AU - Qian, Zhu
AU - Agrawal, Gagan
PY - 2008
Y1 - 2008
N2 - We consider resource allocation for distributed streaming applications running In a grid environment, where continuously streaming data needs to be aggregated and processed to produce output streams. Because such an application comprises a pipeline of processing stages, both communication and computational requirements need to be taken Into account while performing resource allocation. In this paper, we give a rigorous formulation of this resource allocation problem, based on the DAG representation of the application as well as the environment. We have shown how we can use the notion of subgraph Isomorphism and developed an effective resource allocation algorithm. The main observations from the experiments we conducted to evaluate our algorithms were as follows: the overhead caused by our algorithm Is comparable to an existing algorithm, Streamline, which Is based on heuristics. At the same time, the application performance was Improved by 30% on average. When compared to the allocation performed by the Optimal algorithm, which enumerates all mappings, the application performance with our algorithm was within 4%. At the same time, unlike the Optimal algorithm, our algorithm scaled well to large graphs.
AB - We consider resource allocation for distributed streaming applications running In a grid environment, where continuously streaming data needs to be aggregated and processed to produce output streams. Because such an application comprises a pipeline of processing stages, both communication and computational requirements need to be taken Into account while performing resource allocation. In this paper, we give a rigorous formulation of this resource allocation problem, based on the DAG representation of the application as well as the environment. We have shown how we can use the notion of subgraph Isomorphism and developed an effective resource allocation algorithm. The main observations from the experiments we conducted to evaluate our algorithms were as follows: the overhead caused by our algorithm Is comparable to an existing algorithm, Streamline, which Is based on heuristics. At the same time, the application performance was Improved by 30% on average. When compared to the allocation performed by the Optimal algorithm, which enumerates all mappings, the application performance with our algorithm was within 4%. At the same time, unlike the Optimal algorithm, our algorithm scaled well to large graphs.
UR - http://www.scopus.com/inward/record.url?scp=55849113042&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=55849113042&partnerID=8YFLogxK
U2 - 10.1109/ICPP.2008.49
DO - 10.1109/ICPP.2008.49
M3 - Conference contribution
AN - SCOPUS:55849113042
SN - 9780769533742
T3 - Proceedings of the International Conference on Parallel Processing
SP - 414
EP - 421
BT - Proceedings - 37th International Conference on Parallel Processing, ICPP 2008
T2 - 37th International Conference on Parallel Processing, ICPP 2008
Y2 - 9 September 2008 through 12 September 2008
ER -