TY - GEN
T1 - A middleware for developing and deploying scalable remote mining services
AU - Glimcher, Leonid
AU - Agrawal, Gagan
PY - 2008
Y1 - 2008
N2 - In this paper, we consider the problem of developing service-oriented implementations of data-intensive applications that process data on remote servers. While the existing grid and web-service frameworks allow interoperability and flexible resource utilization, achieving efficiency and scalability remains a critical challenge. Similarly, the existing grid and web-service frameworks do not provide transparency in accessing and processing data from grid-based data servers. We present design and evaluation of a system that supports a high-level interface for developing data mining and scientific data processing grid-services and targets data residing on SRB servers. Results of our evaluation using two data mining and one scientific data processing applications show two important observations. First, each of applications we evaluated demonstrated good scalability with respect to dataset size, as well as changing numbers of both data host and compute nodes. Second, there is only a small overhead associated with deploying our middleware-based applications using MPICH-G2 and Globus. This overhead varied between 14% and 22% and is primarily because of a larger memory footprint. Thus, overall, our work shows that it is feasible to develop and deploy scalable and efficient grid-services that process data from remote servers.
AB - In this paper, we consider the problem of developing service-oriented implementations of data-intensive applications that process data on remote servers. While the existing grid and web-service frameworks allow interoperability and flexible resource utilization, achieving efficiency and scalability remains a critical challenge. Similarly, the existing grid and web-service frameworks do not provide transparency in accessing and processing data from grid-based data servers. We present design and evaluation of a system that supports a high-level interface for developing data mining and scientific data processing grid-services and targets data residing on SRB servers. Results of our evaluation using two data mining and one scientific data processing applications show two important observations. First, each of applications we evaluated demonstrated good scalability with respect to dataset size, as well as changing numbers of both data host and compute nodes. Second, there is only a small overhead associated with deploying our middleware-based applications using MPICH-G2 and Globus. This overhead varied between 14% and 22% and is primarily because of a larger memory footprint. Thus, overall, our work shows that it is feasible to develop and deploy scalable and efficient grid-services that process data from remote servers.
UR - http://www.scopus.com/inward/record.url?scp=50649118393&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=50649118393&partnerID=8YFLogxK
U2 - 10.1109/CCGRID.2008.36
DO - 10.1109/CCGRID.2008.36
M3 - Conference contribution
AN - SCOPUS:50649118393
SN - 9780769531564
T3 - Proceedings CCGRID 2008 - 8th IEEE International Symposium on Cluster Computing and the Grid
SP - 242
EP - 249
BT - Proceedings CCGRID 2008 - 8th IEEE International Symposium on Cluster Computing and the Grid
T2 - CCGRID 2008 - 8th IEEE International Symposium on Cluster Computing and the Grid
Y2 - 19 May 2008 through 22 May 2008
ER -