TY - GEN
T1 - DVM
T2 - 47th International Conference on Parallel Processing, ICPP 2018
AU - Al-Mamun, Abdullah
AU - Wang, Ke
AU - Liu, Jialin
AU - Zhao, Dongfang
N1 - Publisher Copyright:
© 2018 Association for Computing Machinery.
PY - 2018/8/13
Y1 - 2018/8/13
N2 - One of the most challenging problems in modern distributed big data systems lies in their memory management: these systems pre-allocate a fixed amount of memory before applications start. In the best case where more memory can be acquired, users have to reconfigure the deployment and re-compute many intermediate results. If no more memory is available, users are then forced to manually partition the job into smaller tasks, incurring both development and performance overhead. This paper presents a user-level utility for scaling the memory in a distributed setup - the Distributed Virtual Memory (DVM). DVM enables to efficiently swap data between memory and disk between arbitrary nodes without users' intervention or applications' awareness.
AB - One of the most challenging problems in modern distributed big data systems lies in their memory management: these systems pre-allocate a fixed amount of memory before applications start. In the best case where more memory can be acquired, users have to reconfigure the deployment and re-compute many intermediate results. If no more memory is available, users are then forced to manually partition the job into smaller tasks, incurring both development and performance overhead. This paper presents a user-level utility for scaling the memory in a distributed setup - the Distributed Virtual Memory (DVM). DVM enables to efficiently swap data between memory and disk between arbitrary nodes without users' intervention or applications' awareness.
UR - http://www.scopus.com/inward/record.url?scp=85054886071&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85054886071&partnerID=8YFLogxK
U2 - 10.1145/3229710.3229737
DO - 10.1145/3229710.3229737
M3 - Conference contribution
AN - SCOPUS:85054886071
SN - 9781450365239
T3 - ACM International Conference Proceeding Series
BT - 47th International Conference on Parallel Processing, ICPP 2018
PB - Association for Computing Machinery
Y2 - 13 August 2018 through 16 August 2018
ER -