TY - GEN
T1 - In-memory Blockchain
T2 - 2018 IEEE International Conference on Big Data, Big Data 2018
AU - Al-Mamun, Abdullah
AU - Li, Tonglin
AU - Sadoghi, Mohammad
AU - Zhao, Dongfang
N1 - Funding Information:
This paper presents a new in-memory blockchain system architecture where ledgers are maintained mostly in memory and a new consensus protocol crafted for the new architecture. The new architecture and new consensus collectively enable an efficient blockchain-like ledger service for trustworthy data provenance on HPC systems. The correctness of the proposed consensus is both theoretically proven and experimentally verified. A lightweight system prototype is implemented and evaluated with more than one million transactions, showing 32× speedup compared to the filesystem-based provenance service and four orders of magnitude speedup compared to the database-based provenance service. Acknowledgement. This work is in part supported by a Microsoft Azure Research Award.
Publisher Copyright:
© 2018 IEEE.
PY - 2019/1/22
Y1 - 2019/1/22
N2 - The state-of-the-art approaches for tracking data provenance on high-performance computing (HPC) systems are either supported by file systems or relational databases. These techniques shared the same critique on the provenance data's fidelity and the associated I/O overhead. This paper envisions to track the HPC data provenance using a distributed in-memory ledger - the core technique leveraged by blockchains and proven to be highly trustworthy by many large-scale applications. We pinpoint two system challenges - storage architecture and consensus protocol - for adopting blockchains to HPC and make the following contributions: (i) We design a new in-memory blockchain architecture for HPC systems, exploiting the high-performance network infrastructure InfiniBand and greatly reducing the I/O overhead; and (ii) We develop a new consensus protocol, namely proof-of-reproducibility (PoR), crafted for the new architecture, which takes into account both proof-of-work (PoW) and proof-of-stake (PoS) mechanisms. The correctness of PoR is both theoretically proven and experimentally verified. A prototype system is implemented and evaluated with more than one million transactions, showing 32× speedup compared to the filesystem-based provenance service and four orders of magnitude speedup compared to the database-based provenance service.
AB - The state-of-the-art approaches for tracking data provenance on high-performance computing (HPC) systems are either supported by file systems or relational databases. These techniques shared the same critique on the provenance data's fidelity and the associated I/O overhead. This paper envisions to track the HPC data provenance using a distributed in-memory ledger - the core technique leveraged by blockchains and proven to be highly trustworthy by many large-scale applications. We pinpoint two system challenges - storage architecture and consensus protocol - for adopting blockchains to HPC and make the following contributions: (i) We design a new in-memory blockchain architecture for HPC systems, exploiting the high-performance network infrastructure InfiniBand and greatly reducing the I/O overhead; and (ii) We develop a new consensus protocol, namely proof-of-reproducibility (PoR), crafted for the new architecture, which takes into account both proof-of-work (PoW) and proof-of-stake (PoS) mechanisms. The correctness of PoR is both theoretically proven and experimentally verified. A prototype system is implemented and evaluated with more than one million transactions, showing 32× speedup compared to the filesystem-based provenance service and four orders of magnitude speedup compared to the database-based provenance service.
UR - http://www.scopus.com/inward/record.url?scp=85062553669&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85062553669&partnerID=8YFLogxK
U2 - 10.1109/BigData.2018.8621897
DO - 10.1109/BigData.2018.8621897
M3 - Conference contribution
AN - SCOPUS:85062553669
T3 - Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018
SP - 3808
EP - 3813
BT - Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018
A2 - Song, Yang
A2 - Liu, Bing
A2 - Lee, Kisung
A2 - Abe, Naoki
A2 - Pu, Calton
A2 - Qiao, Mu
A2 - Ahmed, Nesreen
A2 - Kossmann, Donald
A2 - Saltz, Jeffrey
A2 - Tang, Jiliang
A2 - He, Jingrui
A2 - Liu, Huan
A2 - Hu, Xiaohua
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 10 December 2018 through 13 December 2018
ER -