TY - GEN
T1 - Toward Scalable Analysis of Multidimensional Scientific Data
T2 - 2018 IEEE International Conference on Big Data, Big Data 2018
AU - Niu, Ye
AU - Al-Mamun, Abdullah
AU - Lin, Hui
AU - Li, Tonglin
AU - Zhao, Yi
AU - Zhao, Dongfang
N1 - Publisher Copyright:
© 2018 IEEE.
PY - 2018/7/2
Y1 - 2018/7/2
N2 - Many modern scientific applications involve large volumes of multidimensional data and extensive computation. Although distributed systems and tools are becoming increasingly scalable, they are still far away to catch up the exponential growth rate exhibited by many of those scientific big-data applications. This paper presents our early effort on overcoming the exponential complexity of one widely deployed workload over multidimensional scientific data - the n×n numerical analysis on two-dimensional arrays. More specifically, we propose a new approach to reduce the exponentially-grown data into a semantically-equivalent polynomial form in the context of two-dimensional electrode arrays, which are widely used in biomedical engineering, electrical engineering, and mechanical engineering. We have implemented a system prototype in Python, preliminary results show that the proposed approach outperforms the state-of-the-practice in various metrics: (i) the consumed space is six orders of magnitude smaller; (ii) the execution time is three orders of magnitude faster; and (iii) the scalability is improved by two orders of magnitude - from 6×6 to 100 × 100 - on mainstream servers in reasonable time.
AB - Many modern scientific applications involve large volumes of multidimensional data and extensive computation. Although distributed systems and tools are becoming increasingly scalable, they are still far away to catch up the exponential growth rate exhibited by many of those scientific big-data applications. This paper presents our early effort on overcoming the exponential complexity of one widely deployed workload over multidimensional scientific data - the n×n numerical analysis on two-dimensional arrays. More specifically, we propose a new approach to reduce the exponentially-grown data into a semantically-equivalent polynomial form in the context of two-dimensional electrode arrays, which are widely used in biomedical engineering, electrical engineering, and mechanical engineering. We have implemented a system prototype in Python, preliminary results show that the proposed approach outperforms the state-of-the-practice in various metrics: (i) the consumed space is six orders of magnitude smaller; (ii) the execution time is three orders of magnitude faster; and (iii) the scalability is improved by two orders of magnitude - from 6×6 to 100 × 100 - on mainstream servers in reasonable time.
KW - Multidimensional Data
KW - Parallel Computing
KW - Scientific Computing
UR - http://www.scopus.com/inward/record.url?scp=85062590256&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85062590256&partnerID=8YFLogxK
U2 - 10.1109/BigData.2018.8622423
DO - 10.1109/BigData.2018.8622423
M3 - Conference contribution
AN - SCOPUS:85062590256
T3 - Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018
SP - 3899
EP - 3904
BT - Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018
A2 - Abe, Naoki
A2 - Liu, Huan
A2 - Pu, Calton
A2 - Hu, Xiaohua
A2 - Ahmed, Nesreen
A2 - Qiao, Mu
A2 - Song, Yang
A2 - Kossmann, Donald
A2 - Liu, Bing
A2 - Lee, Kisung
A2 - Tang, Jiliang
A2 - He, Jingrui
A2 - Saltz, Jeffrey
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 10 December 2018 through 13 December 2018
ER -