Zicong Hong
Albert Zomaya
Skip Abstract Section
Abstract
Blockchain databases have attracted widespread attention but suffer from poor scalability due to underlying non-scalable blockchains. While blockchain sharding is necessary for a scalable blockchain database, it poses a new challenge named on-chain cross-shard database services. Each cross-shard database service (e.g., cross-shard queries or inter-shard load balancing) involves massive cross-shard data exchanges, while the existing cross-shard mechanisms need to process each cross-shard data exchange via the consensus of all nodes in the related shards (i.e., on-chain) to resist a Byzantine environment of blockchain, which eliminates sharding benefits.
To tackle the challenge, this paper presents GriDB, the first scalable blockchain database, by designing a novel off-chain cross-shard mechanism for efficient cross-shard database services. Borrowing the idea of off-chain payments, GriDB delegates massive cross-shard data exchange to a few nodes, each of which is randomly picked from a different shard. Considering the Byzantine environment, the untrusted delegates cooperate to generate succinct proof for cross-shard data exchanges, while the consensus is only responsible for the low-cost proof verification. However, different from payments, the database services' verification has more requirements (e.g., completeness, correctness, freshness, and availability); thus, we introduce several new authenticated data structures (ADS). Particularly, we utilize consensus to extend the threat model and reduce the complexity of traditional accumulator-based ADS for verifiable cross-shard queries with a rich set of relational operators. Moreover, we study the necessity of inter-shard load balancing for a scalable blockchain database and design an off-chain and live approach for both efficiency and availability during balancing. An evaluation of our prototype shows the performance of GriDB in terms of scalability in workloads with queries and updates.
- Mustafa Al-Bassam, Alberto Sonnino, Shehar Bano, Dave Hrycyszyn, and George Danezis. 2017. Chainspace: A Sharded Smart Contracts Platform. CoRR abs/1708.03778 (2017). arXiv:1708.03778 http://arxiv.org/abs/1708.03778Google Scholar
- Dan Boneh and Xavier Boyen. 2008. Short Signatures Without Random Oracles and the SDH Assumption in Bilinear Groups. J. Cryptol. 21, 2 (2008), 149--177.Google ScholarCross Ref
- Ran Canetti, Omer Paneth, Dimitrios Papadopoulos, and Nikos Triandopoulos. 2014. Verifiable Set Operations over Outsourced Databases. In Public-Key Cryptography - PKC 2014, Hugo Krawczyk (Ed.). Springer Berlin Heidelberg, 113--130.Google ScholarDigital Library
- Miguel Castro and Barbara Liskov. 1999. Practical Byzantine Fault Tolerance. In Proceedings of the Third Symposium on Operating Systems Design and Implementation (OSDI 99). USENIX Association.Google ScholarDigital Library
- Hung Dang, Tien Tuan Anh Dinh, Dumitrel Loghin, Ee-Chien Chang, Qian Lin, and Beng Chin Ooi. 2019. Towards Scaling Blockchain Systems via Sharding. In Proceedings of the 2019 International Conference on Management of Data (Amsterdam, Netherlands) (SIGMOD '19). Association for Computing Machinery, New York, NY, USA, 123--140. Google ScholarDigital Library
- Azure SQL Database. 2022. Scaling out with Azure SQL Database. Retrieved March 20, 2023 from https://docs.microsoft.com/en-us/azure/azure-sql/database/elastic-scale-introductionGoogle Scholar
- DoltHub. 2023. go-mysql-server. Retrieved March 20, 2023 from https://github.com/dolthub/go-mysql-serverGoogle Scholar
- Muhammad El-Hindi, Carsten Binnig, Arvind Arasu, Donald Kossmann, and Ravi Ramamurthy. 2019. BlockchainDB: A Shared Database on Blockchains. Proc. VLDB Endow. 12, 11 (jul 2019), 1597--1609. Google ScholarDigital Library
- Aaron J. Elmore, Vaibhav Arora, Rebecca Taft, Andrew Pavlo, Divyakant Agrawal, and Amr El Abbadi. 2015. Squall: Fine-Grained Live Reconfiguration for Partitioned Main Memory Databases. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data (Melbourne, Victoria, Australia) (SIGMOD '15). 299--313. Google ScholarDigital Library
- Aaron J. Elmore, Sudipto Das, Divyakant Agrawal, and Amr El Abbadi. 2011. Zephyr: Live Migration in Shared Nothing Databases for Elastic Cloud Platforms. In Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data (Athens, Greece) (SIGMOD '11). Association for Computing Machinery, New York, NY, USA, 301--312. Google ScholarDigital Library
- Ethereum. 2022. Hardware requirements for Go-Ethereum. Retrieved March 20, 2023 from https://geth.ethereum.org/docs/getting-started/hardware-requirementsGoogle Scholar
- Ethereum. 2023. Go Ethereum. Retrieved March 20, 2023 from https://github.com/ethereum/go-ethereumGoogle Scholar
- Emmanuelle Ganne. 2018. Can Blockchain revolutionize international trade? World Trade Organization Geneva.Google Scholar
- Zerui Ge, Dumitrel Loghin, Beng Chin Ooi, Pingcheng Ruan, and Tianwen Wang. 2022. Hybrid Blockchain Database Systems: Design and Performance. Proc. VLDB Endow. 15, 5 (2022), 1092--1104. Google ScholarDigital Library
- Yossi Gilad, Rotem Hemo, Silvio Micali, Georgios Vlachos, and Nickolai Zeldovich. 2017. Algorand: Scaling Byzantine Agreements for Cryptocurrencies. In Proceedings of the 26th Symposium on Operating Systems Principles (Shanghai, China) (SOSP '17). Association for Computing Machinery, New York, NY, USA, 51--68. Google ScholarDigital Library
- Google. 2023. The Go Programming Language. Retrieved March 20, 2023 from https://golang.org/Google Scholar
- Lewis Gudgeon, Pedro Moreno-Sanchez, Stefanie Roos, Patrick McCorry, and Arthur Gervais. 2019. SoK: Layer-Two Blockchain Protocols. Cryptology ePrint Archive, Paper 2019/360. https://eprint.iacr.org/2019/360 https://eprint.iacr.org/2019/360.Google Scholar
- Harmony. 2023. Harmony. Retrieved March 20, 2023 from https://github.com/harmony-one/harmonyGoogle Scholar
- Harmony. 2023. Harmony consensus protocol design. Retrieved March 20, 2023 from https://github.com/harmony-one/harmony/tree/main/consensusGoogle Scholar
- Jelle Hellings and Mohammad Sadoghi. 2021. ByShard: Sharding in a Byzantine Environment. Proc. VLDB Endow. 14, 11 (2021), 2230--2243. Google ScholarDigital Library
- Herumi. 2020. High-Speed Software Implementation of the Optimal Ate Pairing over Barreto-Naehrig Curves. Retrieved March 20, 2023 from https://github.com/herumi/ate-pairingGoogle Scholar
- Zicong Hong, Song Guo, Peng Li, and Wuhui Chen. 2021. Pyramid: A Layered Sharding Blockchain System. In IEEE INFOCOM 2021 - IEEE Conference on Computer Communications. 1--10. Google ScholarDigital Library
- Zicong Hong, Song Guo, Rui Zhang, Peng Li, Yufen Zhan, and Wuhui Chen. 2022. Cycle: Sustainable Off-Chain Payment Channel Network with Asynchronous Rebalancing. In 2022 52nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). 41--53. Google ScholarCross Ref
- IBM. 2020. Blockchain for supply chain solutions. Retrieved March 20, 2023 from https://www.ibm.com/blockchain/industries/supply-chainGoogle Scholar
- R. Karp, C. Schindelhauer, S. Shenker, and B. Vocking. 2000. Randomized rumor spreading. In Proceedings 41st Annual Symposium on Foundations of Computer Science. 565--574.Google ScholarCross Ref
- Rami Khalil, Alexei Zamyatin, Guillaume Felley, Pedro Moreno-Sanchez, and Arthur Gervais. 2018. Commit-chains: Secure, scalable off-chain payments. Retrieved March 20, 2023 from https://eprint.iacr.org/2018/642.pdfGoogle Scholar
- Eleftherios Kokoris-Kogias, Philipp Jovanovic, Nicolas Gailly, Ismail Khoffi, Linus Gasser, and Bryan Ford. 2016. Enhancing Bitcoin Security and Performance with Strong Consistency via Collective Signing. In Proceedings of the 25th USENIX Conference on Security Symposium (Austin, TX, USA) (SEC'16). USENIX Association, USA, 279--296.Google ScholarDigital Library
- Eleftherios Kokoris-Kogias, Philipp Jovanovic, Linus Gasser, Nicolas Gailly, Ewa Syta, and Bryan Ford. 2018. OmniLedger: A Secure, Scale-Out, Decentralized Ledger via Sharding. In 2018 IEEE Symposium on Security and Privacy (SP). 583--598. Google ScholarCross Ref
- Lightning. 2021. The Lightning Network. Retrieved March 20, 2023 from https://lightning.network/Google Scholar
- Loi Luu, Viswesh Narayanan, Chaodong Zheng, Kunal Baweja, Seth Gilbert, and Prateek Saxena. 2016. A Secure Sharding Protocol For Open Blockchains. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (Vienna, Austria) (CCS '16). Association for Computing Machinery, New York, NY, USA, 17--30. Google ScholarDigital Library
- Loi Luu, Jason Teutsch, Raghav Kulkarni, and Prateek Saxena. 2015. Demystifying Incentives in the Consensus Computer. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (Denver, Colorado, USA) (CCS '15). Association for Computing Machinery, New York, NY, USA, 706--719. Google ScholarDigital Library
- Ralph C Merkle. 1987. A digital signature based on a conventional encryption function. In Conference on the theory and application of cryptographic techniques. Springer, 369--378.Google ScholarDigital Library
- Andrew Miller, Iddo Bentov, Surya Bakshi, Ranjit Kumaresan, and Patrick McCorry. 2019. Sprites and State Channels: Payment Networks that Go Faster Than Lightning. In Financial Cryptography and Data Security. Springer International Publishing, 508--526.Google Scholar
- Atsuki Momose and Ling Ren. 2021. Multi-Threshold Byzantine Fault Tolerance. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (Virtual Event, Republic of Korea) (CCS '21). Association for Computing Machinery, New York, NY, USA, 1686--1699. Google ScholarDigital Library
- Einar Mykletun, Maithili Narasimha, and Gene Tsudik. 2006. Authentication and Integrity in Outsourced Databases. ACM Trans. Storage 2, 2 (2006), 107--138. Google ScholarDigital Library
- MySQL. 2023. MySQL 8.0 Reference. Retrieved March 20, 2023 from https://dev.mysql.com/doc/refman/8.0/en/sql-data-manipulation-statements.htmlGoogle Scholar
- Satoshi Nakamoto. 2008. Bitcoin: A peer-to-peer electronic cash system. Retrieved March 20, 2023 from https://bitcoin.org/bitcoin.pdfGoogle Scholar
- Lan Nguyen. 2005. Accumulators from Bilinear Pairings and Applications. In Proceedings of the 2005 International Conference on Topics in Cryptology (San Francisco, CA) (CT-RSA'05). Springer-Verlag, Berlin, Heidelberg, 275--292. Google ScholarDigital Library
- Charalampos Papamanthou, Roberto Tamassia, and Nikos Triandopoulos. 2011. Optimal Verification of Operations on Dynamic Sets. In Advances in Cryptology - CRYPTO 2011, Phillip Rogaway (Ed.). Springer Berlin Heidelberg, 91--110.Google ScholarCross Ref
- Q. Pei, E. Zhou, Y. Xiao, D. Zhang, and D. Zhao. 2020. An Efficient Query Scheme for Hybrid Storage Blockchains Based on Merkle Semantic Trie. In 2020 International Symposium on Reliable Distributed Systems (SRDS). 51--60. Google ScholarCross Ref
- Yanqing Peng, Min Du, Feifei Li, Raymond Cheng, and Dawn Song. 2020. FalconDB: Blockchain-Based Collaborative Database. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (Portland, OR, USA) (SIGMOD '20). Association for Computing Machinery, New York, NY, USA, 637--652. Google ScholarDigital Library
- George Pîrlea, Amrit Kumar, and Ilya Sergey. 2021. Practical Smart Contract Sharding with Ownership and Commutativity Analysis. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation (Virtual, Canada) (PLDI 2021). Association for Computing Machinery, New York, NY, USA, 1327--1341. Google ScholarDigital Library
- Joseph Poon and Vitalik Buterin. 2017. Plasma: Scalable autonomous smart contracts. Retrieved March 20, 2023 from https://www.plasma.io/plasma.pdfGoogle Scholar
- Raiden. 2021. The Raiden Network. Retrieved March 20, 2023 from https://raiden.network/Google Scholar
- Raghu Ramakrishnan and Johannes Gehrke. 2000. Database Management Systems (2nd ed.). McGraw-Hill, Inc.Google ScholarDigital Library
- Sukriti Ramesh, Odysseas Papapetrou, and Wolf Siberski. 2009. Optimizing Distributed Joins with Bloom Filters. In Distributed Computing and Internet Technology. Springer Berlin Heidelberg, 145--156.Google Scholar
- Pingcheng Ruan, Gang Chen, Tien Tuan Anh Dinh, Qian Lin, Beng Chin Ooi, and Meihui Zhang. 2019. Fine-Grained, Secure and Efficient Data Provenance on Blockchain Systems. Proc. VLDB Endow. 12, 9 (may 2019), 975--988. Google ScholarDigital Library
- Pingcheng Ruan, Tien Tuan Anh Dinh, Dumitrel Loghin, Meihui Zhang, Gang Chen, Qian Lin, and Beng Chin Ooi. 2021. Blockchains vs. Distributed Databases: Dichotomy and Fusion. In Proceedings of the 2021 International Conference on Management of Data (Virtual Event, China) (SIGMOD '21). Association for Computing Machinery, New York, NY, USA, 1504--1517. Google ScholarDigital Library
- SCIPRLab. 2020. libsnark: a C++ library for zkSNARK proofs. Retrieved March 20, 2023 from https://github.com/scipr-lab/libsnarkGoogle Scholar
- Rebecca Taft, Essam Mansour, Marco Serafini, Jennie Duggan, Aaron J. Elmore, Ashraf Aboulnaga, Andrew Pavlo, and Michael Stonebraker. 2014. E-Store: FineGrained Elastic Partitioning for Distributed Transaction Processing Systems. Proc. VLDB Endow. 8, 3 (2014), 245--256. Google ScholarDigital Library
- Yuechen Tao, Bo Li, Jingjie Jiang, Hok Chu Ng, Cong Wang, and Baochun Li. 2020. On Sharding Open Blockchains with Smart Contracts. In 2020 IEEE 36th International Conference on Data Engineering (ICDE). 1357--1368. Google ScholarCross Ref
- TPC. 2023. TPC-H Benchmark. Retrieved March 20, 2023 from http://www.tpc.org/tpch/Google Scholar
- Gang Wang, Zhijie Jerry Shi, Mark Nixon, and Song Han. 2019. SoK: Sharding on Blockchain. In Proceedings of the 1st ACM Conference on Advances in Financial Technologies (Zurich, Switzerland) (AFT '19). Association for Computing Machinery, New York, NY, USA, 41--61. Google ScholarDigital Library
- Jiaping Wang and Hao Wang. 2019. Monoxide: Scale out Blockchains with Asynchronous Consensus Zones. In 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19). USENIX Association, Boston, MA, 95--112. https://www.usenix.org/conference/nsdi19/presentation/wang-jiapingGoogle Scholar
- Xingda Wei, Sijie Shen, Rong Chen, and Haibo Chen. 2017. Replication-Driven Live Reconfiguration for Fast Distributed Transaction Processing. In Proceedings of the 2017 USENIX Conference on Usenix Annual Technical Conference (Santa Clara, CA, USA) (USENIX ATC '17). USENIX Association, USA, 335--347.Google Scholar
- Cheng Xu,Ce Zhang, and Jianliang Xu. 2019. VChain: Enabling Verifiable Boolean Range Queries over Blockchain Databases. In Proceedings of the 2019 International Conference on Management of Data (Amsterdam, Netherlands) (SIGMOD '19). Association for Computing Machinery, New York, NY, USA, 141--158. Google ScholarDigital Library
- Mahdi Zamani, Mahnush Movahedi, and Mariana Raykova. 2018. RapidChain: Scaling Blockchain via Full Sharding. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (Toronto, Canada) (CCS '18). Association for Computing Machinery, New York, NY, USA, 931--948. Google ScholarDigital Library
- Bo Zhang, Boxiang Dong, and Wendy Hui Wang. 2021. Integrity Authentication for SQL Query Evaluation on Outsourced Databases: A Survey. IEEE Transactions on Knowledge and Data Engineering 33, 4 (2021), 1601--1618. Google ScholarCross Ref
- Ce Zhang, Cheng Xu, Jianliang Xu, Yuzhe Tang, and Byron Choi. 2019. GEM2-Tree: A Gas-Efficient Structure for Authenticated Range Queries in Blockchain. In 2019 IEEE 35th International Conference on Data Engineering (ICDE). 842--853. Google ScholarCross Ref
- Yupeng Zhang, Daniel Genkin, Jonathan Katz, Dimitrios Papadopoulos, and Charalampos Papamanthou. 2017. vSQL: Verifying Arbitrary SQL Queries over Dynamic Outsourced Databases. In 2017 IEEE Symposium on Security and Privacy (SP). 863--880. Google ScholarCross Ref
- Yupeng Zhang, Jonathan Katz, and Charalampos Papamanthou. 2015. IntegriDB: Verifiable SQL for Outsourced Databases. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (Denver, Colorado, USA) (CCS '15). Association for Computing Machinery, New York, NY, USA, 1480--1491. Google ScholarDigital Library
- Yanchao Zhu, Zhao Zhang, Cheqing Jin, Aoying Zhou, and Ying Yan. 2019. SEBDB: Semantics Empowered BlockChain DataBase. In 2019 IEEE 35th International Conference on Data Engineering (ICDE). 1820--1831. Google ScholarCross Ref
Recommendations
Towards Scaling Blockchain Systems via Sharding
SIGMOD '19: Proceedings of the 2019 International Conference on Management of DataExisting blockchain systems scale poorly because of their distributed consensus protocols. Current attempts at improving blockchain scalability are limited to cryptocurrency. Scaling blockchain systems under general workloads (i.e., non-cryptocurrency ...
Sharding for Scalable Blockchain Networks
AbstractBlockchain technology has been extended from bitcoin transactions to applications in multiple domains. Scalability has been identified as a major challenge in large-scale blockchain networks because of the number of possible participants. ...
BSS-ITS: Blockchain Scaling Scheme with Sharding for Intelligent Transportation System: Scale Blockchain for Better Data Exchange and Storage with Full Sharding for Intelligent Transportation System
ICBTA '21: Proceedings of the 2021 4th International Conference on Blockchain Technology and ApplicationsWith the development of modern technologies, Intelligent Transportation System (ITS) has made great progress and brought convenience to every aspect of people's lives. ITS has become more complicated and uncertain due to the existing security and ...
Comments