Mark Raasveldt
ABSTRACT
The immense popularity of SQLite shows that there is a need for unobtrusive in-process data management solutions. However, there is no such system yet geared towards analytical workloads. We demonstrate DuckDB, a novel data management system designed to execute analytical SQL queries while embedded in another process. In our demonstration, we pit DuckDB against other data management solutions to showcase its performance in the embedded analytics scenario. DuckDB is available as Open Source software under a permissive license.
- Peter A. Boncz, Marcin Zukowski, and Niels Nes. 2005. MonetDB/X100: Hyper-Pipelining Query Execution. In CIDR 2005, Second Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 4--7, 2005 . 225--237. http://cidrdb.org/cidr2005/papers/P19.pdfGoogle Scholar
- Lukas Fittl. 2019. C library for accessing the PostgreSQL parser outside of the server environment. https://github.com//fittl/libpg_query .Google Scholar
- Richard Hipp. 2019 a. Database File Format. https://www.sqlite.org/fileformat.html .Google Scholar
- Richard Hipp. 2019 b. Most Widely Deployed and Used Database Engine. https://www.sqlite.org/mostdeployed.html .Google Scholar
- Harald Lang, Tobias Mü hlbauer, Florian Funke, et almbox. 2016. Data Blocks: Hybrid OLTP and OLAP on Compressed Storage using both Vectorization and Compilation. In Proceedings of the 2016 International Conference on Management of Data, SIGMOD Conference 2016, San Francisco, CA, USA, June 26 - July 01, 2016. 311--326. Google ScholarDigital Library
- Wes McKinney. 2010. Data Structures for Statistical Computing in Python. In Proceedings of the 9th Python in Science Conference, Stéfan van der Walt and Jarrod Millman (Eds.). 51 -- 56.Google ScholarCross Ref
- Guido Moerkotte and Thomas Neumann. 2008. Dynamic programming strikes back. In Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2008, Vancouver, BC, Canada, June 10--12, 2008. 539--552. Google ScholarDigital Library
- Thomas Neumann. 2011. Efficiently Compiling Efficient Query Plans for Modern Hardware. PVLDB, Vol. 4, 9 (2011), 539--550. Google ScholarDigital Library
- Thomas Neumann and Alfons Kemper. 2015. Unnesting Arbitrary Queries. In Datenbanksysteme fü r Business, Technologie und Web (BTW), 16. Fachtagung des GI-Fachbereichs "Datenbanken und Informationssysteme" (DBIS), 4.-6.3.2015 in Hamburg, Germany. Proceedings . 383--402. https://dl.gi.de/20.500.12116/2418Google Scholar
- Thomas Neumann, Tobias Mü hlbauer, and Alfons Kemper. 2015. Fast Serializable Multi-Version Concurrency Control for Main-Memory Database Systems. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, Melbourne, Victoria, Australia, May 31 - June 4, 2015 . 677--689. Google ScholarDigital Library
- Thomas Neumann and Bernhard Radke. 2018. Adaptive Optimization of Very Large Join Queries. In Proceedings of the 2018 International Conference on Management of Data (SIGMOD '18). ACM, New York, NY, USA, 677--692. Google ScholarDigital Library
- Mark Raasveldt and Hannes Mü hleisen. 2017. Don't Hold My Data Hostage - A Case For Client Protocol Redesign. PVLDB, Vol. 10, 10 (2017), 1022--1033.Google ScholarDigital Library
- Mark Raasveldt and Hannes Mü hleisen. 2018. MonetDBLite: An Embedded Analytical Database. CoRR, Vol. abs/1805.08520 (2018). arxiv: 1805.08520 http://arxiv.org/abs/1805.08520Google ScholarDigital Library
- Hadley Wickham, Romain Franccois, Lionel Henry, and Kirill Müller. 2018. dplyr: A Grammar of Data Manipulation . https://CRAN.R-project.org/package=dplyr R package version 0.7.8.Google Scholar
Index Terms
- DuckDB: an Embeddable Analytical Database
Recommendations
DuckDB-wasm: fast analytical processing for the web
We introduce DuckDB-Wasm, a WebAssembly version of the database system DuckDB, to provide fast analytical processing for the Web. DuckDB-Wasm evaluates SQL queries asynchronously in web workers, supports efficient user-defined functions written in ...
DuckPGQ: Bringing SQL/PGQ to DuckDB
We demonstrate the most important new feature of SQL:2023, namely SQL/PGQ, which eases querying graphs using SQL by introducing new syntax for pattern matching and (shortest) path-finding. We show how support for SQL/PGQ can be integrated into an RDBMS, ...
Comments