Sage Journals: Discover world-class research

Abstract

Classical databases represent the traditional DBMS’s and the most widely used DBMS in the world of databases and information systems; they have been regarded as the best systems for managing data. Today with the growth of data for both applications and consumers, and its openness to the public, traditional databases are not able to meet the needs of a large number of applications, including Online Analytical Processing models: OLAP data processing and Business Intelligence analysis. As a result, many dedicated DBMS have emerged like: Column Store, In Memory and NoSQL databases. They meet users’ expectations and fit well with current needs. Consequently, the scope of classical databases has become increasingly restricted to handle Online Transaction Processing OLTP models and small problems. However, those dedicated solutions hardly cope with rich features of DBMS like simplicity, flexibility or scalability. They remain limited to mainly process single database models, and users cannot deal with both OLTP and OLAP queries in a single environment. To deal with this problem, vertical fragmentation is the best way to effectively handle the OLAP model, but this technique fails to handle some analytical queries with low selectivity, presenting poor results in some cases. In this perspective, we propose a new vertical fragmentation design T-Plotter which makes it possible to deal effectively with the whole of analytical queries and improve the performance of DBMS’s to process the OLAP data models.

Keywords

OLAP databases OLTP databases vertical fragmentation physical design TPCH benchmark

Get full access to this article

View all access options for this article.

References

Abadi

D.J.

Myers

D.S.

DeWitt

D.J.

and Madden

S.R.

, Materialization Strategies in a Column-Oriented DBMS, in: IEEE International Conference on Data Engineering (ICDE), Istanbul, Turkey, 2007, pp. 466–475. https://doi.org/10.1109/ICDE.2007.367892.

Abadi

D.J.

Boncz

P.A.

Harizopoulos

Idreos

and Madden

, The design and implementation of modern column-oriented database systems, Foundations and Trends in Databases 5(3) (2013), 197–280. https://doi.org/10.1561/1900000024.

Abadi

D.J.

Madden

and Ferreira

, Integrating compression and execution in column-oriented database systems, in: International Conference on Management of Data (SIGMOD), ACM, New York, USA, 2006, pp. 671–682. https://doi.org/10.1145/1142473.1142548.

Abadi

D.J.

Madden

and Hachem

, Column-stores vs. row-stores: How different are they really? in: ACM International Conference on Management of Data (SIGMOD), Vancouver, BC, Canada, 2008, pp. 967–980. https://doi.org/10.1145/1376616.1376712.

Agrawal

and Srikant

, Fast algorithms for mining association rules in large databases, in: International Conference on Very Large Data Bases (VLDB), San Francisco, CA, USA, 1994, pp. 487–499.

Boissier

Spivak

and Meyer

, Improving tuple reconstruction for tiered column stores: A workload-aware ansatz based on table reordering, in: Proceedings of the Australasian Computer Science Week Multiconference (ACSW), ACM, Geelong, Australia, 2017, pp. 25:1–25:10.

Boncz

P.A.

Zukowski

and Nes

, MonetDB/X100: Hyper-Pipelining Query Execution, in: Biennial Conference on Innovative Data Systems Research (CIDR), CA, USA, 2005, pp. 225–237.

Chaalel

and Belbachir

, An optimized vertical fragmentation approach, IJITEE International Journal of Innovative Technology and Exploring Engineering 3(4) (2013), 33–39.

Chaudhuri

and Dayal

, An overview of data warehousing and olap technology, International Conference on Management of Data (SIGMOD) Rec 26(1) (1997), 65–74. https://doi.org/10.1145/248603.248616.

10.

Cheng

C.H.

Lee

W.-K.

and Wong

K.-F.

, A genetic algorithm-based clustering approach for database partitioning, IEEE Trans Systems, Man, and Cybernetics, Part C 32 (2002), 215–230.

11.

Gorla

and Betty

P.W.

, Vertical fragmentation in databases using data-mining technique, International Journal of Data Warehousing and Mining (IJDWM) 4(3) (2008), 35–53.

12.

Hammer

and Niamir

, A heuristic approach to attribute partitioning, in: Proceedings of the International Conference on Management of Data (SIGMOD), ACM, New York, NY, USA, 1979, pp. 93–101. https://doi.org/10.1145/582095.582110.

13.

Hoffer

J.A.

and Severance

D.G.

, The use of cluster analysis in physical data base design, in: International Conference on Very Large Data Bases (VLDB), Framingham, Massachusetts, USA, 1975, pp. 69–86.

14.

Idreos

Kersten

M.L.

and Manegold

, Self-organizing tuple reconstruction in column-stores, in: International Conference on Management of Data (SIGMOD), ACM, New York, USA, 2009, pp. 297–308. https://doi.org/10.1145/1559845.1559878.

15.

Navathe

S.B.

Ceri

Wiederhold

and Dou

, Vertical partitioning algorithms for database design, ACM Trans Database Syst 9 (1984), 680–710.

16.

Navathe

S.B.

and Ra

, Vertical partitioning for database design: A graphical algorithm, in: Proceedings of International Conference on Management of Data (SIGMOD), ACM, New York, NY, USA, 1989, pp. 440–450. https://doi.org/10.1145/67544.66966.

17.

Petraki

Idreos

and Manegold

, Holistic indexing in main-memory column-stores, in: Proceedings of International Conference on Management of Data (SIGMOD), ACM, Melbourne, Victoria, Australia, 2015, pp. 1153–1166. https://doi.org/10.1145/2723372.2723719.

18.

Plattner

, The impact of columnar in-memory databases on enterprise systems: Implications of eliminating transaction-maintained aggregates, Proc VLDB Endow 7(13) (2014), 1722–1729. https://doi.org/10.14778/2733004.2733074.

19.

Song

S.-K.

and Gorla

, A genetic algorithm for vertical fragmentation and access path selection, The Computer Journal 43(1) (2000), 81–93. https://doi.org/10.1093/comjnl/43.1.81.

20.

Stonebraker

Abadi

D.J.

Batkin

Chen

Cherniack

Ferreira

Lau

Lin

Madden

O’Neil

E.J.

O’Neil

P.E.

Rasin

Tran

and Zdonik

, C-store: A column-oriented DBMS, in: Making Databases Work: the Pragmatic Wisdom of Michael Stonebraker, 2019, pp. 491–518. https://doi.org/10.1145/3226595.3226638.

21.

Transaction Processing Performance Council (TPC), TPCH: ad-hoc, decision support benchmark (1999), Standard Specification Revision 218.0., http://www.tpc.org/tpch.

22.

Vermeij

Quak

Kersten

and Nes

, MonetDB, a novel spatial columnstore DBMS, in: Academic Proceedings of the 2008 Free and Open Source for Geospatial (FOSS4G) Conference, OSGeo, Cape Town, South Africa, 2008, pp. 193–199.

23.

Zukowski

Boncz

P.A.

Nes

and Héman

, Monetdb/x100 – A DBMS in the CPU cache, IEEE Data Eng Bull 28(2) (2005), 17–22. ftp://ftp.research.microsoft.com/pub/debull/A05june/mzukowski.ps.

T-plotter: A new data structure to reconcile OLAP and OLTP models

Abstract

Keywords

Get full access to this article

References