Database Systems: Big Data Evolution and Efficiency

Big Data: A Continuing Evolution

Big Data today is continuing to evolve, and appears to be in the beginning stages of evolution. It will continue to grow and need constant research initiatives to keep up. This paper will look at the definition of Big Data and how it is being used, why the current DBMS is unable to handle Big Data efficiently, what hardware and software solutions are being tested, and what challenges the researchers are facing.

Big Data is a term used today to talk about the vastly growing amounts of data, (mainly unstructured, but can also include structured and semi structured data), out there to be mined [1]. Data mining attempts to derive meaningful information from data. As the amount of data in different varieties keeps increasing, it becomes harder to process useful information at an acceptable return time rate. Current software tools and hardware are failing to keep up with Big Data needs. Big Data requires one to be able to process complex computer data at the Petabyte or Exabyte level. [2].

Big Data is developing from many sources, and with storage capacity has been doubling in size every 14 months for the past 30 years, keeping data has become cheaper and cheaper [3]. Some of the sources of data are social media on the Internet, mobile sensors, astronomy, transaction logs, and many more [4]. Companies today desire to collect mass amounts of data that may not be useful today, but could be later. The popular social media site, Facebook, collects over 500 Terabytes of day a day [4]. The term Big Data is not only defined by its volume, by its ability to retrieve knowledge in a reasonable amount of time. For example, Netflix, a video streaming service, utilizes a machine-learning technique ...

... middle of paper ...

...the Future," ACM SIGKDD Explorations Newslette, vol. 14, no. 2, pp. 1-5, 2012.

[6] C. Ordonez, "Can We Analyze Big Data inside a DBMS?," in DOLAP '13 Proceedings of the sixteenth international workshop on Data warehousing and OLAP, San Francisco, 2013.

[7] C. Bear, A. Lamb and N. Tran, "The Vertica Database: SQL RDBMS For Managing Big Data," in MBDS '12 Proceedings of the 2012 workshop on Management of big data systems, San Jose, 2012.

[8] N. Tran, S. Bodagala and J. Dave, "Designing query optimizers for big data problems of the future," Proceedings of the VLDB Endowment, vol. 6, no. 11, pp. 1168-1169, 2013.

[9] S. Madden, "From Databases to Big Data," IEEE Internet Computing, vol. 16, no. 3, pp. 4-6, May 2012.

[10] S. W. Cunningham, "Big Data and Technology Readiness Levels," Engineering Management Review, IEEE , vol. 42, no. 1, pp. 8-9, 2014.