Benchmarks

Overview

• Overview paper: Setting the Direction for Big Data Benchmark Standards by C. Baru, M. Bhandarkar, R. Nambiar, M. Poess, and T. Rabl, was published in Selected Topics in Performance Evaluation and Benchmarking, Lecture Notes in Computer Science, Volume 7755, 2013, pp 197-208. [ Abstract ] [ Full Paper ]
• Presentations on BigData Top100 List initiative:
• Bhandarkar and Baru at the Strata Conference, February 26-28, 2103, Santa Clara, CA [ pdf ]
• Baru, Database Seminar, CSE Dept, UC San Diego, March 1, 2013 [ pdf ]

Benchmark specifications currently under consideration

• Data Analytics Pipeline

  • Introductory paper: Benchmarking Big Data Systems and the BigData Top100 List by Baru, Bhandarkar, Nambiar, Poess, Rabl, Big Data Journal, Vol.1, No.1, March 2013.
  • BDBC Presentation: Deep Analytics Pipeline: A Benchmark Proposal, Milind Bhandarkar, March 7, 2013 [ Audio. Enter your name to access the ReadyTalk recording. ][ Slides (pdf) ]

• BigBench

  Proposal to extend TPC-DS specification to include unstructured and semi-structured data; modify the TPC-DS query set to include operations on these data; and incorporate data mining procedures in some of the queries. A data model for BigBench was proposed in the First WBDB workshop by Ghazal [ PDF ]. This was expanded with a set of associated queries at the Second WBDB workshop by Ghazal et al [ PDF ].