Benchmarks

Overview

  • Overview paper: Setting the Direction for Big Data Benchmark Standards by C. Baru, M. Bhandarkar, R. Nambiar, M. Poess, and T. Rabl, was published in Selected Topics in Performance Evaluation and Benchmarking, Lecture Notes in Computer Science, Volume 7755, 2013, pp 197-208. [ Abstract ] [ Full Paper ]
  • Presentations on BigData Top100 List initiative:
    • Bhandarkar and Baru at the Strata Conference, February 26-28, 2103, Santa Clara, CA [ pdf ]
    • Baru, Database Seminar, CSE Dept, UC San Diego, March 1, 2013 [ pdf ]

Benchmark specifications currently under consideration

  • Data Analytics Pipeline

  • BigBench

    Proposal to extend TPC-DS specification to include unstructured and semi-structured data; modify the TPC-DS query set to include operations on these data; and incorporate data mining procedures in some of the queries. A data model for BigBench was proposed in the First WBDB workshop by Ghazal [ PDF ]. This was expanded with a set of associated queries at the Second WBDB workshop by Ghazal et al [ PDF ].