Research Projects

Evaluating Starfish in the Real World

Speaker:Jie Li
jieli at cs.duke.edu
Date: Thursday, May 10, 2012
Time: 4:00pm - 5:00pm
Location: D344 LSRC, Duke
Jun Yang, Benjamin Lee

Abstract

Starfish is a self-tuning analytics system designed for the Hadoop distributed computing platform. Starfish, which has been under development since 2010, addresses many challenges that Hadoop users are facing. Starfish has shown promising results for several benchmark workloads. Meanwhile, the Hadoop platform has also been growing rapidly and getting widely adopted. As the next step, we seek to partner with heavy users of Hadoop (including Baidu and Twitter) to evaluate Starfish in the real world. Based on the results of the evaluation, we will revisit some core decisions in the design and implementation of Starfish. The goals of this project are to: (i) evaluate the effectiveness of Starfish on real world applications; (ii) examine the extensibility of Starfish to address new problems in the growing Hadoop ecosystem; and (iii) develop a deep understanding of the Hadoop industry to identify upcoming challenges that will guide our future research.
Advisor(s): Shivnath Babu