Loading
Research Projects
Evaluating Starfish in the Real World
|
Abstract
Starfish is a self-tuning analytics system designed for the Hadoop
distributed computing platform. Starfish, which has been under
development since 2010, addresses many challenges that Hadoop users
are facing. Starfish has shown promising results for several benchmark
workloads. Meanwhile, the Hadoop platform has also been growing
rapidly and getting widely adopted. As the next step, we seek to
partner with heavy users of Hadoop (including Baidu and Twitter) to
evaluate Starfish in the real world. Based on the results of the
evaluation, we will revisit some core decisions in the design and
implementation of Starfish. The goals of this project are to: (i)
evaluate the effectiveness of Starfish on real world applications;
(ii) examine the extensibility of Starfish to address new problems in
the growing Hadoop ecosystem; and (iii) develop a deep understanding
of the Hadoop industry to identify upcoming challenges that will guide
our future research.
Advisor(s): Shivnath Babu
Jun Yang, Benjamin Lee