Dandelion: Cooperative Content Distribution with Robust Incentives

Michael Sirivianos, Jong Han Park, Xiaowei Yang and  Stanislaw Jarecki

Background

Resume

Research Interests

Projects

Presentations

Links


      

dandelion.jpeg

Motivation

Online content distribution has been increasingly gaining popularity among the entertainment industry and the consumers alike.  A key challenge in online content distribution is a cost-efficient solution to handle demand peaks. To address this challenge, we propose Dandelion, a system for cooperative (peer-to-peer) distribution of paid content. Dandelion explicitly deals with a crucial issue in cooperative content distribution. It provides robust incentives for a client to upload content to its peers, regardless of whether its peers have content that interest it. A client that honestly serves its peers is rewarded with credit that can be redeemed for monetary rewards such as discounts on paid content. Since credit in Dandelion corresponds to real monetary value, it is imperative that the content provider can reliably keep track of a client's cooperativeness, without running the risk of its clients manipulating the credit mechanism. 

Our  evaluation of a prototype system running on commodity hardware with 5 Mb/s uplink and 5 Mb/s downlink indicates that Dandelion can support ~3100 clients exchanging 256KB chunks with each other at 256KB/sec.


Design

The premise of our design is that a low cost server may have limited outgoing bandwidth but sufficient CPU power, memory and bandwidth to receive and send short messages, execute many short cryptographic operations and maintain TCP connection state with its clients under a flash crowd event.

A Dandelion server can be used to distribute both small and large static files, depending on the specifics of its deployment. It behaves similar to a web/ftp server under normal work load, responding to clients' requests for content. When a Dandelion server is overloaded, it enters a peer-serving mode. Upon receiving a request, the server redirects the client to clients that are able to serve the request. 

A Dandelion server maintains a virtual economy. It rewards cooperative clients that upload to others with virtual credit to provide robust incentives.  The credit is used as ``virtual money'' to purchase future downloads from other clients or from the server itself (at a high credit cost when the server is overloaded), or for other types of rewards. The Dandelion server and the credit bank are logical modules and could be distributed over multiple trusted nodes to improve scalability.

Similar to BitTorrent, a Dandelion server splits a large file into multiple chunks, and disseminates them independently. This allows clients to participate in uploading chunks as soon as they receive a small portion of the file, increasing the efficiency of the distribution pipeline.  

A key challenge in designing a reliable credit system is to prevent client cheating, while keeping both a server and a client's processing and bandwidth costs low.  A dishonest client that does not upload to others or uploads garbage may attempt to claim credit at the server, and to be robust, the server must not award credit to such cheating behavior. To address this challenge, Dandelion employs two cryptographic fair exchange mechanisms. In the first mechanism, a Dandelion server serves as the trusted third party mediating the exchanges of content for credit among its clients in case clients are not mutually interested in each other's content. When a client A uploads to a client B, it sends encrypted content to client B. To decrypt, B must request keys from the server. The requests for keys serve as the ``proof'' that A has uploaded some content to B. Thus, when the server receives a key request, it credits A for uploading to B, and charges B for the content. The second cryptographic mechanism draws from BAR Gossip and classic optimistic fair-exchange mechanisms. It is used when clients are mutually interested in each other's content and ensures absolute fairness in tit-for-tat exchanges of chunks.

A misbehaving client A may send invalid content to B.  B can discover that the content is invalid only after receiving the decryption key and being charged. To address this problem, our design includes a non-repudiable complaint mechanism. If A intentionally sents garbage to BA cannot deny it. In addition, B is prevented from falsely claiming that A has sent it garbage. Figure 1 gives an overview of the Dandelion protocol.

TheProtocol.bmp

Fig 1. The Dandelion protocol



Implementation Overview

The Dandelion server and client are implemented in approximately 10000 lines of C code. We implemented Dandelion's cryptographic operations using the OpenSSL C library and the credit management system using the standard C file I/O libraries. In particular we use CBC-blowfish encryption, the SHA-1 digest function, SHA-1-based HMAC and 1024-bit RSA public key cryptography.  The credit base is stored as a simple file. In the multithreaded version of the server, we use NPTL for efficient multithreading. 


Publications

Michael Sirivianos, Xiaowei Yang and Stanislaw Jarecki: Robust and Efficient Incentives for Cooperative Content Distribution, bibtex, IEEE/ACM Transactions on Networking (to appear) 

Michael Sirivianos, Jong Han Park, Xiaowei Yang and Stanislaw Jarecski. Dandelion: Cooperative Content Distribution with Robust Incentives, USENIX 2007, bibtex, talk.ppt 

The talk reports the latest throughput results for an optimized epoll()-based server. A server running on a  Pentium D 2.8GHZ/1MB CPU with 1GB RAM and 250GB/7200RPM HDD machine running Linux 2.6.5-1.358smp, with 5Mbps uplink and downlink can process ~3100 decryption key requests per second from ~1000 clients. The previous server versions could process ~1200 req/sec from ~200 clients as reported in the USENIX paper.

Michael Sirivianos, Xiaowei Yang and Stanislaw Jarecki. Dandelion: Cooperative Content Distribution with Robust Incentives, NetEcon 2006.

Michael Sirivianos, Jong Han Park, Rex Chen and Xiaowei Yang. Free-riding in BitTorrent Networks with the Large View exploit, IPTPS 2007. 


Source Code

Dandelion's source code, a brief protocol specification,   instructions on how to use and on how to deploy in PlanetLab, as well as the required scripts can be found in dandelion.tar.gz. If you are new to PlanetLab, check out these instructions.

This version, supports a hybrid incentive scheme, in which peers employ crypto fair-exchange incentives when they are not mutually interested in each other's content. If peers are mutually interested, they employ rate-based tit-for-tat with limited trade surplus. The number of parallel downloaders is dynamically adjusted according to the specified max upload rate and the uplink utilization.

The code for Dandelion that uses the optimistic fair-exchange scheme for tit-for-tat incentives as reported in the ToN paper is being currently cleaned up and is readily available at its current state upon email request.