Spring 1999
Instructor: Amin Vahdat (vahdat@cs.duke.edu)
TA: Surendar Chandra (surendar@cs.duke.edu)
Time: MWF 1:10-2:00 (D106 LSRC)
Suggestion
Box (anonymous email to instructor).
| Course Summary/Objectives |
The enormous popularity of the web provides a tremendous
opportunity and challenge to rethink the foundations of wide area
distributed systems. Internet use is doubling every six months
(faster than Moore's Law -- the rate of improvement in hardware
performance); within a few years, the number of active users on
the Internet will pass 100 million. How would you build a web
services able to gracefully scale across an an order of magnitude
variance in demand. For example, could we build a web
server for CNN to recruit additional resources as necessary
during national elections or after breaking news stories (the
Starr report)? Can we provide a reliable email account to
every schoolchild in the US?
While the Web today provides just a read-only file system
(albeit with active content), one emerging vision is to provide
seamless access to computational and data resources around the
Internet, in effect extending the local area network computing
model to the wide area. How should traditional concerns such as
naming, persistence, resource management, and security be
provided in a system of Internet scale? What support do we need
for applications that are geographically distributed, highly
available, incrementally scalable, and dynamically reconfiguring
(e.g., can we support dynamic migration and replication of web
services)?
First-year graduate students and undergraduates with
appropriate background are strongly encouraged to attend. We will
provide some material to be read off-line to fill in gaps for
anyone missing some background.
Introduction/Overview
(Jan. 13 - Jan. 22) |
Question: What is the right model for supporting Internet
applications of the future?
- Jan. 13: Amin Vahdat, Thomas Anderson, Michael Dahlin,
Eshwar Belani, David Culler, Paul Eastham, and Chad
Yoshikawa,WebOS:
Operating System Services For Wide Area Applications.
Proceedings of High Performance Distributed Computing.
July 1998.

- Jan. 15: Michael Lewis and Andrew Grimshaw, The
Core Legion Object Model. Fifth IEEE International
Symposium on High Performance Distributed Computing,
August 1996.

- Jan. 15: Andrew Grimshaw, Michael Lewis, Adam Ferrari,
and John Karpovich, Architectural
Support for Extensibility and Autonomy in Wide-Area
Distributed Object Systems. UVa CS Technical Report
CS-98-12, June 1998.

- Jan 18: Martin Luther King holiday
- Jan. 20: Ulana Legcdza, David Wetherall, and John Guttag,
Improving
the Performance of Distributed Applications Using Active
Networks, Proceedings of IEEE INFOCOMM, 1998.

- Jan. 22: M. van Steen, P. Homburg, and A.S. Tanenbaum. The
Architectural Design of Globe: A Wide-Area Distributed
System. Technical Report IR-422, March 1997.

- Jan. 22: Ian. Foster and Carl Kesselman, Globus:
A Metacomputing Infrastructure Toolkit. International
Journal of Supercomputer Applications11(2):115-128, 1997.

For more depth:
- David Wetherall, Ulana Legedza, and John Guttag, "Introducing
New Internet Services: Why and How." IEEE
Network Magazine, July/August 1998.
- Ian Foster and Carl Kesselman, "The
Globus Project: A Status Report." Proc.
IPPS/SPDP '98 Heterogeneous Computing Workshop, pg. 4-18,
1998.
- John Hartman, Udi Manber, Larry Peterson, and Todd
Proebsting, Liquid
Software: A New Paradigm for Networked Systems.
Technical Report 96-11, Department of Computer Science,
University of Arizona, June 1996.
- Henri Casanova and Jack Dongarra. NetSolve:
A Network Server for Solving Computational Science
Problems
- Peter Danzig, Dante DeLucia, and Katia Obraczka. Massively
Replicating Services in Wide-Area Internetworks.
University of Southern California.
- B. Christiansen, P. Cappello, M. F. Ionescu, M. O. Neary,
K. E. Schauser, and D. Wu "Javelin:
Internet-Based Parallel Computing Using Java."
Proceedings of the ACM Workshop on Java for Science and
Engineering Computation, June 1997.
- Lucent Corp., An Overview of Plan 10. Inferno
Summary, and Inferno
Overview.
- JavaSpaces,
Javasoft.
- Draves et al., "Operating
System Directions for the Next Millennium"
- U.C. Berkeley
Ninja (Brewer, Joseph, Katz)
Locally Scalable Services
(Jan. 25 - 29) |
Question: As a subproblem, how do we build a scalable
Internet service located at a single site (which we might have to
replicate to get end-to-end availability)? What abstractions
should we provide to support scalability to millions of users,
continuous 24x7 operation, etc.
- Jan. 25: Jim Gray, "A Census of Tandem System
Availability Between 1985 and 1990." IEEE
Transactions on Reliability v39 n4 October 1990 pp.
409-418.

- Jan. 27: Vivek Pai, Mohit Aron, Gaurav Banga, Michael
Svendsen, Peter Druschel, Willy Zwaenepoel, and Eric
Nahum, "Locality-Aware
Request Distribution in Cluster-based Network
Servers." Proc. 8th International Conference on
Architectural Support for Programming Languages and
Operating Systems, October 1998.

- Jan. 29: Thomas Anderson, David Culler, and David
Patterson, "A Case
for Networks of Workstations: NOW." IEEE Micro,
February 1995.

- Jan. 29: Armando Fox, Steven Gribble, Yatin Chawathe,
Eric Brewer, and Paul Gauthier, "Cluster-Based
Scalable Network Services." Proceedings of the
Symposium on Operating Systems Principles, October 1997.

For more depth:
- B. Liskov, S. Ghemawat, R. Gruber, P. Johnson, L. Shrira,
and M Williams, "Replication
in the Harp File System." Proceedings of the
Symposium on Operating System Principles, pages 226--238,
October 1991.
- Chandramohan A. Thekkath, Timothy Mann, and Edward K.
Lee, "Frangipani: A Scalable Distributed File
System." Proceedings of Symposium on Operating
Systems Principles, October 1997.
- Kenneth P. Birman. The Process Group Approach to Reliable
Distributed Computing. Communications of the ACM 36(12),
December 1993, pp. 37-53.
- Thomas Anderson, Michael Dahlin, Jeena Neefe, David
Patterson, Drew Roselli, Randy Wang, "Serverless
Network File Systems." ACM Transactions on
Computer Systems, 1996.
- M.J. Feeley, W.E. Morgan, F.P. Pighin, A.R. Karlin, H.M.
Levy, C.A. Thekkath Implementing
Global Memory Management in a Workstation Cluster,
Proceedings of the Symposium on Operating System
Principles, December 1995.
- Greg Pfister, "In Search of In Search of Clusters:
The Coming Battle in Lowly Parallel Computing."
1995.
- John Chapin, Mendel Rosenblum, Scott Devine, Tirthankar
Lahiri, Dan Teodosiu, Anoop Gupta, "Hive:
Fault Containment for Shared-Memory Multiprocessors."
Proceedings of the Symposium on Operating System
Principles, December 1995.
Question: How should we translate from virtual names to
physical addresses? How do we find objects or services or users
in the Internet if they migrate?
- Feb. 1: P. Mockapetris and K. Dunlap "Development of
the Domain Name System." Proc. SIGCOMM
Computer Communications Review Vol 18 No 4 August 1988
pp. 123-133.

- Feb.1: Ron Daniel and Michael Mealling. Resolution
of Uniform Resource Identifiers using the Domain Name
System. Internet Draft, September 1996.

- Feb. 3: Michael Schroeder, Andrew Birrell, and Roger
Needham, "Experience with Grapevine: The Growth of a
Distributed System." ACM Transactions on Computer
Systems, Vol. 2, No. 1 (Feb 1984), pp. 3-23.

- Feb. 5: Butler Lampson, "Designing a Global Name
Service." Proceedings of the Fifth Annual ACM
Symposium on Principles of Distributed Computing, pages
1-10, August 1986.

- Feb. 8: J. D. Guyton and M. F. Schwartz, "Locating
Nearby Copies of Replicated Internet Servers",
In Proceedings of ACM SIGCOMM, 1995.

- Feb. 8: M. van Steen, F.J. Hauck, P. Homburg, and A.S.
Tanenbaum. "Locating
Objects in Wide-Area Systems," IEEE
Communications Magazine, January 1998.

- Feb. 10: Amin Vahdat, Michael Dahlin, Thomas Anderson,
and Amit Aggarwal, "Active
Names: Programmable Location and Transport of Wide-area
resources" Technical Report, February 1999.

For more depth:
- B. Clifford Neuman. The Prospero File System: A global
file system based on the Virtual System Model. Computing
Systems, 5(4):407-432, Fall 1992.
- Chad Yoshikawa, Brent Chun, Paul Eastham, Amin Vahdat,
Thomas Anderson, and David Culler. ``Using Smart Clients
to Build Scalable Services.'' Proceedings of the 1997
USENIX Conference, January 1997.
- Center for Networked Information Discovery and Retrieval.
The
Whois++ Directory Service Testbed.
- Steven Breitstein. Inferno
Namespaces.
- Edward Slottow. Engineering
a Global Resolution Service, Nov. 1996.
- C. Partridge, T. Mendez, and W. Milliken. Host
Anycasting Service.
- Zongming Fei, Samrat Bhattacharjee, Ellen W. Zegura and
Mostafa Ammar. A
Novel Server Selection Technique for Improving the
Response Time of a Replicated Service. Infocom'98.
- S. Bhattacharjee, M. Ammar, E. Zegura, V. Shah and Z.
Fei. Application-Layer
Anycasting. IEEE Infocom '97, Kobe, Japan, April
1997.
- A
Directory Service for Configuring High-Performance
Distributed Computations. S. Fitzgerald, I. Foster,
C. Kesselman, G. von Laszewski, W. Smith, and S. Tuecke,
Proc. 6th IEEE Symp. on High-Performance Distributed
Computing 1997.
Wide Area Communication and Persistence
(Feb. 12 - 22) |
Question: How do we coordinate the activities of a
geographically distributed application? What if the application
needs to update persistent state?
- Feb. 12: H. Eriksson "MBone: The Multicast
Backbone" CACM Aug1994 v37 pp 54--60.

- Feb. 15: M. Spasojevic and M. Satyanarayanan.
"A
Usage Profile and Evaluation of a Wide-Area Distributed
File System." Proceedings of the USENIX Winter
Technical Conference, Jan. 1994.

- Feb. 15: Brian Oki, Manfred Pflug, Alex Siegel, and Dale
Skeen. "The Information Bus - An Architecture for
Extensible Distributed Systems." Proceedings of the
Symposium on Operating System Principles, December 1993.

- Feb 17: B. Noble, M. Satyanarayanan, D. Narayanan, J. E.
Tilton, J. Flinn, K. Walker. "Agile
Application-Aware Adaptation for Mobility."
Proceedings of the 16th ACM Symposium on Operating System
Principles.
- Feb. 19: Karin Petersen, Mike J. Spreitzer, Douglas B.
Terry, Marvin M. Theimer, and Alan J. Demers. "Flexible
Update Propagation for Weakly Consistent
Replication." Proceedings of the Symposium on
Operating System Principles, October 1997.

- Feb. 22: Björn Grönvall, Assar Westerlund, and Stephen
Pink. "The
Design of a Multicast-based Distributed File
System." Proceedings of Operating System Design
and Implementation, February 1999.

For more depth:
- Douglas Terry, Marvin Theimer, Karin Petersen and Alan
Demers and Michael Spreitzer and Carl Hauser, "Managing
Update Conflicts in Bayou, a Weakly Connected Replicated
Storage System". Proceedings of the
Symposium on Operating System Principles, December 1995.
- Radek Vingralek, Yuri Breitbart, Mehmet Sayal, Peter Scheuermann, "Web++: A System for Fast and Reliable Web Service." Proceedings of the USENIX Technical Conference, June 1999.
- B. Callaghan "WebNFS:
The File System for the World Wide Web" Sun
White Paper May 1996.
- D. Wessels, K. Claffy, "Internet
Cache Protocol (ICP), version 2". May 1997.
- J. Kistler and M. Satyanarayanan, "Disconnected
Operation in the Coda File System." ACM Transactions
on Computer Systems, Vol. 10, No. 1, pp. 3-25, February
1992.
- Randy Wang and Thomas Anderson, "xFS: A Wide Area
Mass Storage File System." Proc, Fourth Workshop on
Workstation Operating Systems, October 1993.
- P. Homburg, M. van Steen, and A.S. Tanenbaum. "Unifying
Internet Services Using Distributed Shared Objects."
Technical Report IR-409, October 1996.
Wide-Area Caching
(Feb. 26 - Mar. 3) |
- Feb. 26: Sally Floyd, Van Jacobson, Lixia Zhang, "Adaptive
Web Caching", Proc. 2nd Web Cache Workshop,
1997.

- Feb. 26: Bradley M. Duska, David Marwood, and Michael J.
Feeley, "The
Measured Access Characteristics of World-Wide-Web Client
Proxy Caches." Proc. 1st USENIX Symposium
on Internet Technologies and Systems, December 1997.

- Mar. 1: Li Fan, Pei Cao, Jussara Almeida and Andrei
Broder, "Summary
Cache: A Scalable Wide-Area Web Cache Sharing
Protocol." Proceedings of SIGCOMM'98, September
1998.

- Mar. 3: Pei Cao, Jin Zhang and Kevin Beach, "Active
Cache: Caching Dynamic Contents on the Web."
Proc. Middleware, 1998.

- Mar. 3: Amin Vahdat and Thomas Anderson, "Transparent
Result Caching." Proc. USENIX Technical
Conference, June 1998.

For more depth:
- Edith Cohen, Balachander Krishnamurthy, and Jeniffer Rexford, "Improving End-to-End Performance of the Web Using Server Volumes and Proxy Filters." Proceedings of ACM SIGCOMM, September 1998.
- Jeffrey Gilbert and Robert Brodersen, "Globally Progressive Interactive Web Delivery." Proceedings of IEEE INFOCOM'99, March 1999.
- Eric Levy-Abegnoli, Arun Iyenger, Junehwa Song, Daniel Dias, "Design and Performance of a Web Server Accelerator." Proceedings of IEEE INFOCOM'99, March 1999.
- Quinn Jacobson and Pei Cao, "Potential
and Limits of Web Prefetching Between Low-Bandwidth
Clients and Proxies." Proc. 3rd Web
Caching Workshop, April 1998.
- James Gwertzman and Margo Seltzer, "World-Wide
Web Cache Consistency." Proceedings of the 1996
Usenix Technical Conference, January 1996.
- Anawat Chankhunthod, Peter B. Danzig, Chuck Neerdaels,
Michael F. Schwartz and Kurt J. Worrell. "A
Hierarchical Internet Object Cache." Proceedings
of the 1996 Usenix Technical Conference, January 1996.
- Pei Cao and Sandy Irani, "Cost-Aware
WWW Proxy Caching Algorithms." Proceedings of
the 1997 USENIX Symposium on Internet Technology and
Systems, December, 1997.
- Jeffrey Mogul, Fred Douglis, Anja Feldmann and
Balachander Krishnamurthy, "Potential
benefits of delta encoding and data compression for
HTTP." Proceedings of ACM SIGCOMM '97, August
1997.
- Balachander Krishnamurthy and Craig Wills, "Study
of Piggyback Cache Validation for Proxy Caches in the
WWW." Proceedings of the 1997 USENIX Symposium
on Internet Technology and Systems, December 1997.
Security
(Mar. 5 - Mar. 24) |
Question: How do we provide secure access to wide area
remote resources (cycles, storage, and data)?
- Mar. 5: Butler Lampson, Martin Abadi, Michael Burrows,
and Edward Wobber. "Authentication
in Distributed Systems: Theory and Practise."
Proceedings of the Symposium on Operating System
Principles, 1991. This link is for the full version
published as a technical report.

- Mar. 8: Ronald Rivest and Butler Lampson,
SDSI - A Simple Distributed Security Infrastructure,
1996.

- Mar. 10: Eshwar Belani, Amin Vahdat, Thomas Anderson, and
Michael Dahlin. "The
CRISIS Wide Area Security Architecture."
Proceedings of the USENIX Security Symposium, January
1998.

- Mar. 12: Dan S. Wallach, Dirk Balfanz, Drew Dean, and
Edward W. Felten, "Extensible
Security Architecture for Java." Proceedings of
the Symposium on Operating System Principles, October,
1997.

- Mar. 15 - 19: Spring Break.
- Mar. 22: Ian. Foster, Carl Kesselman, Gene Tsudik, Steven
Tuecke, "A
Security Architecture for Computational Grids."
Proc. 5th ACM Conference on Computer and Communication
Security, 1998.

- Mar. 24: Li Gong, Marianne Mueller, Hemma Prafullchandra,
and Roland Schemers, "Going Beyond the Sandbox: An
Overview of the New Security Architecture in the Java
Development Kit 1.2." Proc. 1st USENIX Symposium on
Internet Technologies and Systems, December 1997.

For more depth:
- Edward Wobber, Martin Abadi, Mike Burrows, and Butler
Lampson, "Authentication
in the Taos Operating System." Proceedings of
the Symposium on Operating System Principles, December
1993. This link is for the full version published
as a technical report.
- David Wagner, Ian Goldberg, and Randi Thomas, "A
Secure Environment for Untrusted Helper Applications:
Confining the Wily Hacker." Proceedings of the
USENIX Security Symposium, 1996.
- Martin Abadi and Roger Needham, "Prudent
Engineering Practice for Cryptographic Protocols".
- J. Steiner and C. Neuman and J. Schiller, "Kerberos:
An Authentication Service for Open Network Systems."
Proceedings of the USENIX Winter Technical Conference,
February 1988.
- S M Bellovin and M Merritt, "Limitations of the
Kerberos Authentication System."
- M. Satyanarayanan. "Integrating security in a large
distributed system." ACM Transactions on Computer
Systems, 7(3):247-280, August 1989.
- Michael. Burrows, Martin Abadi, and Roger Needham, "A
Logic of Authentication." ACM Transactions on
Computer Systems v8 n1, February 1990.
- Sirer et al., Kimera:
A Java System Architecture.
Resource Management
(Mar. 29 - Apr. 5) |
Question: How do we allocate resources among competing
needs, whether those be CPU cycles to run agents for end users or
geographically distributed Internet services? How do we find idle
resources if they exist?
- Mar. 29: Carl A. Waldspurger and William E. Weihl. "Stride
Scheduling: Deterministic Proportional-Share Resource
Management." Technical Memo MIT/LCS/TM-528, June
1995.

- Mar. 31: Michael. Stonebraker, Paul Aoki, Witold Litwin,
Avi Pfeffer, Adam Sah, Jeff Sidell, Carl Staelin and
Andrew Yu, "Mariposa:
A Wide-Area Distributed Database System." VLDB
Journal 5 1, January 9, 1996.

- Apr. 2: Gaurav Banga, Peter Druschel, and Jeff Mogul, "Resource
containers: A New Facility for Resource Management in
Server Systems." Proceedings of Operating
Systems Design and Implementation, February 1999.

- Apr. 5: Grzegorz Czajkowski and Thorsten von Eicken. "JRes:
A Resource Accounting Interface for Java."
Proceedings of 1998 ACM OOPSLA Conference. October 1998.

For more depth:
- Michael B. Jones, Daniela Rosu, Marcel-Catalin Rosu, "CPU Reservations and Time Constraints: Efficient, Predictable Scheduling of Independent Activities." Proceedings of the 16th ACM Symposium on Operating Systems Principles, October 1997.
- Rich Wolski, Dynamically
Forecasting Network Performance Using the Network Weather
Service. Proceedings of the 6th High-Performance
Distributed Computing Conference, August, 1997.
- B. Clifford Neuman and Santosh Rao, "The Prospero
Resource Manager: A Scalable Framework for Processor
Allocation in Distributed Systems." Concurrency:
Practice and Experience, 6(4): 339-355, June 1994.
- Fran Berman, Richard Wolski, Silvia Figueira, Jennifer
Schopf, Gary Shao, "Application-Level
Scheduling on Distributed Heterogeneous Networks."
Proceedings of Supercomputing, November 1996.
- Michael B. Jones, Daniela Rosu, Marcel-Catalin Rosu,
"CPU Reservations and Time Constraints: Efficient,
Predictable Scheduling of Independent Activities."
Proceedings of the 16th ACM Symposium on Operating
Systems Principles. Saint-Malo, France, pages 198-211,
October, 1997.
Infrastructure and Applications
(Apr. 7 - Apr. 16) |
Question: How do we make sure what we build works before it
gets deployed? How do we deploy what we build?
- Apr. 7: L. Brakmo and L. Peterson. "Experiences
with Network Simulation." Proceedings of
SIGMETRICS, June 1996.

- Apr 7: Brian Noble, M. Satyanarayanan, Giao Nguyen, and
Randy Katz. "Trace-Based
Mobile Network Emulation." Proceedings of
SIGCOMM, September 1997.

- Apr. 9: David Wetherall, John Guttag, and David
Tennenhouse, "ANTS:
A Toolkit for Building and Dynamically Deploying Network
Protocols." IEEE OPENARCH, April 1998.

- Apr. 9: Sharon Brunett, Karl Czajkowski, Steven
Fitzgerald, Ian Foster, Andrew Johnson, Carl Kesselman,
Jason Leigh, Steven Tuecke, "Application
Experiences with the Globus Toolkit."
7th IEEE Symp. on High Performance Distributed Computing,
July 1998.

- Apr. 12: Stefan Savage, Andy Collins, Eric Hoffman, John Snell, and Thomas Anderson, "The End-to-End Effects of Internet Path Selection." Technical Report, February 1999.

- Apr. 12: Stefan Savage, Thomas Anderson, Amit Aggarwal, David Becker, Neal Cardwell, Andy Collins, Eric Hoffman, John Snell, Amin Vahdat, Geoff Voelker, and John Zahorjan, "Detour: A Case for Intelligent Internet Routing and Transport." IEEE Micro, February 1999.

- Apr. 14: Srinivasan Seshan, Mark Stemm, and Randy H.
Katz, "SPAND:
Shared Passive Network Performance Discovery."
Proc. 1st USENIX Symposium on Internet Technologies and
Systems, December 1997.

- Apr. 16: Gaurav Banga and Jeff Mogul, "Scalable
Kernel Performance for Internet Servers Under Realistic
Loads." Proc. USENIX 1998 Technical Conference,
June 1998.

For more depth:
Wrapup and Final Projects
(Apr. 19 - 23) |