Gigabit TCP with Trapeze
Recently the Trapeze project has received some press attention
for our demonstration of TCP/IP bandwidths above a gigabit per
second. Please see the Trapeze site for technical
information about the experiments and the results.
Some of the press coverage about the experiment is misleading.
We would like to emphasize, clarify, and expand on key points
touched on in the original Duke press release:
What we did was to combine several software techniques, some
of which are not novel, to produce what we believe is the first
demonstration on public record of TCP/IP protocols running faster
than a gigabit per second. Our contribution included drivers and
FreeBSD operating system kernel enhancements for zero-copy sockets
(written by Andrew Gallatin) and the Trapeze/Myrinet firmware
(written by Ken Yocum).
We did not improve the TCP/IP protocol software and we did
not build or modify high-speed LAN hardware. These are
significant technical achievements that we claim no credit for.
The credit for the LAN is due to Myricom,
and the credit for
gigabit-capable TCP/IP software is due to others, e.g., Van
Jacobson, Dave Borman, and many others who have contributed to
TCP/IP over the years, as well as to its implementation in
FreeBSD. Moreover, our result was made possible by advancing
endstation hardware, most notably 64-bit PCI.
- Typical TCP socket implementations running over typical
gigabit LANs (e.g., a Gigabit Ethernet using the standard
1500-byte MTU) deliver about half a gigabit per-second. We set
up a DEC/Compaq Alpha 21264 (XP1000) workstation with prototype
LANai-7 Myrinet cards donated by Myricom, installed our firmware,
booted an enhanced FreeBSD kernel, and measured TCP bandwidths
above 1.1 gigabits per second -- twice as fast as typical
workstations and operating systems connected to current
- It is possible to transmit data with TCP faster than half a
gigabit per-second without our software. Certainly there are
other LANs on the market capable of TCP bandwidths higher than
half a gigabit per second. In fact, our published results show
that Gigabit Ethernet products from Alteon
(using Jumbo Frames)
are capable of end-to-end TCP bandwidths of about 970 Mb/s on the
XP1000 even without our enhancements, assuming a driver with
support for checksum offloading. Alteon's web site currently reports
several experiments in which a host services several simultaneous
TCP connections which together approach one gigabit per-second.
- The previous end-to-end TCP bandwidth record listed at netperf.org was 750 Mb/s.
However, it is unlikely that the netperf.org site is
regularly updated with new results as LAN and workstation
hardware continue to advance. It is quite possible that others
have approached or even exceeded the gigabit milestone without
promoting the result. The previous highest bandwidth recorded on
Myricom's web site is 687 Mb/s, set by our group last year with an earlier
generation of Myricom's LAN and DEC Alpha workstations.
- The work is one aspect of a larger project funded by the National
Science Foundation and equipment grants from Intel Corporation and
Last modified: Sun Apr 30 12:59:35 EDT 2000