Duke Trapeze/Myrinet TCP/IP Performance

If you are looking for information about the "gigabit TCP" news reports, start start here.

IP Performance Using Duke's Trapeze API

We are currently using our Trapeze API and firmware in a normal IP device driver on Digital UNIX and FreeBSD. The results reported here were obtained using an unmodified copy of Netperf v2.1pl3.

TCP Bandwidth

 Machine Lanai Window Size OS Zero Copy Result
PowerEdge 4400@733Mhz 9.0 524288 FreeBSD 4.0-RELEASE Yes 1765 Mbits/sec
XP1000@500Mhz 7.0 524288 FreeBSD 4.0-current Yes 1147 Mbits/sec
XP1000@500Mhz 7.0 524288 FreeBSD 4.0-current No 970 Mbits/sec
XP1000@500Mhz 5.22 262144 FreeBSD 4.0-current No 952 Mbits/sec
PWS500au@500Mhz 5.22 1048576 FreeBSD 4.0-current Yes 880 Mbits/sec
PII@450Mhz 4.1 524288 FreeBSD 4.0-current Yes 808 Mbits/sec
PII@450Mhz 4.1 524288 FreeBSD 4.0-current No 666 Mbits/sec

TCP/UDP Latency

 Machine Lanai Protocol OS Req/Resp Trans Rate per sec
XP1000@500Mhz 7.0 UDP, checksums off FreeBSD 4.0-current 1 byte / 1 byte 10534
PII@450Mhz 4.1 UDP, checksums off FreeBSD 4.0-current 1 byte / 1 byte 10341
PII@450Mhz 4.1 UDP, checksums on FreeBSD 4.0-current 1 byte / 1 byte 9896
PII@450Mhz 4.1 TCP FreeBSD 4.0-current 1 byte / 1 byte 8525

The zero-copy modifications we've made to the network stack have been to avoid copyout() at the socket layer on receives via a locally implemented page-renaming scheme which avoids any semantic changes to the socket interface. On transmits, we also map the user's page into kernel memory (marking it copy on write) to avoid copyin(). On the Lanai 5 and greater, we take advantage of hardware checksum offloading on the receiver.

Older results, for Lanai4 on Digital UNIX as well as FreeBSD can now be found here.


This work is one aspect of the cluster computing project.
Andrew Gallatin


Last modified: Thu Jun 15 10:43:15 EDT 2000