Duke Trapeze/Myrinet TCP/IP Performance
If you are looking for information about the "gigabit TCP" news reports, start start here.
IP Performance Using Duke's Trapeze
API
We are currently using our Trapeze API and firmware in a normal IP device
driver on Digital UNIX and FreeBSD. The results reported here were obtained
using an unmodified copy of Netperf v2.1pl3.
TCP Bandwidth
| Machine |
Lanai |
Window Size |
OS |
Zero Copy |
Result |
| PowerEdge 4400@733Mhz |
9.0 |
524288 |
FreeBSD 4.0-RELEASE |
Yes |
1765 Mbits/sec |
| XP1000@500Mhz |
7.0 |
524288 |
FreeBSD 4.0-current |
Yes |
1147 Mbits/sec |
| XP1000@500Mhz |
7.0 |
524288 |
FreeBSD 4.0-current |
No |
970 Mbits/sec |
| XP1000@500Mhz |
5.22 |
262144 |
FreeBSD 4.0-current |
No |
952 Mbits/sec |
| PWS500au@500Mhz |
5.22 |
1048576 |
FreeBSD 4.0-current |
Yes |
880 Mbits/sec |
| PII@450Mhz |
4.1 |
524288 |
FreeBSD 4.0-current |
Yes |
808 Mbits/sec |
| PII@450Mhz |
4.1 |
524288 |
FreeBSD 4.0-current |
No |
666 Mbits/sec |
TCP/UDP Latency
| Machine |
Lanai |
Protocol |
OS |
Req/Resp |
Trans Rate per sec |
| XP1000@500Mhz |
7.0 |
UDP, checksums off |
FreeBSD 4.0-current |
1 byte / 1 byte |
10534 |
| PII@450Mhz |
4.1 |
UDP, checksums off |
FreeBSD 4.0-current |
1 byte / 1 byte |
10341 |
| PII@450Mhz |
4.1 |
UDP, checksums on |
FreeBSD 4.0-current |
1 byte / 1 byte |
9896 |
| PII@450Mhz |
4.1 |
TCP |
FreeBSD 4.0-current |
1 byte / 1 byte |
8525 |
The zero-copy modifications we've made to the network stack have
been to avoid copyout() at the socket layer on receives via a locally
implemented page-renaming scheme which avoids any semantic changes to
the socket interface. On transmits, we also map the user's page into
kernel memory (marking it copy on write) to avoid copyin(). On the
Lanai 5 and greater, we take advantage of hardware checksum offloading
on the receiver.
Older results, for Lanai4 on Digital UNIX as well as FreeBSD can now be found here.
This work is one aspect of the
cluster computing project.
Andrew Gallatin
Last
modified: Thu Jun 15 10:43:15 EDT 2000