[RndTbl] NIC performance with iperf
athompso at athompso.net
Fri Apr 16 15:36:36 CDT 2010
FYI. Windows networking sucks, across all versions of Windows. (At
least, at Gigabit speeds, it does.)
Between my Windows 7 PC
(3.06Ghz HT Pentium 4, Intel PRO 1000/MT Desktop Adapter [82540EM])
and Gentoo Linux server
(2 x 2.2GHz Xeon [P4 core], PCI-X [64-bit,66Mhz] Intel PRO 1000/MT
Dual-Port Server Adapter [82546EB])
Using "iperf -s" and "iperf -c 192.168.x.x", I get - consistently -
between 238Mbps and 250Mbps **in both directions**.
Interesting to note is that the Windows 7 PC is CPU-limited on both
sending and receiving, despite having all the offload and acceleration
features turned on. Simultaneously, the Gentoo box sees a spike on one
CPU... to 1%.
In UDP traffic, however, I see a startling asymmetry: PC-to-Linux can only
send UDP at ~50Mbps, Linux-to-PC sends UDP at ~475Mbps. WTF?
I also tried the Parallel option, which runs multiple streams in parallel.
Results are consistent, aggregate bandwidth is still only about 260Mbps.
Two streams pegs both logical (HT) CPUs on the client at ~95% utilization.
I tried enabling and disabling both the Windows Firewall and my AV
software, which made essentially no difference at all.
Then I also tried running the same set of tests from a Windows 2003 R2
(2 x 2.13GHz Athlon MP, PCI-X [64-bit,66Mhz] Broadcom NetXtreme
N.B. this is the same model of motherboard Kevin is using in his Linux
and observed slightly better throughput to the Linux server: about
380Mbps. Still better was Linux-to-W2K3R2, at about 440Mbps. UDP
performance was similar to Kevin's results, at 103Mbps W2K3-to-Linux &
CPU utilization on the W2K3R2 box exhibited similar properties as the Win7
client, although none of the tests actually managed to max out either CPU,
instead peaking at about 70% utilization.
Just for kicks, I ran a test between a RHEL5 VM (under VMWare ESXi 4.0U1,
VMXNET adapter) and my Linux server... 920+ Mbps every time! Of course,
that number may be suspect since the UDP test reports speeds of 45.3Gbps
Conclusion: the culprit is the Windows networking stack. Note that W2K3R2
is supposed to support zero-copy TCP offload, theoretically allowing
network communications at wirespeed with extremely low latency. So much
for that idea...
<athompso at athompso.net>
More information about the Roundtable