[RndTbl] NIC performance with iperf

Adam Thompson athompso at athompso.net
Fri Apr 16 15:36:36 CDT 2010

FYI.  Windows networking sucks, across all versions of Windows.  (At 
least, at Gigabit speeds, it does.)

Between my Windows 7 PC
	(3.06Ghz HT Pentium 4, Intel PRO 1000/MT Desktop Adapter [82540EM])
and Gentoo Linux server
	(2 x 2.2GHz Xeon [P4 core], PCI-X [64-bit,66Mhz] Intel PRO 1000/MT 
Dual-Port Server Adapter [82546EB])

Using "iperf -s" and "iperf -c 192.168.x.x", I get - consistently - 
between 238Mbps and 250Mbps **in both directions**.

Interesting to note is that the Windows 7 PC is CPU-limited on both 
sending and receiving, despite having all the offload and acceleration 
features turned on.  Simultaneously, the Gentoo box sees a spike on one 
CPU... to 1%.

In UDP traffic, however, I see a startling asymmetry: PC-to-Linux can only 
send UDP at ~50Mbps, Linux-to-PC sends UDP at ~475Mbps.  WTF?

I also tried the Parallel option, which runs multiple streams in parallel. 
Results are consistent, aggregate bandwidth is still only about 260Mbps. 
Two streams pegs both logical (HT) CPUs on the client at ~95% utilization.

I tried enabling and disabling both the Windows Firewall and my AV 
software, which made essentially no difference at all.

Then I also tried running the same set of tests from a Windows 2003 R2 
	(2 x 2.13GHz Athlon MP, PCI-X [64-bit,66Mhz] Broadcom NetXtreme 
	N.B. this is the same model of motherboard Kevin is using in his Linux 
and observed slightly better throughput to the Linux server: about 
380Mbps.  Still better was Linux-to-W2K3R2, at about 440Mbps.  UDP 
performance was similar to Kevin's results, at 103Mbps W2K3-to-Linux & 
950Mbps Linux-to-W2K3.

CPU utilization on the W2K3R2 box exhibited similar properties as the Win7 
client, although none of the tests actually managed to max out either CPU, 
instead peaking at about 70% utilization.

Just for kicks, I ran a test between a RHEL5 VM (under VMWare ESXi 4.0U1, 
VMXNET adapter) and my Linux server... 920+ Mbps every time!  Of course, 
that number may be suspect since the UDP test reports speeds of 45.3Gbps 

Conclusion: the culprit is the Windows networking stack.  Note that W2K3R2 
is supposed to support zero-copy TCP offload, theoretically allowing 
network communications at wirespeed with extremely low latency.  So much 
for that idea...

-Adam Thompson
 <athompso at athompso.net>
 (204) 291-7950

More information about the Roundtable mailing list