Expected file transfer speeds over Gigabit?

[ohaya]

Hi,

If anyone has done any timed testing or benchmarking for file transfers
between two machines running Windows 2000 Server with GigE adapters, I
was wondering if you could post your results?

I'm doing some testing between two servers with Xeon processors and
Intel NICs, with two large compressed files, and am getting results from
about 11 Mbytes/sec to 26 Mbytes/sec, depending upon what software I use
for testing the transfers. The test files are about 700 Mbytes and
about 4 Gbytes. The systems are booted from RAID'ed SAN volumes, and
I've measured the 'drive' performance using HDTach and Sandra, and got
about 90 Mbytes/sec average speed, so 'disk' speed should be a small
factor.

I've been testing with FTP and with Robocopy (from the Windows Resource
Kit). I've done some tweaking of Registry, with little or no
improvement, but the results above were on clean, untweaked installs.

I've also done pure network testing using netperf, etc., and got about
300 Mbits/sec using default parameters, and 900+ Mbits/sec using
non-default parameters, but now I'm trying to see how 'real' transfer
performance would be.

For FTP, on the server side, I'm using the MS FTP server, and for the
client side, I've used both the MS command line FTP client and the NCFTP
client.

The 26 Mbytes/sec results were using Robocopy, and in all cases, CPU
Utilization is less than 10% using both Perfmon and Task Manager, so CPU
load doesn't appear to be an issue.

If anyone has done any similar testing, I'd be very interested in
hearing about your results, as I'm trying to 'sanity check' what I'm
finding, i.e., that, at best, it's only possible to get about 1/4 - 1/3
of the true network bandwidth.

Thanks,
Jim

 

[Roland Hall]

in message : If anyone has done any timed testing or benchmarking for file transfers
: between two machines running Windows 2000 Server with GigE adapters, I
: was wondering if you could post your results?
:
: I'm doing some testing between two servers with Xeon processors and
: Intel NICs, with two large compressed files, and am getting results from
: about 11 Mbytes/sec to 26 Mbytes/sec, depending upon what software I use
: for testing the transfers. The test files are about 700 Mbytes and
: about 4 Gbytes. The systems are booted from RAID'ed SAN volumes, and
: I've measured the 'drive' performance using HDTach and Sandra, and got
: about 90 Mbytes/sec average speed, so 'disk' speed should be a small
: factor.
:
: I've been testing with FTP and with Robocopy (from the Windows Resource
: Kit). I've done some tweaking of Registry, with little or no
: improvement, but the results above were on clean, untweaked installs.
:
: I've also done pure network testing using netperf, etc., and got about
: 300 Mbits/sec using default parameters, and 900+ Mbits/sec using
: non-default parameters, but now I'm trying to see how 'real' transfer
: performance would be.
:
: For FTP, on the server side, I'm using the MS FTP server, and for the
: client side, I've used both the MS command line FTP client and the NCFTP
: client.
:
: The 26 Mbytes/sec results were using Robocopy, and in all cases, CPU
: Utilization is less than 10% using both Perfmon and Task Manager, so CPU
: load doesn't appear to be an issue.
:
: If anyone has done any similar testing, I'd be very interested in
: hearing about your results, as I'm trying to 'sanity check' what I'm
: finding, i.e., that, at best, it's only possible to get about 1/4 - 1/3
: of the true network bandwidth.

Now tell us about the infrastructure of your interconnectivity equipment
since that might be where your bottleneck is and it is a controlled
environment or are you in the middle of production?

I'll have additional follow-up questions after your reply.

--
Roland Hall
/* This information is distributed in the hope that it will be useful, but
without any warranty; without even the implied warranty of merchantability
or fitness for a particular purpose. */
Online Support for IT Professionals -
http://support.microsoft.com/servicedesks/technet/default.asp?fr=0&sd=tech
How-to: Windows 2000 DNS:
http://support.microsoft.com/default.aspx?scid=kb;EN-US;308201
FAQ W2K/2K3 DNS:
http://support.microsoft.com/default.aspx?scid=kb;EN-US;291382

 

[ohaya]

Now tell us about the infrastructure of your interconnectivity equipment
since that might be where your bottleneck is and it is a controlled
environment or are you in the middle of production?

I'll have additional follow-up questions after your reply.

Hi Roland,

Both machines are essentially identical, with Xeon 3.0 GHz CPUs, 16 GB
memory and Intel 1000 NICs with latest drivers. Right now, PAE is not
enabled in the boot.ini file because we had some problems with
compatibility with some apps earlier, when PAE was enabled. But, thie
current testing is being done on clean installs, with virtually nothing
running except whatever software I use for this testing.

Both machines are booted from RAID5 LUNs on a SAN (EMC Clariion). FC
HBAs are Qlogic with PCI-X interfaces to the system.

I've previously tested "drive performance" on each machines using
HDTach3, and got ~130 Mbytes/sec burst speed and ~110 MBytes/sec average
speed, so I think that raw drive performance shouldn't be a major
factor.

I can connect the NICs on each machine together either via a fiber
cross-over cable or via a GigE switch (don't know the actual
manufacturer, as I think it's a re-branded switch) for testing. Based
on my network-only testing (e.g., Netperf), I saw about a 3-5 Mbits/sec
difference going through the cross-over vs. the GigE switch.

The two machines, network connections (e.g., cross-over or via GigE
switch are isolated, i.e., the entire configuration is essentially
standalone.

Jim

 

[chamila]

*


Hi Roland,

Both machines are essentially identical, with Xeon 3.0 GHz CPUs, 1
GB
memory and Intel 1000 NICs with latest drivers. Right now, PAE i
not
enabled in the boot.ini file because we had some problems with
compatibility with some apps earlier, when PAE was enabled. But
thie
current testing is being done on clean installs, with virtuall
nothing
running except whatever software I use for this testing.

Both machines are booted from RAID5 LUNs on a SAN (EMC Clariion).
FC
HBAs are Qlogic with PCI-X interfaces to the system.

I've previously tested "drive performance" on each machines using
HDTach3, and got ~130 Mbytes/sec burst speed and ~110 MBytes/se
average
speed, so I think that raw drive performance shouldn't be a major
factor.

I can connect the NICs on each machine together either via a fiber
cross-over cable or via a GigE switch (don't know the actual
manufacturer, as I think it's a re-branded switch) for testing.
Based
on my network-only testing (e.g., Netperf), I saw about a 3-
Mbits/sec
difference going through the cross-over vs. the GigE switch.

The two machines, network connections (e.g., cross-over or via GigE
switch are isolated, i.e., the entire configuration is essentially
standalone.

Jim *

DO YOU KNOW HOW TO USE NET USE COMMAND IN WINDOWS 2003 SERVER? They d
not show a freaking example like if I want to add JOE in to activ
directory using net use command what is the syntax


-
chamil

 

[Phillip Windell]

DO YOU KNOW HOW TO USE NET USE COMMAND IN WINDOWS 2003 SERVER? They do
not show a freaking example like if I want to add JOE in to active
directory using net use command what is the syntax?

Where'd this come from??

You *Don't*. That is not what Net Use is for. Use the regular GUI, go into
"Active Directory Users and Computers" and add them.

 

[Roland Hall]

in message
: > Now tell us about the infrastructure of your interconnectivity equipment
: > since that might be where your bottleneck is and it is a controlled
: > environment or are you in the middle of production?
: >
: > I'll have additional follow-up questions after your reply.
:
:
: Hi Roland,
:
: Both machines are essentially identical, with Xeon 3.0 GHz CPUs, 16 GB
: memory and Intel 1000 NICs with latest drivers. Right now, PAE is not
: enabled in the boot.ini file because we had some problems with
: compatibility with some apps earlier, when PAE was enabled. But, thie
: current testing is being done on clean installs, with virtually nothing
: running except whatever software I use for this testing.
:
: Both machines are booted from RAID5 LUNs on a SAN (EMC Clariion). FC
: HBAs are Qlogic with PCI-X interfaces to the system.
:
: I've previously tested "drive performance" on each machines using
: HDTach3, and got ~130 Mbytes/sec burst speed and ~110 MBytes/sec average
: speed, so I think that raw drive performance shouldn't be a major
: factor.
:
: I can connect the NICs on each machine together either via a fiber
: cross-over cable or via a GigE switch (don't know the actual
: manufacturer, as I think it's a re-branded switch) for testing. Based
: on my network-only testing (e.g., Netperf), I saw about a 3-5 Mbits/sec
: difference going through the cross-over vs. the GigE switch.
:
: The two machines, network connections (e.g., cross-over or via GigE
: switch are isolated, i.e., the entire configuration is essentially
: standalone.

Which one?
http://downloadfinder.intel.com/scr...sp?agr=Y&Inst=Yes&ProductID=1098&DwnldID=4233
Which driver version. The latest is not clear on this end.
The 3-5mbps difference is up or down x-over vs switch?

--
Roland Hall
/* This information is distributed in the hope that it will be useful, but
without any warranty; without even the implied warranty of merchantability
or fitness for a particular purpose. */
Online Support for IT Professionals -
http://support.microsoft.com/servicedesks/technet/default.asp?fr=0&sd=tech
How-to: Windows 2000 DNS:
http://support.microsoft.com/default.aspx?scid=kb;EN-US;308201
FAQ W2K/2K3 DNS:
http://support.microsoft.com/default.aspx?scid=kb;EN-US;291382

 

[Buddy Ackerman]

I have a similar question. I have three servers using Tyan Dual Athlon MP
processor motherboards (with two processors each) that have gigabit Ethernet
ports on the motherboard. They are Intel(R) PRO/1000 MT Server Adapter
interfaces (and I keep up with the driver updates through Windows Update so
I assume that I have the latest drivers). I am using a 5 port Linksys
switch (don't know the exact model but it was pretty cheap, about $100).
I'm only getting about 10MB/sec transfer speed on file transfers between the
machines (based on a file manager copy operation). I used the Network
Monitor to get the transfer rate information. I was expecting something
along the lines of 50MB/sec transfer speed. Why is it so slow? Is it the
cheap switch or are there some network setting that I can change to improve
the performance?


--Buddy

 

[Bob I]

The switch is the problem. For "gigabit" speed, everything in the path
has to be gigabit.

 

[ohaya]

Buddy,

As Bob mentioned in his post, you have to use a GigE switch. If you are
using a Linksys GigE switch, and are still getting ~10 MBytes/sec:

I've been told that doing a file copy (e.g., file manager, copy-paste,
etc.) is always going to be relatively slow because it uses the SMB
protocol.

The suggestion that I got was to use a utility called "Robocopy", which
is part of the Win2K Resource Kit.

I've been testing file transfers with that, and, depending upon the
machines, disk drives, etc., I've been able to get ~57 Mbytes/sec file
transfers between machines connected with a GigE switch and both using
Intel 1000 NICs.

One nice thing about Robocopy is that it reports statistics on the
file(s) transferred, e.g., Bytes/sec, etc.

The other suggestion that I got was to try using WebDAV instead of
copy-paste, but for various reasons, I couldn't use IIS, so that isn't
an option for me at this time.

I had spent weeks on this, doing all kinds of tests, tweaking the
Registry, etc., without much improvement, so I hope that this helps.

Jim

 

[Leythos]

I had spent weeks on this, doing all kinds of tests, tweaking the
Registry, etc., without much improvement, so I hope that this helps.

As I understand it you should test by using FTP, not a file copy. I've
used RoboCopy, but not as a speed test. I will check and see if RC is as
fast as FTP.

 

[ohaya]

As I understand it you should test by using FTP, not a file copy. I've
used RoboCopy, but not as a speed test. I will check and see if RC is as
fast as FTP.

Leythos,

Among the things that were suggested and which I tried were FTP and
NCFTP, and their performance, at least in my testing was way slower than
Robocopy or copying-pasting in Windows. On the same two machines, and
on a couple of other pairs of machines (different brands, all with
GigEs), the best that I got was ~18 Mbytes/sec.

Jim

 

[Leythos]

Leythos,

Among the things that were suggested and which I tried were FTP and
NCFTP, and their performance, at least in my testing was way slower than
Robocopy or copying-pasting in Windows. On the same two machines, and
on a couple of other pairs of machines (different brands, all with
GigEs), the best that I got was ~18 Mbytes/sec.

I have a mix of GIG cards in workstations and servers using a couple D-
Link gig switches. I run FileZilla server on a Dual Xeon Server as the
FTP service and use FileZilla as the client on a workstation. I tested
moving a 700MB file from workstation to server using FTP and got a
consistent 168KBytes/sec.

I'll try RC later this weekend and let you know what I see...