PC Review


Reply
Thread Tools Rate Thread

how come lots of small files transfer slower than 1 big one?

 
 
James
Guest
Posts: n/a
 
      31st Jan 2004
how come my transfers are so slow when copying files from my xp machine to
my win2k machine on my 100mbps switched lan?
large file = 100MB .mdb
small files = total 7MB's of 400 text files

it takes 12 mins to transfer the 7 MB's and only 30 seconds to transfer the
large file.


--
(E-Mail Removed)


 
Reply With Quote
 
 
 
 
Bob Willard
Guest
Posts: n/a
 
      31st Jan 2004
James wrote:
> how come my transfers are so slow when copying files from my xp machine to
> my win2k machine on my 100mbps switched lan?
> large file = 100MB .mdb
> small files = total 7MB's of 400 text files
>
> it takes 12 mins to transfer the 7 MB's and only 30 seconds to transfer the
> large file.
>
>


And how come digging a ditch with one backhoe is faster than using
ten men with shovels? Or using a thousand men with teaspoons?

Files are transferred one at a time. For each file, the source PC must
look up the file's metadata (to translate the file name to the file
location, check security params, etc.). Worse still, is what the target
PC must do for each file: it must find some unused space on the HD, then
allocate some of that space, then create the metadata for the new file,
then ask the source PC for the data, then copy the data to the allocated
space, then update the metadata (when all data has been copied), then
notify the source PC that the copy is done.

It takes a lot of work to copy a file across a net, whether the file is
tiny or huge. For a huge file, copying data take most of the time; for a
tiny file, copying data takes far less time than all of the other overhead
work.
--
Cheers, Bob

 
Reply With Quote
 
James
Guest
Posts: n/a
 
      2nd Feb 2004
but lots of files tranfer fast when i use another workstation to the same
server.



"Bob Willard" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> James wrote:
> > how come my transfers are so slow when copying files from my xp machine

to
> > my win2k machine on my 100mbps switched lan?
> > large file = 100MB .mdb
> > small files = total 7MB's of 400 text files
> >
> > it takes 12 mins to transfer the 7 MB's and only 30 seconds to transfer

the
> > large file.
> >
> >

>
> And how come digging a ditch with one backhoe is faster than using
> ten men with shovels? Or using a thousand men with teaspoons?
>
> Files are transferred one at a time. For each file, the source PC must
> look up the file's metadata (to translate the file name to the file
> location, check security params, etc.). Worse still, is what the target
> PC must do for each file: it must find some unused space on the HD, then
> allocate some of that space, then create the metadata for the new file,
> then ask the source PC for the data, then copy the data to the allocated
> space, then update the metadata (when all data has been copied), then
> notify the source PC that the copy is done.
>
> It takes a lot of work to copy a file across a net, whether the file is
> tiny or huge. For a huge file, copying data take most of the time; for a
> tiny file, copying data takes far less time than all of the other overhead
> work.
> --
> Cheers, Bob
>



 
Reply With Quote
 
Bob Willard
Guest
Posts: n/a
 
      2nd Feb 2004
James wrote:

> but lots of files tranfer fast when i use another workstation to the same
> server.
>
>
>
> "Bob Willard" <(E-Mail Removed)> wrote in message
> news:(E-Mail Removed)...
>
>>James wrote:
>>
>>>how come my transfers are so slow when copying files from my xp machine

>
> to
>
>>>my win2k machine on my 100mbps switched lan?
>>>large file = 100MB .mdb
>>>small files = total 7MB's of 400 text files
>>>
>>>it takes 12 mins to transfer the 7 MB's and only 30 seconds to transfer

>
> the
>
>>>large file.
>>>
>>>

>>
>>And how come digging a ditch with one backhoe is faster than using
>>ten men with shovels? Or using a thousand men with teaspoons?
>>
>>Files are transferred one at a time. For each file, the source PC must
>>look up the file's metadata (to translate the file name to the file
>>location, check security params, etc.). Worse still, is what the target
>>PC must do for each file: it must find some unused space on the HD, then
>>allocate some of that space, then create the metadata for the new file,
>>then ask the source PC for the data, then copy the data to the allocated
>>space, then update the metadata (when all data has been copied), then
>>notify the source PC that the copy is done.
>>
>>It takes a lot of work to copy a file across a net, whether the file is
>>tiny or huge. For a huge file, copying data take most of the time; for a
>>tiny file, copying data takes far less time than all of the other overhead
>>work.
>>--
>>Cheers, Bob
>>

>
>
>


From your question, I assumed that you were interested in why bunches
of files take longer than a single file to transfer; that is the issue
I addressed. If you want to understand why one PC is faster than another,
then you should conduct some measurements in which only one variable at
a time is changed -- and "the workstation" is not one variable, it is
probably many.
--
Cheers, Bob

 
Reply With Quote
 
James
Guest
Posts: n/a
 
      2nd Feb 2004
what things could be differn't. both are clean installs of xp sp1


"Bob Willard" <(E-Mail Removed)> wrote in message
news:uEr$(E-Mail Removed)...
> James wrote:
>
> > but lots of files tranfer fast when i use another workstation to the

same
> > server.
> >
> >
> >
> > "Bob Willard" <(E-Mail Removed)> wrote in message
> > news:(E-Mail Removed)...
> >
> >>James wrote:
> >>
> >>>how come my transfers are so slow when copying files from my xp machine

> >
> > to
> >
> >>>my win2k machine on my 100mbps switched lan?
> >>>large file = 100MB .mdb
> >>>small files = total 7MB's of 400 text files
> >>>
> >>>it takes 12 mins to transfer the 7 MB's and only 30 seconds to transfer

> >
> > the
> >
> >>>large file.
> >>>
> >>>
> >>
> >>And how come digging a ditch with one backhoe is faster than using
> >>ten men with shovels? Or using a thousand men with teaspoons?
> >>
> >>Files are transferred one at a time. For each file, the source PC must
> >>look up the file's metadata (to translate the file name to the file
> >>location, check security params, etc.). Worse still, is what the target
> >>PC must do for each file: it must find some unused space on the HD, then
> >>allocate some of that space, then create the metadata for the new file,
> >>then ask the source PC for the data, then copy the data to the allocated
> >>space, then update the metadata (when all data has been copied), then
> >>notify the source PC that the copy is done.
> >>
> >>It takes a lot of work to copy a file across a net, whether the file is
> >>tiny or huge. For a huge file, copying data take most of the time; for

a
> >>tiny file, copying data takes far less time than all of the other

overhead
> >>work.
> >>--
> >>Cheers, Bob
> >>

> >
> >
> >

>
> From your question, I assumed that you were interested in why bunches
> of files take longer than a single file to transfer; that is the issue
> I addressed. If you want to understand why one PC is faster than another,
> then you should conduct some measurements in which only one variable at
> a time is changed -- and "the workstation" is not one variable, it is
> probably many.
> --
> Cheers, Bob
>



 
Reply With Quote
 
Bob Willard
Guest
Posts: n/a
 
      2nd Feb 2004
James wrote:
> what things could be differn't. both are clean installs of xp sp1
>
>
> "Bob Willard" <(E-Mail Removed)> wrote in message
> news:uEr$(E-Mail Removed)...
>
>>James wrote:
>>
>>
>>>but lots of files tranfer fast when i use another workstation to the

>
> same
>
>>>server.
>>>
>>>
>>>
>>>"Bob Willard" <(E-Mail Removed)> wrote in message
>>>news:(E-Mail Removed)...
>>>
>>>
>>>>James wrote:
>>>>
>>>>
>>>>>how come my transfers are so slow when copying files from my xp machine
>>>
>>>to
>>>
>>>
>>>>>my win2k machine on my 100mbps switched lan?
>>>>>large file = 100MB .mdb
>>>>>small files = total 7MB's of 400 text files
>>>>>
>>>>>it takes 12 mins to transfer the 7 MB's and only 30 seconds to transfer
>>>
>>>the
>>>
>>>
>>>>>large file.
>>>>>
>>>>>
>>>>
>>>>And how come digging a ditch with one backhoe is faster than using
>>>>ten men with shovels? Or using a thousand men with teaspoons?
>>>>
>>>>Files are transferred one at a time. For each file, the source PC must
>>>>look up the file's metadata (to translate the file name to the file
>>>>location, check security params, etc.). Worse still, is what the target
>>>>PC must do for each file: it must find some unused space on the HD, then
>>>>allocate some of that space, then create the metadata for the new file,
>>>>then ask the source PC for the data, then copy the data to the allocated
>>>>space, then update the metadata (when all data has been copied), then
>>>>notify the source PC that the copy is done.
>>>>
>>>>It takes a lot of work to copy a file across a net, whether the file is
>>>>tiny or huge. For a huge file, copying data take most of the time; for

>
> a
>
>>>>tiny file, copying data takes far less time than all of the other

>
> overhead
>
>>>>work.
>>>>--
>>>>Cheers, Bob
>>>>
>>>
>>>
>>>

>> From your question, I assumed that you were interested in why bunches
>>of files take longer than a single file to transfer; that is the issue
>>I addressed. If you want to understand why one PC is faster than another,
>>then you should conduct some measurements in which only one variable at
>>a time is changed -- and "the workstation" is not one variable, it is
>>probably many.
>>--
>>Cheers, Bob
>>

>
>
>


NIC speed, RWIN, protocols used, DNS settings, HD speed, CPU frequency,
IDE speed, RAM speed and quantity, pagefile min/max params and HD(s),
XP PRO v. HE, NTFS v. FAT32, user credentials, other traffic on the LAN,
HD RPM, HD fragmentation, CPU type, cache params, MB type/revision, BIOS
type, version, and settings, NIC vendor/model, NIC card v. chip, concurrent
apps running, domain v. workgroup, etc.

And, maybe, lots of other stuff. It takes care and effort to compare two
things that are *exactly* alike except for one variable.
--
Cheers, Bob

 
Reply With Quote
 
James
Guest
Posts: n/a
 
      3rd Feb 2004
both computers are pretty new i think they should be able to support
transfering files just fine with their hardware. im guessing either a NIC
settting is the case here or a bad NIC. this NIC im using is the onboard one
that came with this msi 865PE neo-2 ls motherboard. any nic hardware
settings u can think of that would cause this?

"Bob Willard" <(E-Mail Removed)> wrote in message
news:uO%(E-Mail Removed)...
> James wrote:
> > what things could be differn't. both are clean installs of xp sp1
> >
> >
> > "Bob Willard" <(E-Mail Removed)> wrote in message
> > news:uEr$(E-Mail Removed)...
> >
> >>James wrote:
> >>
> >>
> >>>but lots of files tranfer fast when i use another workstation to the

> >
> > same
> >
> >>>server.
> >>>
> >>>
> >>>
> >>>"Bob Willard" <(E-Mail Removed)> wrote in message
> >>>news:(E-Mail Removed)...
> >>>
> >>>
> >>>>James wrote:
> >>>>
> >>>>
> >>>>>how come my transfers are so slow when copying files from my xp

machine
> >>>
> >>>to
> >>>
> >>>
> >>>>>my win2k machine on my 100mbps switched lan?
> >>>>>large file = 100MB .mdb
> >>>>>small files = total 7MB's of 400 text files
> >>>>>
> >>>>>it takes 12 mins to transfer the 7 MB's and only 30 seconds to

transfer
> >>>
> >>>the
> >>>
> >>>
> >>>>>large file.
> >>>>>
> >>>>>
> >>>>
> >>>>And how come digging a ditch with one backhoe is faster than using
> >>>>ten men with shovels? Or using a thousand men with teaspoons?
> >>>>
> >>>>Files are transferred one at a time. For each file, the source PC

must
> >>>>look up the file's metadata (to translate the file name to the file
> >>>>location, check security params, etc.). Worse still, is what the

target
> >>>>PC must do for each file: it must find some unused space on the HD,

then
> >>>>allocate some of that space, then create the metadata for the new

file,
> >>>>then ask the source PC for the data, then copy the data to the

allocated
> >>>>space, then update the metadata (when all data has been copied), then
> >>>>notify the source PC that the copy is done.
> >>>>
> >>>>It takes a lot of work to copy a file across a net, whether the file

is
> >>>>tiny or huge. For a huge file, copying data take most of the time;

for
> >
> > a
> >
> >>>>tiny file, copying data takes far less time than all of the other

> >
> > overhead
> >
> >>>>work.
> >>>>--
> >>>>Cheers, Bob
> >>>>
> >>>
> >>>
> >>>
> >> From your question, I assumed that you were interested in why bunches
> >>of files take longer than a single file to transfer; that is the issue
> >>I addressed. If you want to understand why one PC is faster than

another,
> >>then you should conduct some measurements in which only one variable at
> >>a time is changed -- and "the workstation" is not one variable, it is
> >>probably many.
> >>--
> >>Cheers, Bob
> >>

> >
> >
> >

>
> NIC speed, RWIN, protocols used, DNS settings, HD speed, CPU frequency,
> IDE speed, RAM speed and quantity, pagefile min/max params and HD(s),
> XP PRO v. HE, NTFS v. FAT32, user credentials, other traffic on the LAN,
> HD RPM, HD fragmentation, CPU type, cache params, MB type/revision, BIOS
> type, version, and settings, NIC vendor/model, NIC card v. chip,

concurrent
> apps running, domain v. workgroup, etc.
>
> And, maybe, lots of other stuff. It takes care and effort to compare two
> things that are *exactly* alike except for one variable.
> --
> Cheers, Bob
>



 
Reply With Quote
 
Bob Willard
Guest
Posts: n/a
 
      3rd Feb 2004
James wrote:
> both computers are pretty new i think they should be able to support
> transfering files just fine with their hardware. im guessing either a NIC
> settting is the case here or a bad NIC. this NIC im using is the onboard one
> that came with this msi 865PE neo-2 ls motherboard. any nic hardware
> settings u can think of that would cause this?
>
> "Bob Willard" <(E-Mail Removed)> wrote in message
> news:uO%(E-Mail Removed)...
>
>>James wrote:
>>
>>>what things could be differn't. both are clean installs of xp sp1
>>>
>>>
>>>"Bob Willard" <(E-Mail Removed)> wrote in message
>>>news:uEr$(E-Mail Removed)...
>>>
>>>
>>>>James wrote:
>>>>
>>>>
>>>>
>>>>>but lots of files tranfer fast when i use another workstation to the
>>>
>>>same
>>>
>>>
>>>>>server.
>>>>>
>>>>>
>>>>>
>>>>>"Bob Willard" <(E-Mail Removed)> wrote in message
>>>>>news:(E-Mail Removed)...
>>>>>
>>>>>
>>>>>
>>>>>>James wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>>how come my transfers are so slow when copying files from my xp

>
> machine
>
>>>>>to
>>>>>
>>>>>
>>>>>
>>>>>>>my win2k machine on my 100mbps switched lan?
>>>>>>>large file = 100MB .mdb
>>>>>>>small files = total 7MB's of 400 text files
>>>>>>>
>>>>>>>it takes 12 mins to transfer the 7 MB's and only 30 seconds to

>
> transfer
>
>>>>>the
>>>>>
>>>>>
>>>>>
>>>>>>>large file.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>And how come digging a ditch with one backhoe is faster than using
>>>>>>ten men with shovels? Or using a thousand men with teaspoons?
>>>>>>
>>>>>>Files are transferred one at a time. For each file, the source PC

>
> must
>
>>>>>>look up the file's metadata (to translate the file name to the file
>>>>>>location, check security params, etc.). Worse still, is what the

>
> target
>
>>>>>>PC must do for each file: it must find some unused space on the HD,

>
> then
>
>>>>>>allocate some of that space, then create the metadata for the new

>
> file,
>
>>>>>>then ask the source PC for the data, then copy the data to the

>
> allocated
>
>>>>>>space, then update the metadata (when all data has been copied), then
>>>>>>notify the source PC that the copy is done.
>>>>>>
>>>>>>It takes a lot of work to copy a file across a net, whether the file

>
> is
>
>>>>>>tiny or huge. For a huge file, copying data take most of the time;

>
> for
>
>>>a
>>>
>>>
>>>>>>tiny file, copying data takes far less time than all of the other
>>>
>>>overhead
>>>
>>>
>>>>>>work.
>>>>>>--
>>>>>>Cheers, Bob
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>From your question, I assumed that you were interested in why bunches
>>>>of files take longer than a single file to transfer; that is the issue
>>>>I addressed. If you want to understand why one PC is faster than

>
> another,
>
>>>>then you should conduct some measurements in which only one variable at
>>>>a time is changed -- and "the workstation" is not one variable, it is
>>>>probably many.
>>>>--
>>>>Cheers, Bob
>>>>
>>>
>>>
>>>

>>NIC speed, RWIN, protocols used, DNS settings, HD speed, CPU frequency,
>>IDE speed, RAM speed and quantity, pagefile min/max params and HD(s),
>>XP PRO v. HE, NTFS v. FAT32, user credentials, other traffic on the LAN,
>>HD RPM, HD fragmentation, CPU type, cache params, MB type/revision, BIOS
>>type, version, and settings, NIC vendor/model, NIC card v. chip,

>
> concurrent
>
>>apps running, domain v. workgroup, etc.
>>
>>And, maybe, lots of other stuff. It takes care and effort to compare two
>>things that are *exactly* alike except for one variable.
>>--
>>Cheers, Bob
>>

>
>
>


10 Mb/s, either explicitly set or the result of a bad auto-detect.
Or, HDX instead of FDX, explicit or bad auto-detect.

And, I would look at the DNS settings and protocols used for diff's
between those two PCs. Also, make sure you have the same credentials
and the same filesystem (preferably NTFS) with the same file sharing
(simple or ACLs) on both.
--
Cheers, Bob

 
Reply With Quote
 
 
 
Reply

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
macros making files slower and slower Raja Microsoft Excel Misc 1 9th May 2008 12:26 AM
Lots of error logs if I transfer files over my network kimiraikkonen Windows XP General 1 17th Sep 2007 11:19 PM
Lots of error logs if I transfer files over my network kimiraikkonen Windows XP Networking 1 10th Sep 2007 08:43 PM
IE/Wininet cache breaks when it has lots of small files in it, isthere any work around? William Bardwell Windows XP Internet Explorer 0 5th Apr 2005 09:41 AM
GC with lots of small ones AlexS Microsoft Dot NET 10 28th May 2004 12:52 AM


Features
 

Advertising
 

Newsgroups
 


All times are GMT +1. The time now is 04:36 PM.