PC Review


Reply
Thread Tools Rate Thread

Copy large # of files from one directory

 
 
DeanB
Guest
Posts: n/a
 
      14th Sep 2007
I have 100,000 small image files in a directory. I press Select All
and it takes minutes (if not hours) to respond, and to copy them just
generally fails. Not sure why - I'm assuming its a memory issue in
loading up file information (header info), but I don't really know.
Makes it a PITA to move files around - I had to write a little program
that moves files one at a time, rather than doing the windows select
all.

Any advice welcomed.

 
Reply With Quote
 
 
 
 
Pegasus \(MVP\)
Guest
Posts: n/a
 
      14th Sep 2007

"DeanB" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
>I have 100,000 small image files in a directory. I press Select All
> and it takes minutes (if not hours) to respond, and to copy them just
> generally fails. Not sure why - I'm assuming its a memory issue in
> loading up file information (header info), but I don't really know.
> Makes it a PITA to move files around - I had to write a little program
> that moves files one at a time, rather than doing the windows select
> all.
>
> Any advice welcomed.


Robocopy.exe is the recommended tool for such large volumes.
You can download it from the Microsoft site.


 
Reply With Quote
 
Homer J. Simpson
Guest
Posts: n/a
 
      14th Sep 2007
Personally I'd just drop to a command prompt and do the copy from there.


 
Reply With Quote
 
Pegasus \(MVP\)
Guest
Posts: n/a
 
      14th Sep 2007

"Homer J. Simpson" <root@127.0.0.1> wrote in message
news:(E-Mail Removed)...
> Personally I'd just drop to a command prompt and do the copy from there.
>


This works in theory but in practice it may fail because
of a known bug with xcopy.exe when copying large
numbers of files.


 
Reply With Quote
 
Patrick Keenan
Guest
Posts: n/a
 
      15th Sep 2007
"DeanB" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
>I have 100,000 small image files in a directory. I press Select All
> and it takes minutes (if not hours) to respond, and to copy them just
> generally fails. Not sure why - I'm assuming its a memory issue in
> loading up file information (header info), but I don't really know.
> Makes it a PITA to move files around - I had to write a little program
> that moves files one at a time, rather than doing the windows select
> all.
>
> Any advice welcomed.


In addition to robocopy, you might look at FileSync. www.fileware.com

HTH
-pk


 
Reply With Quote
 
Homer J. Simpson
Guest
Posts: n/a
 
      23rd Sep 2007
>> Personally I'd just drop to a command prompt and do the copy from there.
>
> This works in theory but in practice it may fail because
> of a known bug with xcopy.exe when copying large
> numbers of files.


How "large" is large? I used to use xcopy to do just that on a regular
basis, with a set of ~150,000 files. The number of files in the target
folder always ended up matching the number of files in the source.

Was this some older version of xcopy? Any idea if this has been fixed in
more recent Windows versions?


 
Reply With Quote
 
Pegasus \(MVP\)
Guest
Posts: n/a
 
      23rd Sep 2007

"Homer J. Simpson" <root@127.0.0.1> wrote in message
news:eFLLfGg$(E-Mail Removed)...
>>> Personally I'd just drop to a command prompt and do the copy from there.

>>
>> This works in theory but in practice it may fail because
>> of a known bug with xcopy.exe when copying large
>> numbers of files.

>
> How "large" is large? I used to use xcopy to do just that on a regular
> basis, with a set of ~150,000 files. The number of files in the target
> folder always ended up matching the number of files in the source.
>
> Was this some older version of xcopy? Any idea if this has been fixed in
> more recent Windows versions?


I don't know what the limit is - perhaps > 1000 files?. I have observed
the phenomenon on several occasions on different machines, both on
Win2000 and WinXP. It is also reported regularly in newsgroups.


 
Reply With Quote
 
Homer J. Simpson
Guest
Posts: n/a
 
      23rd Sep 2007
> I don't know what the limit is - perhaps > 1000 files?. I have observed
> the phenomenon on several occasions on different machines, both on
> Win2000 and WinXP. It is also reported regularly in newsgroups.


1000 is pretty low. I'm not saying it doesn't happen, but I don't recall
seeing any evidence this ever has caused me any problem...since my DOS 2.0
days.


 
Reply With Quote
 
 
 
Reply

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Help to copy files names in a column from one directory to another directory VBA Paul B Microsoft Excel Programming 4 10th Nov 2008 01:54 AM
vista can't copy large files? another XP file-copy bug? Dave Windows Vista General Discussion 2 26th Jan 2008 07:47 PM
Would like to write batch file to copy all TXT files in a given directory and all subdirectories to a single target directory. Rob Windows XP General 5 20th Aug 2007 02:42 PM
Access directory with zip file (especially large zip files) extremely slow Andy Shiu Windows XP General 1 23rd Jun 2004 05:55 PM
Copy all files and sub-directory under a specific directory Phil Microsoft Access 1 20th Dec 2003 11:06 AM


Features
 

Advertising
 

Newsgroups
 


All times are GMT +1. The time now is 05:24 AM.