XP SP2 File Handling Sucks

[justintime]

I am running XP SP2 and am having a lot of problems with file copying
and deleting large numbers of files. I am managing the wifes collection
of embroidery designs and she has in excess of 250,000 recently her Hard
drive went bad that had her designs. I have them backup ed on many cd's
and dvd's.

So I decided to take this opportunity to try and organize them into a
decent structure due to the fact that there are many duplicates with
different folders names just spelled different etc.
The problem is I am first copying all the cd's and dvd's onto a hard
drive so as to weed out the dups. Each folder may have from 1 to 100
directories and each directory may have 1 to 100 small files < 100k in
them. When I tag a group of directories first it takes anywhere from 1
to 15 min for windows to Get READY to copy, then it starts copying. The
more files it copies the slower each file gets copied. In other words
the first 100 may take 10 sec then it gets down to seconds per file.

Is there a better way or more efficient way to copy files from one dir
to another or from the CD to a folder. Maby another program?

I also have the problem of one dir withen another. Ex.
Main = Dir1
Sub = Dir1
Files = file1, file2, file3 etc.

I want to move all the files from the sub dir into the main due to
duplicate dir names. Sometimes it goes 3 or more sub dirs deep.

Anybody know of something to aid in this?

Thanks

 

[justintime]

I am running an AMD 2600 and 1gb of ram with 2 160gb HD and 1 60gb where
Im putting the files.

 

[Phil Weldon]

'justintime' wrote, in part:
| I am running XP SP2 and am having a lot of problems with file copying
| and deleting large numbers of files. I am managing the wifes collection
| of embroidery designs and she has in excess of 250,000 recently her Hard
| drive went bad that had her designs. I have them backup ed on many cd's
| and dvd's
..
| Is there a better way or more efficient way to copy files from one dir
| to another or from the CD to a folder. Maby another program?
_____

Sure.

'Copy' one ENTIRE CD or DVD to one folder on the hard drive.

THEN
'Move' the individual files to the new directory structure you have set
up on the hard drive, weeding out as you go (use 'Move' rather than 'Copy' -
'Move' just changes the directory information, and does NOT move the files;
the files are just reassigned to new directories.

Repeat until done.

If I understand your post correctly, what you are currently doing requires
random access from the CD or DVD drive. For random access on a CD/DVD drive
the access time is in the hundreds of milliseconds. Compare that to less
than 20 ms access time for random access on a hard drive.

Also consider the following
* Make sure you have space on the new hard drive for the data, plus
space for organizing the data, and that afterwards you will still have more
than 15% free space. In fact, if you will have less than 50% free space on
the new drive, then you should have purchased a larger capacity drive.
* Clean up the drive to free as much space as possible, then defragment
before beginning the transfers.
* Since you are adding so many files, some of them very small, the file
system may have to increase the Master File Table size.



By the way, the subject line you used is not helpful. I realize you are
frustrated, but asking a question and getting an answer is the best relief.
Just remember, the problem you now face in reorganizing took years to
create; don't expect complete relief B^)

Phil Weldon

| I am running XP SP2 and am having a lot of problems with file copying
| and deleting large numbers of files. I am managing the wifes collection
| of embroidery designs and she has in excess of 250,000 recently her Hard
| drive went bad that had her designs. I have them backup ed on many cd's
| and dvd's.
|
| So I decided to take this opportunity to try and organize them into a
| decent structure due to the fact that there are many duplicates with
| different folders names just spelled different etc.
| The problem is I am first copying all the cd's and dvd's onto a hard
| drive so as to weed out the dups. Each folder may have from 1 to 100
| directories and each directory may have 1 to 100 small files < 100k in
| them. When I tag a group of directories first it takes anywhere from 1
| to 15 min for windows to Get READY to copy, then it starts copying. The
| more files it copies the slower each file gets copied. In other words
| the first 100 may take 10 sec then it gets down to seconds per file.
|
| Is there a better way or more efficient way to copy files from one dir
| to another or from the CD to a folder. Maby another program?
|
| I also have the problem of one dir withen another. Ex.
| Main = Dir1
| Sub = Dir1
| Files = file1, file2, file3 etc.
|
| I want to move all the files from the sub dir into the main due to
| duplicate dir names. Sometimes it goes 3 or more sub dirs deep.
|
| Anybody know of something to aid in this?
|
| Thanks

 

[justintime]

Phil:
Thank you for an answer. Most of the files I am reorganizing I am using
cut and past, so I can see what I have done.
For instance I have a dir OSED and another one osed1 both may contain
the same files so I tag all in osed1 then cut then past into the OSED
dir. I realize it has to copy and delete but I find that I dont get lost
if something goes wrong and can see what I have done. I have way more
than enough hard drive space Im using a new 60gb and it only has used
10gb, Im also using another drive that has 13gb free on a 40gb
partition. Soon I will be "moving" the ones on the 40gb part to the 60gb.

I hadn't emptied the recycle bin in some time and it took 15 min to
empty first it emptied then started again and again each time the empty
time got longer.
Next I have to look for dups that will be fun I bet I have thousands.

thanks

 

[Phil Weldon]

'justintime' wrote, in part:
| Thank you for an answer. Most of the files I am reorganizing I am using
| cut and past, so I can see what I have done.
| For instance I have a dir OSED and another one osed1 both may contain
| the same files so I tag all in osed1 then cut then past into the OSED
| dir. I realize it has to copy and delete but I find that I dont get lost
| if something goes wrong and can see what I have done
_____

I probably don't understand why you are doing what you are doing, but
'moving' is safer than 'copying' because NOTHING IS ERASED. Only the
pointers are changed. When you copy, a copy is made to a different physical
location, then the original is erased. Besides, you still have the backup.

When you 'Move' from one partition to another, the data IS copied to a
different physical location, then the original is erased.

I think you might want to rethink how you go about reorganization. A few
hours with paper and pencil might make your task a lot easier, not just at
this time, but in the future.

Cut and past is the very slowest way to move files around.

One thing you should do is use two windows tiled horizontally in Windows
Explorer, then use drag and drop, (use the move option.)
For dups, you could sort by size in the two windows that should make it
easier to find them.

I won't offer any more suggestions because I really don't get what you are
doing.

Phil Weldon

| Phil:
| Thank you for an answer. Most of the files I am reorganizing I am using
| cut and past, so I can see what I have done.
| For instance I have a dir OSED and another one osed1 both may contain
| the same files so I tag all in osed1 then cut then past into the OSED
| dir. I realize it has to copy and delete but I find that I dont get lost
| if something goes wrong and can see what I have done. I have way more
| than enough hard drive space Im using a new 60gb and it only has used
| 10gb, Im also using another drive that has 13gb free on a 40gb
| partition. Soon I will be "moving" the ones on the 40gb part to the 60gb.
|
| I hadn't emptied the recycle bin in some time and it took 15 min to
| empty first it emptied then started again and again each time the empty
| time got longer.
| Next I have to look for dups that will be fun I bet I have thousands.
|
| thanks
|
| Phil Weldon wrote:
| > 'justintime' wrote, in part:
| > | I am running XP SP2 and am having a lot of problems with file copying
| > | and deleting large numbers of files. I am managing the wifes
collection
| > | of embroidery designs and she has in excess of 250,000 recently her
Hard
| > | drive went bad that had her designs. I have them backup ed on many
cd's
| > | and dvd's
| > .
| > | Is there a better way or more efficient way to copy files from one dir
| > | to another or from the CD to a folder. Maby another program?
| > _____
| >
| > Sure.
| >
| > 'Copy' one ENTIRE CD or DVD to one folder on the hard drive.
| >
| > THEN
| > 'Move' the individual files to the new directory structure you have
set
| > up on the hard drive, weeding out as you go (use 'Move' rather than
'Copy' -
| > 'Move' just changes the directory information, and does NOT move the
files;
| > the files are just reassigned to new directories.
| >
| > Repeat until done.
| >
| > If I understand your post correctly, what you are currently doing
requires
| > random access from the CD or DVD drive. For random access on a CD/DVD
drive
| > the access time is in the hundreds of milliseconds. Compare that to
less
| > than 20 ms access time for random access on a hard drive.
| >
| > Also consider the following
| > * Make sure you have space on the new hard drive for the data, plus
| > space for organizing the data, and that afterwards you will still have
more
| > than 15% free space. In fact, if you will have less than 50% free space
on
| > the new drive, then you should have purchased a larger capacity drive.
| > * Clean up the drive to free as much space as possible, then
defragment
| > before beginning the transfers.
| > * Since you are adding so many files, some of them very small, the
file
| > system may have to increase the Master File Table size.
| >
| >
| >
| > By the way, the subject line you used is not helpful. I realize you are
| > frustrated, but asking a question and getting an answer is the best
relief.
| > Just remember, the problem you now face in reorganizing took years to
| > create; don't expect complete relief B^)
| >
| > Phil Weldon
| >
| > | > | I am running XP SP2 and am having a lot of problems with file copying
| > | and deleting large numbers of files. I am managing the wifes
collection
| > | of embroidery designs and she has in excess of 250,000 recently her
Hard
| > | drive went bad that had her designs. I have them backup ed on many
cd's
| > | and dvd's.
| > |
| > | So I decided to take this opportunity to try and organize them into a
| > | decent structure due to the fact that there are many duplicates with
| > | different folders names just spelled different etc.
| > | The problem is I am first copying all the cd's and dvd's onto a hard
| > | drive so as to weed out the dups. Each folder may have from 1 to 100
| > | directories and each directory may have 1 to 100 small files < 100k in
| > | them. When I tag a group of directories first it takes anywhere from 1
| > | to 15 min for windows to Get READY to copy, then it starts copying.
The
| > | more files it copies the slower each file gets copied. In other words
| > | the first 100 may take 10 sec then it gets down to seconds per file.
| > |
| > | Is there a better way or more efficient way to copy files from one dir
| > | to another or from the CD to a folder. Maby another program?
| > |
| > | I also have the problem of one dir withen another. Ex.
| > | Main = Dir1
| > | Sub = Dir1
| > | Files = file1, file2, file3 etc.
| > |
| > | I want to move all the files from the sub dir into the main due to
| > | duplicate dir names. Sometimes it goes 3 or more sub dirs deep.
| > |
| > | Anybody know of something to aid in this?
| > |
| > | Thanks
| >
| >

 

[justintime]

I do it by opening 2 windows side by side, thats what I meant by cut
and paste, if you drag and drop from one window to the other it does the
same as cut and paste. It copies the file and then deletes it where it
was. Its just that when you drag 50 directories with 100 or more sub
directories and 500 or more files at a time it takes a long time to move
them.
Im finished that job for now. I just checked for empty dirs and deleted
them now on to finding the dupes.

Thanks

 

[Brian K]

To cut down on some of the Recycle Bin emptying time, right click the
Icon, select properties. Modify the amount of time a deleted file is
retained to a lesser time. The Recycle Bin will then do the emptying
even if you don't remember to do so.

--
________
To email me, Edit "blog" from my email address.
Brian M. Kochera
"Some mistakes are too much fun to only make once!"
View My Web Page: http://home.earthlink.net/~brian1951

 

[gls858]

I do it by opening 2 windows side by side, thats what I meant by cut
and paste, if you drag and drop from one window to the other it does the
same as cut and paste. It copies the file and then deletes it where it
was. Its just that when you drag 50 directories with 100 or more sub
directories and 500 or more files at a time it takes a long time to move
them.
Im finished that job for now. I just checked for empty dirs and deleted
them now on to finding the dupes.

Thanks

t
There are many duplicate file finder programs that could make your job a
lot easier. Do a search for find duplicate files. I know that there are
some that will find duplicate jpegs even if the file name is different.

gls858

 

[gls858]

t
There are many duplicate file finder programs that could make your job a
lot easier. Do a search for find duplicate files. I know that there are
some that will find duplicate jpegs even if the file name is different.

gls858

ahh... found the one I was thinking of http://www.gotdupes.com/
This was one of the programs recommended in one of the photo groups.

gls858

 

[Shenan Stanley]

I probably don't understand why you are doing what you are doing,
but 'moving' is safer than 'copying' because NOTHING IS ERASED.
Only the pointers are changed. When you copy, a copy is made to a
different physical location, then the original is erased. Besides,
you still have the backup.

When you 'Move' from one partition to another, the data IS copied
to a different physical location, then the original is erased.

<snip>

Hmmm.. I see a problem with what you have stated there..

'...'moving' is safer than 'copying' because NOTHING IS ERASED..."

Not really true. Moving is the only one of those two procedures where
something actually *is* erased by the process itself.

If I copy - I now have the original and the copy. Copy the copy and I have
the original, the first copy and the second copy.

If I move - and the media I am moving from can be erased from (so I don't
get an error) - then after the file has "copied" to the new location, the
original file is erased - as part of the move process.

No where does "copy" erase anything innately.
Move is like copying, except at the end there is an automatic erasure of the
original file.

 

[Phil Weldon]

'Shenan Stanley' wrote:
| Hmmm.. I see a problem with what you have stated there..
|
| '...'moving' is safer than 'copying' because NOTHING IS ERASED..."
|
| Not really true. Moving is the only one of those two procedures where
| something actually *is* erased by the process itself.
|
| If I copy - I now have the original and the copy. Copy the copy and I
have
| the original, the first copy and the second copy.
|
| If I move - and the media I am moving from can be erased from (so I don't
| get an error) - then after the file has "copied" to the new location, the
| original file is erased - as part of the move process.
|
| No where does "copy" erase anything innately.
| Move is like copying, except at the end there is an automatic erasure of
the
| original file.
_____

You are correct; I misspoke when stating that when a file is copied the
original is erased.
However, when a file is moved within a partition, it is not copied and then
erased, only the pointers are changed.

Phil Weldon

| Phil Weldon wrote:
| > I probably don't understand why you are doing what you are doing,
| > but 'moving' is safer than 'copying' because NOTHING IS ERASED.
| > Only the pointers are changed. When you copy, a copy is made to a
| > different physical location, then the original is erased. Besides,
| > you still have the backup.
| >
| > When you 'Move' from one partition to another, the data IS copied
| > to a different physical location, then the original is erased.
| <snip>
|
| Hmmm.. I see a problem with what you have stated there..
|
| '...'moving' is safer than 'copying' because NOTHING IS ERASED..."
|
| Not really true. Moving is the only one of those two procedures where
| something actually *is* erased by the process itself.
|
| If I copy - I now have the original and the copy. Copy the copy and I
have
| the original, the first copy and the second copy.
|
| If I move - and the media I am moving from can be erased from (so I don't
| get an error) - then after the file has "copied" to the new location, the
| original file is erased - as part of the move process.
|
| No where does "copy" erase anything innately.
| Move is like copying, except at the end there is an automatic erasure of
the
| original file.
|
| --
| Shenan Stanley
| MS-MVP
| --
| How To Ask Questions The Smart Way
|
|
|

 

[justintime]

Im still working, first I had to copy all the files from CD's and DVD's
to a HD then I had to convert them for the wifes machine, then I
combined many of the duplicate directory's copying over the same files.
Then I had to search for files that did not belong like formats she
couldn't use. Now I am copying all those files to another Hard drive
where I had done a partial copy and convert etc. I found another
problem. I opened 2 windows and dragged from one to the other and after
about 3 hours of copying I get the error out of resources and I have to
reboot the computer.
Hopefully Im doing the last files now which will take another couple
hours. Then again combine like directories and then I can look for dups.
However I dont plan on killing them all because there will be probley
50,000 dups and I dont want to have to make a manual decision on each
one which is what I will have to do, "delete this one or that one" and
when there are 5 or more dups what a pain. Were talking about more than
500,000 files. Then I will have to delete all the files on the drive I
originally copied them to which will take another hour or more. Thats
why I liked cut and paste. I plan on making the recycle bin as small as
possible before I do, although it might be faster to do it in dos.

However thanks for the ideas and help.

 

[justintime]

I just checked there are approx 459,000 files and 54,000 directories.

 

[Phil Weldon]

'justintime' wrote:
|I just checked there are approx 459,000 files and 54,000 directories.

You are using NTFS as your file system, right?

Phil Weldon

|I just checked there are approx 459,000 files and 54,000 directories.
|
|
|

 

[justintime]

Right

 

[justintime]

This is a really good program for renaming and manipulating file names.
It lets you move, copy or delete parts of a file or folder name.
I was able to take series of folders like
OSED 123456pictures,
OSED 654321pictures1
all having the same amount of numbers (123456) and change them to
Pictures-123456 and Pictures-234567 by tagging the folders and clicking
a couple of boxes and its free.

http://www.bulkrenameutility.co.uk/Main_Intro.php

 

[ElakbaR NETwork]

I agree with you.Very slow operating system but alot stable.
I use 2003 server faster and better





Need more help? Search the web http://search.elakbar.net

I am running XP SP2 and am having a lot of problems with file copying
and deleting large numbers of files. I am managing the wifes collection
of embroidery designs and she has in excess of 250,000 recently her Hard
drive went bad that had her designs. I have them backup ed on many cd's
and dvd's.

So I decided to take this opportunity to try and organize them into a
decent structure due to the fact that there are many duplicates with
different folders names just spelled different etc.
The problem is I am first copying all the cd's and dvd's onto a hard
drive so as to weed out the dups. Each folder may have from 1 to 100
directories and each directory may have 1 to 100 small files < 100k in
them. When I tag a group of directories first it takes anywhere from 1
to 15 min for windows to Get READY to copy, then it starts copying. The
more files it copies the slower each file gets copied. In other words
the first 100 may take 10 sec then it gets down to seconds per file.

Is there a better way or more efficient way to copy files from one dir
to another or from the CD to a folder. Maby another program?

I also have the problem of one dir withen another. Ex.
Main = Dir1
Sub = Dir1
Files = file1, file2, file3 etc.

I want to move all the files from the sub dir into the main due to
duplicate dir names. Sometimes it goes 3 or more sub dirs deep.

Anybody know of something to aid in this?

Thanks