Slow performance opening a file in folder with large number of files

  • Thread starter Thread starter Andrew Meador
  • Start date Start date
A

Andrew Meador

I have an Access databse that references image files in a folder.
There are currently 2700 files in this folder and will reach a total
of appx 10000 files when data collection is done. The one of the
database screens allows the user to double-click the image which then
executes an external program with the path and name of the file to
open. So the external app gets the exact path and file name to open,
but the open process takes about 20 seconds to occure. I can go strait
to the folder where the files are a double click the file and the
application still takes and equally long time to open the file.
However, if in a folder with only a small number of files, the app
opens the image almost instantly. So there appears to be a preformance
issue with opening even an explicit file in a folder with a large
number of files. I know I could split the files over multiple folder
to improve performance, but this require a fair bit of re-coding of
the database and re-trainined of the users, etc... Is there any
performance options beides the multiple folder idea? Why does the
number of files in the folder hind the opening process so badly when
specific file is targeted?

Thanks in advance!
Andrew
 
Andrew said:
I have an Access databse that references image files in a folder.
There are currently 2700 files in this folder and will reach a total
of appx 10000 files when data collection is done. The one of the
database screens allows the user to double-click the image which then
executes an external program with the path and name of the file to
open. So the external app gets the exact path and file name to open,
but the open process takes about 20 seconds to occure. I can go strait
to the folder where the files are a double click the file and the
application still takes and equally long time to open the file.
However, if in a folder with only a small number of files, the app
opens the image almost instantly. So there appears to be a preformance
issue with opening even an explicit file in a folder with a large
number of files. I know I could split the files over multiple folder
to improve performance, but this require a fair bit of re-coding of
the database and re-trainined of the users, etc... Is there any
performance options beides the multiple folder idea? Why does the
number of files in the folder hind the opening process so badly when
specific file is targeted?

Thanks in advance!
Andrew
Click to expand...

If you are relying too heavily on the pagefile, adding more RAM will
yield a tremendous performance benefit.

How much RAM do you have?
 
If you are relying too heavily on the pagefile, adding more RAM will
yield a tremendous performance benefit.

How much RAM do you have?- Hide quoted text -

- Show quoted text -
Click to expand...

I think I'm ok there. The system has 2GB RAM with only apx 800MB Total
Commit Charge (on one test system). My dev computer where I have been
doing most of my dev work is also doing the same thing and it has 3GB
or RAM with 1.5GB Total Commit Charge
 
Andrew said:
I think I'm ok there. The system has 2GB RAM with only apx 800MB Total
Commit Charge (on one test system). My dev computer where I have been
doing most of my dev work is also doing the same thing and it has 3GB
or RAM with 1.5GB Total Commit Charge
Click to expand...

The problem seems to be with the application rather than the operating
system, what do the vendors of the application have to say about this
latency when opening files? 2,700 files is not really very many files,
nor is 10,000 for that matter, there could be a delay when
opening/reading the folder for the first time, but it should be quick on
subsequent operations.

John
 
The problem seems to be with the application rather than the operating
system, what do the vendors of the application have to say about this
latency when opening files?  2,700 files is not really very many files,
nor is 10,000 for that matter, there could be a delay when
opening/reading the folder for the first time, but it should be quick on
subsequent operations.

John- Hide quoted text -

- Show quoted text -
Click to expand...

I'll have to follow up with them to see. I guess it could be that
maybe they pull a file list or something each time, even if the file
name is specified.
 
Andrew said:
I have an Access databse that references image files in a folder.
There are currently 2700 files in this folder and will reach a total
of appx 10000 files when data collection is done. The one of the
database screens allows the user to double-click the image which then
executes an external program with the path and name of the file to
open. So the external app gets the exact path and file name to open,
but the open process takes about 20 seconds to occure. I can go strait
to the folder where the files are a double click the file and the
application still takes and equally long time to open the file.
However, if in a folder with only a small number of files, the app
opens the image almost instantly. So there appears to be a preformance
issue with opening even an explicit file in a folder with a large
number of files. I know I could split the files over multiple folder
to improve performance, but this require a fair bit of re-coding of
the database and re-trainined of the users, etc... Is there any
performance options beides the multiple folder idea? Why does the
number of files in the folder hind the opening process so badly when
specific file is targeted?

Thanks in advance!
Andrew
Click to expand...

FAT32 or NTFS?

I can imagine a delay as the system has to meander through a FAT32 directory
sequentially.

You might tickle up Process Explorer to discover who's using the cycles
during a supposed fetch of the file.
 
Back
Top