FileSystemWatcher and Open Files

[Stefan L]

Hi NG,

I have a file driven application (a report server) which has to do some
work when new files arrive or are deleted.

When processing the notifications about newly created files from a
FileSystemWatcher I want to open the file and read some info from it.
Problem is that when opening the file directly when the event is fired I
usually get a "file used by another process".

Has anyone figured out a good way how to handle such events? Is there an
option to get a notification when the file is created and can be read?

My other alternatives I thought about, were:
1. Use a FIFO in a second thread that processes the files which are new.
Should keep iterating (with a short Thread.sleep) until all files are
imported successfully.
2. Start a worker thread from the pool for each file which tries to read
it in.
3. Forget about FileSystemWatcher and check the directory for new files
every 30 secs (Timer driven) [but how to reco if files have changed here?].

Does anyone has experience with this? Any best practices?

TIA,
Stefan

 

[Laura T]

Two tips I'm using, maybe they can help.

First one:

If you can control the sending application, it should first write the file
and then rename it.
Your app could listen then only rename changes. This way the file will be
readable when
you get the change event.
Like: sending app sends FILE.XXX and then renames it as FILE.TXT.
You intercept the rename and read it.

Second:

Use a semaphore file to indicate end of transmission. When you find create
event for the semaphore file,
scan the directory. Requires that you can control the sending app, of
course.

 

[Stefan L]

My problem is that I cannot control the sending app. The ReportServer
will be executed somewhere on a webserver and is intended to react to
files which will be copied (or modified or deleted) in a certain directory.
This can either be the explorer, some kind of FTP tool or whatsoever,
which will do the file work.

But I like your renaming idea better than the ones using file flags I
read when doing some research before I wrote to the group...

Two tips I'm using, maybe they can help.

First one:

If you can control the sending application, it should first write the file
and then rename it.
Your app could listen then only rename changes. This way the file will be
readable when
you get the change event.
Like: sending app sends FILE.XXX and then renames it as FILE.TXT.
You intercept the rename and read it.

Second:

Use a semaphore file to indicate end of transmission. When you find create
event for the semaphore file,
scan the directory. Requires that you can control the sending app, of
course.

Hi NG,

I have a file driven application (a report server) which has to do some
work when new files arrive or are deleted.

When processing the notifications about newly created files from a
FileSystemWatcher I want to open the file and read some info from it.
Problem is that when opening the file directly when the event is fired I
usually get a "file used by another process".

Has anyone figured out a good way how to handle such events? Is there an
option to get a notification when the file is created and can be read?

My other alternatives I thought about, were:
1. Use a FIFO in a second thread that processes the files which are new.
Should keep iterating (with a short Thread.sleep) until all files are
imported successfully.
2. Start a worker thread from the pool for each file which tries to read
it in.
3. Forget about FileSystemWatcher and check the directory for new files
every 30 secs (Timer driven) [but how to reco if files have changed
here?].

Does anyone has experience with this? Any best practices?

TIA,
Stefan

 

[Ignacio Machin \( .NET/ C# MVP \)]

Hi,

Has anyone figured out a good way how to handle such events? Is there an
option to get a notification when the file is created and can be read?

Nop, there is no way to know if you can access it unless you try and if you
get error is that you cannot access it

My other alternatives I thought about, were:
1. Use a FIFO in a second thread that processes the files which are new.
Should keep iterating (with a short Thread.sleep) until all files are
imported successfully.

This is an option, either with a Thread.Sleep or with a timer.

2. Start a worker thread from the pool for each file which tries to read
it in.

I prefer the first option.

3. Forget about FileSystemWatcher and check the directory for new files
every 30 secs (Timer driven) [but how to reco if files have changed
here?].

Does not solve the problem at all. Still the file is there but it;s in use
by another process.


If you can control the generation of the file you could create a "flag" file
and use this for the watcher. (see my other post about this)

If not possible then option 1 sounds good enough.

 

[mabra]

Hi !

I was in a similar situation. I just queued the "new file created" event
and a separate thread checks each file in the queue to be accessible
[open it with " FileMode.Open, FileAccess.ReadWrite, FileShare.None" to
be sure, it is ready processed and you are the only one, who can now
read it). After this test was successful, the item was removed from the
queue and a worker thread cretaed to do the final work.

May be, this helps.

Best regards,
Manfred

My problem is that I cannot control the sending app. The ReportServer
will be executed somewhere on a webserver and is intended to react to
files which will be copied (or modified or deleted) in a certain directory.
This can either be the explorer, some kind of FTP tool or whatsoever,
which will do the file work.

But I like your renaming idea better than the ones using file flags I
read when doing some research before I wrote to the group...

Two tips I'm using, maybe they can help.

First one:

If you can control the sending application, it should first write the
file and then rename it.
Your app could listen then only rename changes. This way the file will
be readable when
you get the change event.
Like: sending app sends FILE.XXX and then renames it as FILE.TXT.
You intercept the rename and read it.

Second:

Use a semaphore file to indicate end of transmission. When you find
create event for the semaphore file,
scan the directory. Requires that you can control the sending app, of
course.

Hi NG,

I have a file driven application (a report server) which has to do
some work when new files arrive or are deleted.

When processing the notifications about newly created files from a
FileSystemWatcher I want to open the file and read some info from it.
Problem is that when opening the file directly when the event is
fired I usually get a "file used by another process".

Has anyone figured out a good way how to handle such events? Is there
an option to get a notification when the file is created and can be
read?

My other alternatives I thought about, were:
1. Use a FIFO in a second thread that processes the files which are
new. Should keep iterating (with a short Thread.sleep) until all
files are imported successfully.
2. Start a worker thread from the pool for each file which tries to
read it in.
3. Forget about FileSystemWatcher and check the directory for new
files every 30 secs (Timer driven) [but how to reco if files have
changed here?].

Does anyone has experience with this? Any best practices?

TIA,
Stefan