FileSystemWatcher

[Mufasa]

I have need to be polling a directory for files ( I am going to be using
files to have programs communication between each other - don't ask! )
Anyway - the question I have is what happens if in the middle of processing
something another files comes in. Does the event get interrupted because the
new file is now ready? If so - is there a way to stop it? Essentially make
the event a critical region that can't be interrupted.

TIA - Jeff.

 

[Mr. Arnold]

I have need to be polling a directory for files ( I am going to be using
files to have programs communication between each other - don't ask! )
Anyway - the question I have is what happens if in the middle of processing
something another files comes in. Does the event get interrupted because
the new file is now ready? If so - is there a way to stop it? Essentially
make the event a critical region that can't be interrupted.

So why don't you spawn a thread on a thread timer or use the Timer control
to query the file queue?

 

[Moty Michaely]

So why don't you spawn a thread on a thread timer or use the Timer control
to query the file queue?

Hi,

You will receive sequential events from the FileSystemWatcher.

I would suggest not to miss an event, to handle the event
asynchronously (async delegete for instance).

For your question, mark the section that handles the event as a
critical section (using lock or Monitor) and the next event processing
will wait until the lock is released...

Hope this helps.
Moty

 

[Mufasa]

OK Great. Thanks.

Hi,

You will receive sequential events from the FileSystemWatcher.

I would suggest not to miss an event, to handle the event
asynchronously (async delegete for instance).

For your question, mark the section that handles the event as a
critical section (using lock or Monitor) and the next event processing
will wait until the lock is released...

Hope this helps.
Moty

 

[Mufasa]

If I spawn a thread to monitor the directory, that still doesn't solve the
problem of FileSystemWatch interrupting itself.

J.

 

[Mr. Arnold]

If I spawn a thread to monitor the directory, that still doesn't solve the
problem of FileSystemWatch interrupting itself.

J.

I guess it would be why use FileSystemWatch at all if you knew files were
coming into a queuing directory.

 

[Moty Michaely]

I guess it would be why use FileSystemWatch at all if you knew files were
coming into a queuing directory.

Hi,

Why not use FileSystemWatcher?
Moty

 

[Mr. Arnold]

Hi,

Why not use FileSystemWatcher?
Moty

It's just the fact that you know files are going to come into a directory.
On a timer basis, one would just read all the files, process them and delete
or move them out of the way when done with the batch. The timer would just
go back to sleep until the time has elapsed again.

FileSystemWatcher, I look at as an event to trigger an action, because a
particular file showed.

 

[Mufasa]

But I don't want to have to constantly poll and I'd like it to be processed
immediately.

 

[Mr. Arnold]

But I don't want to have to constantly poll and I'd like it to be
processed immediately.

Then set the timer to 1 second.

 

[Moty Michaely]

Then set the timer to 1 second.

Hi,

Let's just assume that you are invited to a party of your friend.
Would you like your friend to call you every 10 seconds and ask you
"Are you coming? Where are you?"

Or just make sure your friend will be notified as soon as you arrived
the party?

Back to the discussion: Checking the file system every second is time
consuming, resource consuming and is not the smart thing to do..
FileSystemWatcher uses system calls and is based on the operating
system to notify about file system changes. The operating system
doesn't have to poll every 1 second and ask the file system "How are
you?". Since every file system call is made by the OS, it knows on any
change and just notifies it's subscribers.

Why bother asking the File System every second, if it can notify you
on any change?

What if the directory will not change for the next 3 days? The service
will call the file system (24*60*60*3 = 259200) times!!!! (And that's
just for 3 days with no activity, like weekends.. What about
holidays?). What is it good for???
The operating system has it's algorithms and mechanism for File System
monitoring and managing.

Anyhow, I think you got the point.
Moty

 

[Mr. Arnold]

Hi,

Let's just assume that you are invited to a party of your friend.
Would you like your friend to call you every 10 seconds and ask you
"Are you coming? Where are you?"

Or just make sure your friend will be notified as soon as you arrived
the party?

It depends on the situaition. If there is a hot honey with a girl firend
there too and he needs me to assist, I think you get my point.

Back to the discussion: Checking the file system every second is time
consuming, resource consuming and is not the smart thing to do..

Based on what? That's your opinion and everyone got an opinion.The program
is running regardless using resouces.

FileSystemWatcher uses system calls and is based on the operating
system to notify about file system changes. The operating system
doesn't have to poll every 1 second and ask the file system "How are
you?". Since every file system call is made by the OS, it knows on any
change and just notifies it's subscribers.

I have used FileSystemWatcher on more than a few ocassions.

Why bother asking the File System every second, if it can notify you
on any change?

It depends upon the situation needed.

What if the directory will not change for the next 3 days? The service
will call the file system (24*60*60*3 = 259200) times!!!! (And that's
just for 3 days with no activity, like weekends.. What about
holidays?). What is it good for???

As long as the application is up and running, I would not be concerned about
this in the least bit? It can poll 1 trillon times, as long as it gets the
file or files.

The operating system has it's algorithms and mechanism for File System
monitoring and managing.

It has the resources to do so and resources are being used. It would
certainly be nothing I would be concerned about, in today's world of
computer power and resources as far some kind of polling.

The whole thing depends upon how the application is built and the specs
required. It can either be done with polling , which I prefer, or
FileSystemWatcher, which I don't prefer that much.

 

[Moty Michaely]

Dear Mr. Arnold,

It depends on the situaition. If there is a hot honey with a girl firend
there too and he needs me to assist, I think you get my point.

I got the point, but you got mine too..

Based on what? That's your opinion and everyone got an opinion.The program
is running regardless using resouces.

Based on my Operating Systems knowledge. It's not an opinion, it's a
fact.

I have used FileSystemWatcher on more than a few ocassions.

Why have you used it? Why not poll things out?

It depends upon the situation needed.

Please give me a situation that has an option of trigerring and an
option of polling and you would use polling.

As long as the application is up and running, I would not be concerned about
this in the least bit? It can poll 1 trillon times, as long as it gets the
file or files.

Sorry to bother your concerns.
No you can't poll 1 trillion times if only 1 communication process is
needed. It's a *bad* conceptual thought.

It has the resources to do so and resources are being used. It would
certainly be nothing I would be concerned about, in today's world of
computer power and resources as far some kind of polling.

No it dosen't have the resources to do so. As a developer, you can't
assume such thing and even if you can assume, why use all the
resources just for 1 purpose???
Why use Hash table for fast search if we can use List for worse search
time? The resources are there...
Or even better one: Why not matrix of 10000x10000 to hold Excell style
sheet (with only 1 cell that is currently in use)?
The computer has the resources for it...

The whole thing depends upon how the application is built and the specs
required. It can either be done with polling , which I prefer, or
FileSystemWatcher, which I don't prefer that much.

True. The OP specs is to watch a folder, and polling is a bad idea to
accomplish this.

Computers are not just about development, it's about using the given
tools smartly, and design your software according to your needs but
best for the computer needs.

That's why we, computer scientists, are not just syntactical
professional. We are scientists.
And that is my opinion.

We can transform this discussion to discussion on Polling vs.
Trigerring and I really think Trigerring concept will win...

And that's all I have to say about that.

Cheers,
Moty

 

[Mr. Arnold]

Dear Mr. Arnold,


I got the point, but you got mine too..


Based on my Operating Systems knowledge. It's not an opinion, it's a
fact.

I am not concerned about that.

Why have you used it? Why not poll things out?

That's because I look at FileSystemWatcher as an *event* trigger to do
something. If I need something to fire off because I placed a file in a
directory for the firing of an event for another process, then that's going
to happen.

But if I got x amount of files coming into a directory/queue and I need a
linear method of processing those files in the queue, I am doing a polling.
I got a certain amount in hand of files I am processing, and I got more
files coming into the queue from various processes. I am certainly not going
to be using a FileSystemWatcher. I will be using a polling method.

Please give me a situation that has an option of trigerring and an
option of polling and you would use polling.

See up above.

Sorry to bother your concerns.
No you can't poll 1 trillion times if only 1 communication process is
needed. It's a *bad* conceptual thought.

You don't know what the sitiation may be as to why anything may or may not
be done.

No it dosen't have the resources to do so. As a developer, you can't
assume such thing and even if you can assume, why use all the
resources just for 1 purpose???

You never developed a solution with one purpose and one purpose only that is
running on a dedicated machine? I have certianly done that.

Computers are not just about development, it's about using the given
tools smartly, and design your software according to your needs but
best for the computer needs.

Maybe, you have not developed business solutions in a corporate environment.

That's why we, computer scientists, are not just syntactical
professional. We are scientists.
And that is my opinion.

I produce business solutions that work. I am not trying to play scientist.

We can transform this discussion to discussion on Polling vs.
Trigerring and I really think Trigerring concept will win...

I doubt that. It all sounds good, until you step into the real world of
business/corporate IT.

 

[Moty Michaely]

Hi,

First, you can't claim things you don't know.

Second, I really hope that developers that try to make things to work
and don't take into account how will it work will endorse their
profession.

I am not concerned about that.

Why not? What are you concerned about when developing low level
applications? Their looks?

You don't know what the sitiation may be as to why anything may or may not
be done.

That's exactly what I was trying to say. I suggest to consider the
worst case to process 1 trillion communication requests. Your solution
suggests to process 1 trillion communication requests for all cases!
Since you can't predict the situation, do your best to solve the worst
case!

See up above.

Again, why not using any kind of monitoring mechanism? Still can't get
that.
Are you aware of signaling mechanisms and their use? Why are Event-
Driven applications developed at all if no event's are needed? Polling
is the last choice I would consider, and that's after I gave up on all
the other one's....

You never developed a solution with one purpose and one purpose only that is
running on a dedicated machine? I have certianly done that.

I hope that the IT professional that hired you was aware of your
conceptual thoughts when he had to pay for the dedicated machine...

Maybe, you have not developed business solutions in a corporate environment.

Yes I have and still am. And I also have done lot's of work fixing
stuff that developers that has concepts like you (make things work, no
matter how) developed.

I produce business solutions that work. I am not trying to play scientist.

Unfortunately.


I doubt that. It all sounds good, until you step into the real world of
business/corporate IT.

I am at this world. What world do you think I live at? Lecture?

....
No offense of course, nothing is personal =).

Cheers,
Moty

 

[Mr. Arnold]

Hi,

First, you can't claim things you don't know.

I don't know what you're talking about here.

Second, I really hope that developers that try to make things to work
and don't take into account how will it work will endorse their
profession.

This doesn't make any sense.

Why not? What are you concerned about when developing low level
applications? Their looks?

Do you even know what back-end processing is about?

Yeah know, there are a lot of things that need to happen in IT in processing
data than what's happening up in some user's face at the UI.

That's exactly what I was trying to say. I suggest to consider the
worst case to process 1 trillion communication requests. Your solution
suggests to process 1 trillion communication requests for all cases!
Since you can't predict the situation, do your best to solve the worst
case!

I am not suggesting anything. I am saying it depends upon the situation the
solution was built for. And yes, I have seen solutions poll countless times
throughout the minute, hour, day, week, month and year.

Again, why not using any kind of monitoring mechanism? Still can't get
that.

Well, you come up with a solution that polled a Btrive database for
transactions in all of the tables that's was used for an insurance
administation application, pulling those transactions to a SQL Server
database with like tables doing inserts, updates and deletes, which was
being done 24/7*365 for reporting purposes.

Sometimes, 1 million transactions a day were being pulled across. The whole
application was a monitoring solution running on several timed thread
dedicated to certain areas of the administration transactions, with slowing
things down, speeding things and stopping threads as necessary.

Are you aware of signaling mechanisms and their use? Why are Event-
Driven applications developed at all if no event's are needed? Polling
is the last choice I would consider, and that's after I gave up on all
the other one's....

I am more than aware. And one shoe simply doesn't fit all solutions

I hope that the IT professional that hired you was aware of your
conceptual thoughts when he had to pay for the dedicated machine...

I am the best there is. There is none better, but you see, I am more than
just a code junkie.

And it wasn't all my thoughts as several developers were in on developing
the solution, which was being talked about above. It ran like a champ and
did what it was suppose to do.

Yes I have and still am. And I also have done lot's of work fixing
stuff that developers that has concepts like you (make things work, no
matter how) developed.

You no more know who I am and what I have done throughout my 20+ some years
of developing solutions than the Man in the Moon.

Unfortunately.

It's unfortunate that you think you're some kind of God's gift to
programming. Man, get out of here with this.

I am at this world. What world do you think I live at? Lecture?

Apparently, it's your world the rest of us just live in it.

I see I am going to have to cut you loose and move on. I see you're about to
go off the deep-end with this. I take that back. You have gone off the
deep-end.

<Plank> that's a soft logical <plonk>.

 

[Moty Michaely]

Dear Mr. Arnold,

I don't know what you're talking about here.

You claimed things about my self that weren't true.

This doesn't make any sense.

Why not?

Do you even know what back-end processing is about?

Yeah know, there are a lot of things that need to happen in IT in processing
data than what's happening up in some user's face at the UI.

True, but still, back-end software should and must be as much
efficient it can be.

I am not suggesting anything. I am saying it depends upon the situation the
solution was built for. And yes, I have seen solutions poll countless times
throughout the minute, hour, day, week, month and year.

That doesn't mean it was right to do. If there is any alternative
(better one) to polling, it should be used. That's all I was saying.

Well, you come up with a solution that polled a Btrive database for
transactions in all of the tables that's was used for an insurance
administation application, pulling those transactions to a SQL Server
database with like tables doing inserts, updates and deletes, which was
being done 24/7*365 for reporting purposes.

Sometimes, 1 million transactions a day were being pulled across. The whole
application was a monitoring solution running on several timed thread
dedicated to certain areas of the administration transactions, with slowing
things down, speeding things and stopping threads as necessary.

SQL Server has it's notification services, just for this purposes. SQL
Server 2005 has built in CLR support which gives us the opportunity to
trigger notifications rather than to poll things out 24/7*365 times
for reporting purposes (and much more).

Without notification services, polling was a choice most of the
developers used. But, say that we have 50 client applications that
polls data, it seems quite heavy. But hey, I know nothing about
development, so forget I said something. Let's poll.

I am more than aware. And one shoe simply doesn't fit all solutions

True. I never said that. See the entire post when I said: If a
trigerring mechanism is available, it's better than polling. If not,
try avoid polling and consider it as the last choice.

And it wasn't all my thoughts as several developers were in on developing
the solution, which was being talked about above. It ran like a champ and
did what it was suppose to do.

Again, things can be done in infinite ways. We, as developers, should
choose the best solution. Best means efficient, elegant ,maintainable
etc.
I am glad it worked like a charm. Seems you are quite a professional.

You no more know who I am and what I have done throughout my 20+ some years
of developing solutions than the Man in the Moon.

You are way out of line here. <g>

Facts, facts and more facts.

It's unfortunate that you think you're some kind of God's gift to
programming. Man, get out of here with this.

Thanks for the compliment, but I am far far from being any kind of
God's gift to programming.

Unfortunately, you take things personal. You still havne't given me
one reason to use polling as alternative to trigerring. It's a
professional discussion about what the OP wanted and not about who is
better in programming.
You are much much better. You know anything about programming and I
know nothing. Okay?

Apparently, it's your world the rest of us just live in it.

Vice versa.

I see I am going to have to cut you loose and move on. I see you're about to
go off the deep-end with this. I take that back. You have gone off the
deep-end.

Why? I really want to know what is that thing that I miss? Why is
polling better than trigerring, of any kind. I really want to know, to
be better programmer. To achieve what you've achieved so far. Really.
That's all.
Hope you understand.
Cheers,
Moty

 

[Mr. Arnold]

It's not read.
<disappear>
<Plank> you and the whine.