Best way to design multithreading application

[Dave]

Ok, I appreciate any help you can give me. I am somewhat new to
threading, only understanding it conceptually, and I use the Threading
namespace for the sleep() command. So, any suggestions with code would
also be beneficial.

Basically, I need help designing how my threads should interact.
Currently, the application acquires a bit of data, processes it,
writes it to a file, sounds alarms if necessary, adds it to the UI
(charts and tables), and then loops to acquire more. This results in
data acquisition times of several seconds, and we need it under a
second. The data acquisition is more important than UI updates, but
not at the expense of UI updates never taking place.

Based on the information I know, here is what I would do (but don't
know how): Thread A, which is high priority, sits there and acquires
data over and over, adding it to a Queue 1 as it acquires more. Thread
B, which is medium-to-high priority, loops over and over, and when it
sees data in Queue 1, it writes it to a file and sets any alarm
states. It then puts the data in Queue 2. Thread C, which is lower
priority, loops over and over, and uses any data in Queue 2 to update
the UI.

Is that the correct way to design a good solution for this? If so, I
am worried about a few things:
1) If data collection is high priority, and the other threads cannot
get enough CPU time, will either Queue fill up too quickly,
effectively stalling the application (i.e. how would I "slow down" the
acquisition thread if I notice the queues filling up)?
2) Because of these extra queues filling up, am I actually taking more
time to process each acquisition given the memory needs of such a
system (i.e. should I keep it a single-threaded model)?
3) Is there even a way to prioritize threads in such a way that Thread
A is always exactly (or rather, close to) 1 second per acquisition
run, and then Threads B and C use up the extra time, with B being most
important?
4) What happens when Thread A enqueues a new chunk of data at the same
time Thread B is dequeuing one? Same for B and C. Is this where the
"lock" keyword comes into play?

Thanks in advance, and I hope someone here can shed some light on how
to accomplish this...

Dave Harris

 

[Peter Duniho]

Ok, I appreciate any help you can give me. I am somewhat new to
threading, only understanding it conceptually, and I use the Threading
namespace for the sleep() command. So, any suggestions with code would
also be beneficial.

Basically, I need help designing how my threads should interact.
Currently, the application acquires a bit of data, processes it,
writes it to a file, sounds alarms if necessary, adds it to the UI
(charts and tables), and then loops to acquire more. This results in
data acquisition times of several seconds, and we need it under a
second. The data acquisition is more important than UI updates, but
not at the expense of UI updates never taking place.

The first thing to understand is that introducing threading may or may
not improve performance. Usually it doesn't. If your code currently
takes several seconds to acquire data, it will still take several
seconds to do so even if you introduce threading.

The exception is for algorithms that are CPU bound and which can be
parallelized, and even then only when you have multiple CPUs in the
computer. In that case, you can have more than one thread working on
the problem simultaneously, which can get the job done faster.

That said, performance is not the only reason to use threading. A
classic example is to allow the user-interface thread to be responsive
while some other processing takes places. This appears to be applicable
to your situation, and there are implicit and explicit ways to use
threading to address that situation.

Based on the information I know, here is what I would do (but don't
know how): Thread A, which is high priority, sits there and acquires
data over and over, adding it to a Queue 1 as it acquires more. Thread
B, which is medium-to-high priority, loops over and over, and when it
sees data in Queue 1, it writes it to a file and sets any alarm
states. It then puts the data in Queue 2. Thread C, which is lower
priority, loops over and over, and uses any data in Queue 2 to update
the UI.

Is that the correct way to design a good solution for this? If so, I
am worried about a few things:
1) If data collection is high priority, and the other threads cannot
get enough CPU time, will either Queue fill up too quickly,
effectively stalling the application (i.e. how would I "slow down" the
acquisition thread if I notice the queues filling up)?

I would not mess with thread priority, at least not as an initial
solution. When you have some task that is independent of other tasks,
and which truly is lower or higher priority, then adjusting your
priority can make sense. But when the threads involved are dependent on
each other, as they are here, it makes little sense to make one thread
higher priority than another, and for the reason you note: you really do
need each of these threads to, at least on average, process the data at
the same rate, otherwise you will eventually consume all your resources
(memory in this case, though having the UI lag behind the actual
processing is probably not desirable either).

2) Because of these extra queues filling up, am I actually taking more
time to process each acquisition given the memory needs of such a
system (i.e. should I keep it a single-threaded model)?

You should not create a design that involves queues "filling up". It's
okay for there to be temporary bursts, creating some "peak" memory
usage, but on average all of the threads need to stay in reasonable step
with each other. Otherwise, the end result is consuming all of the memory.

So as long as you manage those bursts to minimize memory usage, there
shouldn't be problem. It seems likely to me that you won't need to do
anything special per se to "manage" the bursts. On a computer with a
decent amount of RAM, you should have plenty to be able to deal with
whatever buffering the i/o requires.

It seems to me that given your stated problem, _some_ kind of threading
is going to be desirable. There seem to be two different things you
want to address:

1) You want to have a responsive UI even while some longer i/o
operation is taking place
2) You want to have a thread that focuses only on data acquisition
and which doesn't waste time saving the acquired data to a file

The three-thread solution you propose should address those needs nicely.
There are other ways to address the issue, but I don't think that they
are necessarily better or worse. Just different.

3) Is there even a way to prioritize threads in such a way that Thread
A is always exactly (or rather, close to) 1 second per acquisition
run, and then Threads B and C use up the extra time, with B being most
important?

No. But if it makes sense for thread A to only do some i/o operation
once a second, you can use the Thread.Sleep() method for that purpose.

4) What happens when Thread A enqueues a new chunk of data at the same
time Thread B is dequeuing one? Same for B and C. Is this where the
"lock" keyword comes into play?

Yes. For sure, if you're going to use threads, you need to synchronize
them at some point, so that they aren't accessible the same data
structures simultaneously. What you're talking about is often called
the "producer/consumer" pattern. Any code that wants to use the queue
needs to prevent other code from accessing the queue while it's using
it. A handy class for this pattern is the Monitor class, as not only
does it provide the enter/exit semantics the lock() statement does, it
includes a wait/signal mechanism as well.

Jon Skeet has a nice chapter on threading here:
http://www.yoda.arachsys.com/csharp/threads/

I thought he had something on his web site that talked about the
producer/consumer pattern, but I can't find it at the moment. Maybe
it's buried in the above section, or maybe I'm just misremembering.

I also thought I had posted an example of a producer/consumer test, over
in the m.p.dotnet.languages.csharp newsgroup, but I can't find that either.

Obviously I am going senile. If I come across it, I'll follow-up with a
link. Otherwise, hopefully the above is enough to get you going in the
right direction.

Pete

 

[buu]

if you would build an multi thread app., I would suggest you to make your
own mts (multithreaded transaction server), a class that would hold, start,
stop, terminate and keep info for all threads.

tasks that should be in threads should be:
- threads that are started by some schedule
- time interval tasks (check space on disk, etc)
- tasks that should be faster if done with threads...

but, threads have 'critical session', parts in wich 2 or more threads could
collide (entering into common repository, updating same data, etc)...
but I think that knowing multithreads have more gains than pitfalls....

it's (from my view) important to read more about threads before making
serious app, because, for example, you could build threads like services -
two threads for one task, not one... one thread is thread you want to start,
another thread is it's listener...
so, there are more posibilities

 

[Guest]

Threading will improve performance on a single processor machine if your
threads are working on disparate resources. For example you may have an
application that encrypts, compresses and sends files to a server. You would
get a significant performance boost if you have one thread encrypting, and
compressing the files, and another thread sending those files over the
network, because you wouldn't wait for the network transfer to occur before
you start on the next file. I've noticed that in your problem you're reading
from the hard drive and processing the data, so you should see a performance
boost, becuase you can have the cpu processing the data while your reading
more data in. That being said I don't know if you'll see the performance
boost you're looking for.

 

[Peter Duniho]

[...]
I also thought I had posted an example of a producer/consumer test, over
in the m.p.dotnet.languages.csharp newsgroup, but I can't find that either.

Obviously I am going senile. If I come across it, I'll follow-up with a
link. Otherwise, hopefully the above is enough to get you going in the
right direction.

Okay, I found it:

http://groups.google.com/group/micr...ages.csharp/msg/1777b0073d421296?dmode=source

I had difficult finding it mainly because I forgot why I wrote it.
While it uses a producer/consumer pattern, I was actually trying to
demonstrate something else (ensuring sequence of background task
reporting). But hopefully it's simple enough to see the basic
producer/consumer queue locking idea as well.

Pete

 

[Dave]

The first thing to understand is that introducing threading may or may
not improve performance. Usually it doesn't. If your code currently
takes several seconds to acquire data, it will still take several
seconds to do so even if you introduce threading.

Currently the data acquisition only takes about 300 ms. Most of the
time is in the UI functions, hence my initial desire to separate those
out. That, and since all modern CPUs are dual-core, I was hoping to
take advantage of that. Part of the problem is that this code, of
course, is legacy that I'm having to maintain. In my redesign I'm
confident I can optimize the whole process enough to linearly fit in
under a second.

That said, performance is not the only reason to use threading. A
classic example is to allow the user-interface thread to be responsive
while some other processing takes places. This appears to be applicable
to your situation, and there are implicit and explicit ways to use
threading to address that situation.

I have used the BackgroundWorker control in the past for UI purposes.
Would this be the most appropriate model in this case? How then should
the work be divided in such a way that data is always acquired once
every second, and when BackgroundWorker can get CPU time it should
update the charts? (note that chart updating does not have to occur
every acquisition, it can occur once every two, etc, which is why I
took a non-linear look at it to save time; but, it is critical that
data is taken, and things like alarms and file output be done)

I would not mess with thread priority, at least not as an initial
solution. When you have some task that is independent of other tasks,
and which truly is lower or higher priority, then adjusting your
priority can make sense. But when the threads involved are dependent on
each other, as they are here, it makes little sense to make one thread
higher priority than another, and for the reason you note: you really do
need each of these threads to, at least on average, process the data at
the same rate, otherwise you will eventually consume all your resources
(memory in this case, though having the UI lag behind the actual
processing is probably not desirable either).

Sounds about right. But, like before, the UI thread is the only one
that is not mission-critical. For example, if I have 5 in the queue
and none of them are an alarm state, I can display only one of them,
saving time over the procedural method. So, then, should I have the
main thread doing the mission critical things, and a BackgroundWorker
used to update UI whenever it can?

You should not create a design that involves queues "filling up". It's
okay for there to be temporary bursts, creating some "peak" memory
usage, but on average all of the threads need to stay in reasonable step
with each other. Otherwise, the end result is consuming all of the memory.

So as long as you manage those bursts to minimize memory usage, there
shouldn't be problem. It seems likely to me that you won't need to do
anything special per se to "manage" the bursts. On a computer with a
decent amount of RAM, you should have plenty to be able to deal with
whatever buffering the i/o requires.

My hope was a simple switch saying: if queue.count > 5, then set a
wait condition to the main thread while the UI thread gets its
updating finished. This way, the UI doesn't get too far behind, and we
might lose just a few seconds on that next acquisition. Or, just drop
4/5 of those queued items, and only process one, depending on the
state of the items.

It seems to me that given your stated problem, _some_ kind of threading
is going to be desirable. There seem to be two different things you
want to address:

1) You want to have a responsive UI even while some longer i/o
operation is taking place
2) You want to have a thread that focuses only on data acquisition
and which doesn't waste time saving the acquired data to a file

The three-thread solution you propose should address those needs nicely.
There are other ways to address the issue, but I don't think that they
are necessarily better or worse. Just different.

Here is basically what my goal is: Each data acquisition is as long as
we allow it to be, in chunks of 30 ms (hardware latency). But since
the data is averaged over that time, the statistics are better with
more data points. Thus, my hope was to have one thread sit there and
just keep collecting data. If the other threads happen to take 2
seconds, the end result is one fewer acquisition overall, but better
statistics on that one acquisition (a tradeoff we are fine with).

Currently with the procedural implementation, it's always 1 second of
acquisition time, regardless of the time it takes to do the other
stuff. And while the charts are updating (1.2 seconds), whatever data
would be coming in at that moment is just never taken. What we don't
want is an interval of time when data is not being taken, except for
small intervals less than half a second. So the goal would be a
constant acquisition loop, and each measurement's length may range
from 1-2 seconds depending on the other two threads. Does that clarify
things? Perhaps a queue would not be appropriate, but rather a flag
telling the acquisition loop it's ok to send the next measurement down
the pope.

How would I set up my threads to keep that happening? Will each thread
automatically attach to different cores of the CPU, or is there a way
to make that happen, at the very least for the acquisition thread?
Basically, I don't care how the file i/o and UI threads are set up, as
long as there is one thread sitting there collecting data the whole
time.

Yes. For sure, if you're going to use threads, you need to synchronize
them at some point, so that they aren't accessible the same data
structures simultaneously. What you're talking about is often called
the "producer/consumer" pattern. Any code that wants to use the queue
needs to prevent other code from accessing the queue while it's using
it. A handy class for this pattern is the Monitor class, as not only
does it provide the enter/exit semantics the lock() statement does, it
includes a wait/signal mechanism as well.

Jon Skeet has a nice chapter on threading here:http://www.yoda.arachsys.com/csharp/threads/

I also found this resource, which is fairly detailed:
http://www.albahari.com/threading/

I really appreciate your time here, it's helping me to wrap my mind
around this stuff. I guess my main question now is whether I should be
using only one thread for the acquisition loop, on its own core, and
then using a BGWorker to do the UI updates and file writing? Or does
the 3-separate thread model still apply somehow?

Thanks again,

Dave Harris

 

[Peter Duniho]

Threading will improve performance on a single processor machine if your
threads are working on disparate resources.

That's right. That would be an example of an algorithm that can be
parallelized, which I mentioned. My specific example was of one that
could do CPU-intensive work in parallel, but it applies to any situation
in which mutually independent tasks are using unrelated resources.

[...] I've noticed that in your problem you're reading
from the hard drive and processing the data, so you should see a performance
boost, becuase you can have the cpu processing the data while your reading
more data in. That being said I don't know if you'll see the performance
boost you're looking for.

It's not my problem. It's Dave's problem.

As for whether threading will improve performance, we don't have the
information to know for sure if it will. However, from his original
post it does not sound like there's any significant CPU work involved.
He's essentially got a chain of i/o, each link dependent on the
previous. Since a given link in the chain can't do anything with the
data until the previous link is done with it, threading isn't going to
improve throughput.

Which is not to say it's not potentially useful. The use of threads can
address other issues, such as allowing the UI to remain responsive.
It's just not clear from the problem description that threading is going
to help overall throughput.

Pete

 

[Peter Duniho]

[...]
I have used the BackgroundWorker control in the past for UI purposes.
Would this be the most appropriate model in this case? How then should
the work be divided in such a way that data is always acquired once
every second, and when BackgroundWorker can get CPU time it should
update the charts? (note that chart updating does not have to occur
every acquisition, it can occur once every two, etc, which is why I
took a non-linear look at it to save time; but, it is critical that
data is taken, and things like alarms and file output be done)

The BackgroundWorker thread is not really much different from any other
thread. However, it's intended for short, individual tasks (as would
any class that uses the thread pool).

Your description seems to imply on-going processing, and so as such I
would say that any thread you introduce is likely to be better
implemented simply by creating a new Thread instance explicitly and
executing your code there.

How many and how these would be designed are a separate matter. But I
wouldn't say that the BackgroundWorker class itself is appropriate.

As far as making sure "that data is always acquired once every second",
that's just a matter of timing. To some extent, you cannot guarantee
any particular timing. But you can easily implement a "once every
second" loop, and this is often possible even without threading. There
are a variety of mechanisms, but two common ones would be:

1) Use a Forms.Timer class, with a 1 second interval. This is a
single-threaded solution that does all processing in the main GUI
thread. Assuming the actual processing is short, this may be the best
solution, since it keeps the code simple.

2) Use a separate thread for data acquisition with a loop to repeat
the data acquisition operation. Measure the time it takes to do the
data acquisition (the Stopwatch class is useful for this), subtract that
time from 1 second, and use the result as the parameter for a call to
Thread.Sleep() at the end of an iteration. This thread will do its
thing once per second.

[...]
Sounds about right. But, like before, the UI thread is the only one
that is not mission-critical. For example, if I have 5 in the queue
and none of them are an alarm state, I can display only one of them,
saving time over the procedural method. So, then, should I have the
main thread doing the mission critical things, and a BackgroundWorker
used to update UI whenever it can?

See above. I don't think BackgroundWorker is necessarily applicable
here. Because the processing of the incoming data flow appears to be a
constant thing, even if it is not operating constantly (that is, it only
does the processing at specific intervals), you might as well just
create any threads necessary, and let them persist constantly, running
them as needed (using Sleep() or some event signaling mechanism to
control their execution).

[...]
Here is basically what my goal is: Each data acquisition is as long as
we allow it to be, in chunks of 30 ms (hardware latency). But since
the data is averaged over that time, the statistics are better with
more data points. Thus, my hope was to have one thread sit there and
just keep collecting data. If the other threads happen to take 2
seconds, the end result is one fewer acquisition overall, but better
statistics on that one acquisition (a tradeoff we are fine with).

To some extent, answers to your questions depend on how exactly you are
acquiring data. Just what methods do you call to actually get this
data? How best to manage a thread doing this work depends on what is
required to do the work.

Currently with the procedural implementation, it's always 1 second of
acquisition time, regardless of the time it takes to do the other
stuff. And while the charts are updating (1.2 seconds), whatever data
would be coming in at that moment is just never taken.

Why not? If you are only interested in acquiring data once per second,
and it takes only 300 ms to do the complete data acquisition and UI
update, where's the bottleneck? It seems like you've got an extra 700
ms to play with. How is it that you're missing data you want to get?

What we don't
want is an interval of time when data is not being taken, except for
small intervals less than half a second. So the goal would be a
constant acquisition loop, and each measurement's length may range
from 1-2 seconds depending on the other two threads. Does that clarify
things? Perhaps a queue would not be appropriate, but rather a flag
telling the acquisition loop it's ok to send the next measurement down
the pope.

What does it mean to have a "measurement length"? You have suggested
that you only take data samples on a period interval (e.g. 1 second),
but this suggests that you are trying to sample data as some much faster
rate (30 ms intervals, corresponding to your latency, perhaps?)

How would I set up my threads to keep that happening? Will each thread
automatically attach to different cores of the CPU, or is there a way
to make that happen, at the very least for the acquisition thread?
Basically, I don't care how the file i/o and UI threads are set up, as
long as there is one thread sitting there collecting data the whole
time.

You don't have much control over whether a thread actually gets to
execute. You can set affinity to a CPU, to ensure a given thread
executes only on a given CPU, but that's not the same as given that
thread exclusive access to the CPU.

Generally, the Windows thread scheduler will select the next thread
eligible for execution (highest-priority runnable thread), and assign it
to the currently available CPU. It does this any time a thread either
becomes unrunnable (due to waiting on i/o, for example) or has exhausted
its timeslice (and so is preempted by the scheduler).

If your data acquisition thread is the only thread that's runnable on a
regular basis, it will get the lion's share of whatever CPU time is
available. But you can't guarantee that that thread will not have to
wait for some other thread to be done with the CPU without changing its
priority. Even doing that, there are other high-priority threads on the
system that will still need to run, and even the lower-priority threads
will eventually get a temporary priority boost so that they aren't
completely starved of CPU time.

[...]
I also found this resource, which is fairly detailed:
http://www.albahari.com/threading/

Just skimmed over it. It mostly looks okay, and seems to cover a number
of the .NET-specific threading stuff. However, the fact that not once
in the entire article is the "volatile" keyword mentioned seems
less-than-good to me.

I really appreciate your time here, it's helping me to wrap my mind
around this stuff. I guess my main question now is whether I should be
using only one thread for the acquisition loop, on its own core, and
then using a BGWorker to do the UI updates and file writing? Or does
the 3-separate thread model still apply somehow?

It's hard to answer that question without knowing more specifics about
the data acquisition aspect. As I mentioned, if you really are just
checking some data once per second, and the whole operation of checking
and updating the UI takes less than a second, it's not clear to me that
you need any threads beyond the main UI thread.

On the other hand, if you are acquiring and saving a stream of data, but
want to provide "snapshots" of the data in the UI, using threads is
probably the way you want to solve it. I don't think you'd need the
BackgroundWorker class, because that's intended for smaller tasks that
have a definite termination.

How may threads you need depends on what you're really trying to do.
But, for example, if you use FileStream.BeginWrite() to save the data to
a file, and you use Control.BeginInvoke() to cause the UI to update, you
may only need the one extra thread for the data acquisition.

Having three threads is more an architectural thing than something
directly related to performance. That is, it would provide for a clear
division of labor between the code in your program. But it would also
require the use of some queuing/streaming mechanism to coordinate data
flow between the threads, adding to the potential contention between
threads for shared resources (which creates overhead).

IMHO, simple is better. If you can do it with fewer threads without
complicating the overall design, you should. If the simplest design
requires three threads, you should use that instead. Which is actually
the case is something I'm not in a position to say at the moment.

Pete

 

[Dave]

Threading will improve performance on a single processor machine if your
threads are working on disparate resources.

That's right. That would be an example of an algorithm that can be
parallelized, which I mentioned. My specific example was of one that
could do CPU-intensive work in parallel, but it applies to any situation
in which mutually independent tasks are using unrelated resources.

[...] I've noticed that in your problem you're reading
from the hard drive and processing the data, so you should see a performance
boost, becuase you can have the cpu processing the data while your reading
more data in. That being said I don't know if you'll see the performance
boost you're looking for.

It's not my problem. It's Dave's problem.

As for whether threading will improve performance, we don't have the
information to know for sure if it will. However, from his original
post it does not sound like there's any significant CPU work involved.
He's essentially got a chain of i/o, each link dependent on the
previous. Since a given link in the chain can't do anything with the
data until the previous link is done with it, threading isn't going to
improve throughput.

Which is not to say it's not potentially useful. The use of threads can
address other issues, such as allowing the UI to remain responsive.
It's just not clear from the problem description that threading is going
to help overall throughput.

UI responsiveness is a guaranteed need, so if nothing else that is
important. Of primary importance, however, is the need to not ever
miss a full second's worth of data coming in. The device only acquires
data when we tell it to, so we can't afford to have the chart updates
keep a second's worth of data from being collected, even if the end
result is more averaging of that data (we're dealing with chemical
detection, of which a second can make all the difference for
alarming). I should mention that the data is being acquired via a USB
device, not off the hard drive or any other resource I'm using. So my
thought was that with one thread tied to CPU and this device, and
another tied to filesystem work, and another tied to UI, it would
increase the overall throughput (i.e. waiting on the disk would not
factor into chart updates, and waiting on the UI to update would not
affect the time it receives data from the USB device.

Am I correct in that assessment? Even if not, and we only have a
single core the program can run on, and thus it cannot constantly
acquire data, it would at least be acquiring in some form of
continuous flow (i.e. it would not be waiting on the other two
functions to completely be done before it reads the device again),
which is good enough for what we can get. We just don't want the flow
to look like:
Reading Data: 929 ms
Writing File: 233 ms
Updating charts: 1276 ms
Reading Data: 966 ms
.. . .

See how it misses a full second or more of reading any data at all?
That is what we cannot do. Even if the acquisition loop is separated
into discrete 30ms chunks simultaneous to all the other functions,
that is fine, as long as we get samples during that dead time. So we
don't necessarily need the whole loop to occur faster (a more
expensive charting solution would likely be the big help there, maybe
some other optimizations), but we do need data being taken all the
time. I am assuming that threading can help with this. If not, perhaps
we should be looking at some sort of client-server process with two
separate executables?

Dave

 

[Peter Duniho]

[...] I should mention that the data is being acquired via a USB
device, not off the hard drive or any other resource I'm using. So my
thought was that with one thread tied to CPU and this device, and
another tied to filesystem work, and another tied to UI, it would
increase the overall throughput (i.e. waiting on the disk would not
factor into chart updates, and waiting on the UI to update would not
affect the time it receives data from the USB device.

Note: the UI is generally going to be mostly CPU-bound. So a thread
"tied to UI" is really "tied to CPU". It will compete with any other
thread also CPU-bound.

Am I correct in that assessment? Even if not, and we only have a
single core the program can run on, and thus it cannot constantly
acquire data, it would at least be acquiring in some form of
continuous flow (i.e. it would not be waiting on the other two
functions to completely be done before it reads the device again),
which is good enough for what we can get. We just don't want the flow
to look like:
Reading Data: 929 ms
Writing File: 233 ms
Updating charts: 1276 ms
Reading Data: 966 ms

On a multi-core system, there's a high likelihood that you can use
threading of some sort of allow the data acquisition thread to cooperate
with the UI update thread. That is, allow both the data acquisition
thread and the UI thread to execute simultaneously, each on a different
core.

That said, if you are using USB then I'm guessing you have some kind of
async method available to do the data acquisition i/o. This means that
you may not need any explicit threads additional to the main UI thread.
Instead, you can use a Forms.Timer to regulate the data acquisition
intervals, and use a BeginXXX style method to request data. It will
complete using a different thread, so it's still a multi-threaded
solution, but it takes advantage of the thread pool and whatever
underlying i/o mechanism is available (probably i/o completion ports,
but without knowing the specifics of your code, I can't say...even
knowing the specifics, I might not be able to say ).

.. . .

See how it misses a full second or more of reading any data at all?
That is what we cannot do. Even if the acquisition loop is separated
into discrete 30ms chunks simultaneous to all the other functions,
that is fine, as long as we get samples during that dead time. So we
don't necessarily need the whole loop to occur faster (a more
expensive charting solution would likely be the big help there, maybe
some other optimizations), but we do need data being taken all the
time. I am assuming that threading can help with this. If not, perhaps
we should be looking at some sort of client-server process with two
separate executables?

Dividing your task into two different processes is not going to be
better than using threads.

I have a more-detailed reply to your other message. I replied here just
to clarify some points; hopefully the other post provides the details
you really need to get the clarification you want.

Pete

 

[Dave]

Note: the UI is generally going to be mostly CPU-bound. So a thread
"tied to UI" is really "tied to CPU". It will compete with any other
thread also CPU-bound.

Good point. Maybe that affirms it's best to at least separate out the
file writing, that may be the most optimization speed-wise I'll be
able to do.

On a multi-core system, there's a high likelihood that you can use
threading of some sort of allow the data acquisition thread to cooperate
with the UI update thread. That is, allow both the data acquisition
thread and the UI thread to execute simultaneously, each on a different
core.

That's pretty much what I'm shooting for, but there will be cases it
runs on one core, but will still need to acquire data while UI is
updating. I guess threads will solve both situations.

That said, if you are using USB then I'm guessing you have some kind of
async method available to do the data acquisition i/o. This means that
you may not need any explicit threads additional to the main UI thread.
Instead, you can use a Forms.Timer to regulate the data acquisition
intervals, and use a BeginXXX style method to request data. It will
complete using a different thread, so it's still a multi-threaded
solution, but it takes advantage of the thread pool and whatever
underlying i/o mechanism is available (probably i/o completion ports,
but without knowing the specifics of your code, I can't say...even
knowing the specifics, I might not be able to say ).

Unfortunately, there is no async. The only interface we have to the
device is a single call to an OEM DLL, which takes a single
acquisition (30ms) and returns the data as a double[2048]. Thus, any
async methods we have to develop ourselves. Right now there is a loop
for acquiring, set up like this:
while(DateTime.now < NextAcqEndDate)
addToMainArray(OEM_DLL.acquireData());

So, for 1 second acquisitions, this will keep adding the result to an
array, and at the end of this loop it averages the array over number
of counts (the number of times acquireData() is called), and returns
it. acquireData() takes 30ms, and that's the portion we would like to
keep going. As you can see, in a linear algorithm, this thing runs for
1 second, stops acquiring, and finishes all the other updates. Due to
the nature of chemical detection, we'd obviously like to keep this
going as long as possible, even if it has to call acquireData() in
discrete chunks, so long as the chunks are not 1s gaps like now.

I have never used the BeginInvoke and BeginWrite methods, so that
alone may offer a good start to this problem.

I have a more-detailed reply to your other message. I replied here just
to clarify some points; hopefully the other post provides the details
you really need to get the clarification you want.

Yes, the other message helps a lot too... I may take a few days to
digest this; luckily this is long-term. I really appreciate your input
here, it's not the easiest subject to just jump into.

Dave

 

[Peter Duniho]

[...]
That's pretty much what I'm shooting for, but there will be cases it
runs on one core, but will still need to acquire data while UI is
updating. I guess threads will solve both situations.

Yes. Threading can solve architectural issues, performance issues, or both.

Unfortunately, there is no async. The only interface we have to the
device is a single call to an OEM DLL, which takes a single
acquisition (30ms) and returns the data as a double[2048]. Thus, any
async methods we have to develop ourselves. Right now there is a loop
for acquiring, set up like this:
while(DateTime.now < NextAcqEndDate)
addToMainArray(OEM_DLL.acquireData());

So, for 1 second acquisitions, this will keep adding the result to an
array, and at the end of this loop it averages the array over number
of counts (the number of times acquireData() is called), and returns
it.

What does it do with the average? What else is done with the array? Is
that what you write to the file?

As far as the average goes, it seems to me that it would be better to
calculate the sum for the average as you get the data. The addition is
a trivial operation, and it will reduce the number of times you
enumerate the array by one.

Of course, if _all_ you're doing is calculating the average then
obviously you don't even need the array. So I'm taking as granted that
you do something else with that data besides calculating the average.

acquireData() takes 30ms, and that's the portion we would like to
keep going. As you can see, in a linear algorithm, this thing runs for
1 second, stops acquiring, and finishes all the other updates. Due to
the nature of chemical detection, we'd obviously like to keep this
going as long as possible, even if it has to call acquireData() in
discrete chunks, so long as the chunks are not 1s gaps like now.

I have never used the BeginInvoke and BeginWrite methods, so that
alone may offer a good start to this problem.

Based on your description, it sounds as though at the least you should
be able to use BeginInvoke() rather than having a separate thread for
dealing with the UI. BeginInvoke() would pass whatever data is useful
for updating the UI, and then the actual work would wind up occuring on
the UI thread itself.

Whether BeginWrite would be helpful or not, I'm not entirely sure. I
just noticed in the docs that it says that for writes shorter than 64KB,
using BeginWrite could actually reduce performance.

I suspect that this applies only to the i/o performance itself and so
assuming you're not pushing the limits of the disk bandwidth (and if
you're only generating 8K of data per second, you shouldn't be ) the
architectural gain should be well worth any minimal performance
reduction (noting that the performance reduction wouldn't affect the
data acquisition, just how fast the data gets to the disk).

If you are looking to support extremely high data rates though, that
issue might be a consideration. If so, you'd either want to batch up
your data so that you're writing 64KB or more at a time, or use a
separate thread to handle the file i/o (as in the original conception in
this discussion thread). It doesn't sound like this would be an issue,
but I mention it just in case.

Pete

 

[Dave]

What does it do with the average? What else is done with the array? Is
that what you write to the file?

As far as the average goes, it seems to me that it would be better to
calculate the sum for the average as you get the data. The addition is
a trivial operation, and it will reduce the number of times you
enumerate the array by one.

Of course, if _all_ you're doing is calculating the average then
obviously you don't even need the array. So I'm taking as granted that
you do something else with that data besides calculating the average.

When I saw "average", I mean each individual point in the data array.
This is why more iterations of data acquisition yield better
statistics. So the entire 2048 elements are iterated every 30ms
acquisition to sum it, and then once more at the end to average each
element by time. Once we get this average array, we send it to a C dll
that does some tricky processing and returns some data we use to
determine alarm states (detection of a chemical compound). That data
is what we write to each file, along with the averaged array. Once the
file is written, we update the UI with two charts, one being the
averaged array, and another representing the data returned from that
special DLL.

Based on your description, it sounds as though at the least you should
be able to use BeginInvoke() rather than having a separate thread for
dealing with the UI. BeginInvoke() would pass whatever data is useful
for updating the UI, and then the actual work would wind up occuring on
the UI thread itself.

So perhaps instead of setting up a secondary queue for UI operations,
I can simply call BeginInvoke with only the data it needs to update
the charts/tables?

Whether BeginWrite would be helpful or not, I'm not entirely sure. I
just noticed in the docs that it says that for writes shorter than 64KB,
using BeginWrite could actually reduce performance.

I suspect that this applies only to the i/o performance itself and so
assuming you're not pushing the limits of the disk bandwidth (and if
you're only generating 8K of data per second, you shouldn't be ) the
architectural gain should be well worth any minimal performance
reduction (noting that the performance reduction wouldn't affect the
data acquisition, just how fast the data gets to the disk).

We're not really pushing the limits of the disk bandwidth, and the
entire write takes about 200ms, so I'm not really looking to gain
performance there in terms of the I/O operation itself. But, I figure
that by putting it on a separate thread, it will at least be allowing
another thread to have some CPU time while it's doing disk I/O, so
there might be a small performance gain from that (maybe not
noticeable), compared to the system now where it sits there waiting
for it to happen. Our files wind up being 72KB on the disk, so they
are right at that crossover of performance. So, I think it might be
good to stick with the original plan of a separate thread for disk I/O
even if it's just a few lines of code executing each time.

Right now our biggest problem is obviously the charting, so perhaps
starting with one thread for everything else and using the form's
thread via BeginInvoke would be the best place to start, and go from
there as needed.

Thanks again,
Dave

 

[Peter Duniho]

[...]
So perhaps instead of setting up a secondary queue for UI operations,
I can simply call BeginInvoke with only the data it needs to update
the charts/tables?

Yes. That's the simplest way to pass data to the UI thread from another
thread without messing around with additional synchronization objects.

You'll want to take care that the data you pass is either a copy of the
actual data, or is a reference to the data that the originating thread
won't be touching again (or at least not until it knows the UI thread is
done with it).

In other words, you avoid some synchronization issues, but not all.

We're not really pushing the limits of the disk bandwidth, and the
entire write takes about 200ms, so I'm not really looking to gain
performance there in terms of the I/O operation itself. But, I figure
that by putting it on a separate thread, it will at least be allowing
another thread to have some CPU time while it's doing disk I/O, so
there might be a small performance gain from that (maybe not
noticeable), compared to the system now where it sits there waiting
for it to happen.

Well, certainly there is the possibility. The thing I don't actually
know is how the i/o stuff is implemented "under the hood". The docs say
that a write to a file stream completes synchronously for small writes
(less than 64KB). Now, this could mean one of two things: either for
those small writes, the method always blocks until it's done writing;
or, the underlying i/o implementation always immediately buffers small
writes and returns right away.

If the latter, then there really is no point at all to using the
asynchronous API, since presumably you can call the synchronous version
and still not have to wait for the actual i/o.

I tend to suspect it's the former, because of the question of dealing
with errors. That is, the only way I see for it to literally be true
that the i/o is completed synchronously even when using the asynchronous
API is for the whole operation to complete so that the complete status
of the operation is known. Just buffering the data doesn't accomplish it.

However, it could just be that the docs are imprecisely describing
what's going on, or are just plain missing some important information.

Our files wind up being 72KB on the disk, so they
are right at that crossover of performance. So, I think it might be
good to stick with the original plan of a separate thread for disk I/O
even if it's just a few lines of code executing each time.

Right now our biggest problem is obviously the charting, so perhaps
starting with one thread for everything else and using the form's
thread via BeginInvoke would be the best place to start, and go from
there as needed.

I would say that because of the uncertainty with respect to the exact
behavior of the async i/o, that's a good plan. And if the i/o itself
does turn out to be something you want to make asynchronous, do keep in
mind the issue of small vs. large writes; you may find that buffering
your own output to ensure writes larger than 64K is required in order to
obtain a performance benefit (or it may not be...it depends on the open
question I don't have the answer to mentioned above). But even there,
as a first pass I would just switch to the async API without any fancy
stuff, and only switch to something more complicated if I didn't get an
improvement I expected.

Pete

 

[cubaman]

Ok, I appreciate any help you can give me. I am somewhat new to
threading, only understanding it conceptually, and I use the Threading
namespace for the sleep() command. So, any suggestions with code would
also be beneficial.

Basically, I need help designing how my threads should interact.
Currently, the application acquires a bit of data, processes it,
writes it to a file, sounds alarms if necessary, adds it to the UI
(charts and tables), and then loops to acquire more. This results in
data acquisition times of several seconds, and we need it under a
second. The data acquisition is more important than UI updates, but
not at the expense of UI updates never taking place.

Based on the information I know, here is what I would do (but don't
know how): Thread A, which is high priority, sits there and acquires
data over and over, adding it to a Queue 1 as it acquires more. Thread
B, which is medium-to-high priority, loops over and over, and when it
sees data in Queue 1, it writes it to a file and sets any alarm
states. It then puts the data in Queue 2. Thread C, which is lower
priority, loops over and over, and uses any data in Queue 2 to update
the UI.

Is that the correct way to design a good solution for this? If so, I
am worried about a few things:
1) If data collection is high priority, and the other threads cannot
get enough CPU time, will either Queue fill up too quickly,
effectively stalling the application (i.e. how would I "slow down" the
acquisition thread if I notice the queues filling up)?
2) Because of these extra queues filling up, am I actually taking more
time to process each acquisition given the memory needs of such a
system (i.e. should I keep it a single-threaded model)?
3) Is there even a way to prioritize threads in such a way that Thread
A is always exactly (or rather, close to) 1 second per acquisition
run, and then Threads B and C use up the extra time, with B being most
important?
4) What happens when Thread A enqueues a new chunk of data at the same
time Thread B is dequeuing one? Same for B and C. Is this where the
"lock" keyword comes into play?

Thanks in advance, and I hope someone here can shed some light on how
to accomplish this...

Dave Harris

Well, it's better to teach how to get fishes than givin it
Here is a couple of links that may be usefull
http://blogs.msdn.com/csharpfaq/archive/2004/03/17/91685.aspx
http://www.albahari.com/threading/
http://research.microsoft.com/~birrell/papers/ThreadsCSharp.pdf
There are the answer to your questions, best regards
Oscar Acosta