Threading and IPC

  • Thread starter Thread starter SRLoka
  • Start date Start date
S

SRLoka

I am developing a C# windows service that gets a continuous stream of bytes
via TCP. As soon as it finds the delimitters(that we have defined), it needs
to pass the data to another thread for processing. The receiving thread does
not care what happend to the data(no thread sync needed). I am thinking of 2
approaches
1) Have a parser class with a constructor and create a new class(passing the
data as parameter) every time there is data and start a new thread that will
process the data and terminate.
2) Have a global queue(FIFO) and just add the data to the Q. The parser
thread is continuously running in a loop(probably with some sleep time)
checking the Q.

Which is the better method and why ?

Thank You for any suggestions or links.

SLoka
 
SRLoka said:
I am developing a C# windows service that gets a continuous stream of bytes
via TCP. As soon as it finds the delimitters(that we have defined), it needs
to pass the data to another thread for processing. The receiving thread does
not care what happend to the data(no thread sync needed). I am thinking of 2
approaches
1) Have a parser class with a constructor and create a new class(passing the
data as parameter) every time there is data and start a new thread that will
process the data and terminate.
2) Have a global queue(FIFO) and just add the data to the Q. The parser
thread is continuously running in a loop(probably with some sleep time)
checking the Q.
Which is the better method and why ?
Thank You for any suggestions or links.
SLoka
Click to expand...

Not 1. Better to create a processing-thread once and let it wait until it
has something to do. It is too expensive (cpu-time) to create a thread each
time.

I would take the Q. The parser thread should not poll nor sleep. The thread
should wait until work has arrived. Check ManualResetEvent and WaitHandle
classes for signaling and waiting.
 
If single cpu, you probably don't need the second thread and both threads
will only contend for cpu (assuming thread2 is not blocking on output io)
If multiple cpu's then I would use a blocking circular queue. This is a
classic Producer-Consumer example. Your thread1 is the producer that fills
a fixed size blocking queue. When the q is full, it blocks. Your thread2
is the consumer, it keeps poping objects off the queue until empty and
blocks for more data or until EOQ object (or other such way to signal
thread2 to stop) is found in the queue.
 
Thank You both for your feedback.
Looks like Q is the way to go. Is there any particular reason I should use
fixed size queues only ?
For my situation, this will be running on a dual CPU machine. The TCP stream
is coming from a third party application which cannot be blocked for too
long. Since these are just simple strings in queue, memory may not be an
issue(if thats the reason for fixed Q). My guess is, the maximum number of
messages we get in a second might be a 1000. There are small strings about
100 to 200 bytes. So I can use a Q with 2000 items to start with, with a
growFactor of 1.1
Is there anyway to control thread/application priorities on a Win2K server
?

Thanks
 
If you don't use a fixed size blocking queue, then a producer could just
fill all available ram with objects. This also does not help balance the
load either. Pick something reasonable. Some simple testing I did awhile
ago showed me 30-40 was about right for running producer-consumer. More
then that was just wasting slots. The .net queue class is not the best for
this as it is not syncronized. You can use the sync wrapper, but that is
not tuned for this work (but would work.) It also grows and does not block.
Blocking is good in this case.
 
Back
Top