UDP server scalability question

[andrew queisser]

I've got the following situation and I'm wondering about scalability of
different solutions:

1) Client sends a request to a fixed port on a UDP server
2) UDP server sends an ACK and launches the request (worker thread)
3) UDP server sends response data back to client
4) Client sends ACK back to server

My question is about how to distribute the work among the main server and
the worker threads. My initial solution is to have the client send
everything to the fixed UDP port and then have the main server thread filter
out duplicate requests and duplicate ACKs. The server thread also sends the
response from step 2). The worker thread would only send the response back
to the client via a socket that each worker thread opens up upon creation.

However, I could also have the worker thread execute steps 2), 3) and
receive the ACK from step 4). That would relieve some load from the main
server thread, especially matching up the ACK with the thread that's
handling the request.

On the other hand, everything has to go through one network interface
anyway, so is there some advantage of structuring the code one way or the
other?

Also, does it make sense to use asynchronous BeginReceiveFrom with UDP in
order to get back to reading the socket or is the buffering on the incoming
UDP socket flexible enough to buffer out reasonable delays in the main
thread?

Thanks,
Andrew

 

[Cowboy \(Gregory A. Beamer\)]

First, what is the load you are expecting? Unless it is rather high,
offloading work is just extra code. And, spinning up threads can be
problematic, esp. if more than one thread can contend for the same resource.
Best to keep it simple until you need the extra

Second, why are you setting up an ACK on UDP? If simply to give something
back and then work, there is not an issue. If you are attempting to add
reliability ... that is another story. UDP is meant to be a fast, yet
unreliable, protocol. It is best for streaming and other types of uses that
do not require reliability of every packet. If you ACKing is designed to add
reliability, you are better served using TCP, as it is a reliable transport.
Why create your own reliability mechanism, which is likely to be slow, when
TCP has one built in for you?

If this is all the server does, spawning multiple threads will most likely
just add complexity to the application. Multiple threads are useful when
there is long running work that can be done in parallel.

As for the BeginReceive versus buffer, a buffer should slow down delays as
long as you are not attempting to introduce reliability. If the client
continues to ask for packets, you will have a bottleneck. The asynch comes
into play more when you are dealing with spawning up threads, which I am not
fully convinced you need to do.

Many of the questions you have asked require additional details, as UDP,
itself, is not a given, esp. if you wish a reliable transport (sounding like
a broken record? ). Depending on how quickly you start streaming out
bytes, the ACK may be extra work that is unnecessary (getting bytes back is
an ACK, of sorts ;->). Unless the ACK is giving data (what the client should
expect, for example), which makes it more than a simple ACK, it may simply
add more steps to the process. I also question why the client needs to ACK
back, except for reliability (yeah, broken record), which is better served
with TCP.

--
Gregory A. Beamer

*************************************************
Think Outside the Box!
*************************************************

 

[andrew queisser]

Hi Gregory,

You're pretty much confirming what I'm suspecting:

First, what is the load you are expecting? Unless it is rather high,
offloading work is just extra code. And, spinning up threads can be
problematic, esp. if more than one thread can contend for the same
resource. Best to keep it simple until you need the extra

Load is fairly low, there may be a few hundred clients but only a few
packets per second. The worker threads handle backend transactions to a
large data system that we don't know much about.

Second, why are you setting up an ACK on UDP? If simply to give something
back and then work, there is not an issue. If you are attempting to add
reliability ... that is another story. UDP is meant to be a fast, yet
unreliable, protocol. It is best for streaming and other types of uses
that do not require reliability of every packet. If you ACKing is designed
to add reliability, you are better served using TCP, as it is a reliable
transport. Why create your own reliability mechanism, which is likely to
be slow, when TCP has one built in for you?

Good point and one that we've discussed at length. Yes, ACK is for
reliability and experience shows that TCP connection initiation and teardown
overhead can lead to problems since this is a harsh wireless environment.
The payloads on each packet are very low and there are no overlapping
transactions so request/ACK/response/ACK is extremely easy to implement.

If this is all the server does, spawning multiple threads will most likely
just add complexity to the application. Multiple threads are useful when
there is long running work that can be done in parallel.

The server issues a lengthy transaction (e.g. 1 second duration) that
produces the response data. The devices are completely independent so any
number of transactions, only limited by the number of threads in my server
application, may be active at one time. My thought was to either use
ThreadPool or manually creating a number of threads that are blocked on a
synchronization object.

As for the BeginReceive versus buffer, a buffer should slow down delays as
long as you are not attempting to introduce reliability. If the client
continues to ask for packets, you will have a bottleneck. The asynch comes
into play more when you are dealing with spawning up threads, which I am
not fully convinced you need to do.

Yeah, I'm still unclear on the value of BeginReceiveFrom. Since my main
server thread will have to do minimal checking of incoming requests anyway
it seems like a blocking read in the main thread followed by dispatching of
the transaction work to a worker is more straightforward and possibly more
efficient.

Many of the questions you have asked require additional details, as UDP,
itself, is not a given, esp. if you wish a reliable transport (sounding
like a broken record? ). Depending on how quickly you start streaming
out bytes, the ACK may be extra work that is unnecessary (getting bytes
back is an ACK, of sorts ;->). Unless the ACK is giving data (what the
client should expect, for example), which makes it more than a simple ACK,
it may simply add more steps to the process. I also question why the
client needs to ACK back, except for reliability (yeah, broken record),
which is better served with TCP.

The ACK will probably carry additional status data but I am considering
using the response data as an implicit ACK. Implicit ACK makes the
architecture a lot more complex, though, since I have to decide whether the
response comes fast enough or whether a plain ACK is needed. Then I have to
handle the race between the timer and the worker thread although that should
be doable by having a timed wait in the worker thread. Hmmm, getting some
ideas here while I'm typing...

Thanks for your comments,
Andrew

 

[Vadym Stetsyak]

Hello, andrew!

Take a look about for reliable network I/O via UDP

http://msdn.microsoft.com/msdnmag/issues/06/02/UDP/default.aspx

--
Regards, Vadym Stetsyak
www: http://vadmyst.blogspot.com

 

[andrew queisser]

Thanks, I read that article when it came out and I got a lot out of it.