Asynchronous calls to a web service - 100,000 a minute

[kebalex]

Hi,

I've got a requirement to poll a web service for data but it needs to
be done very frequently. The web service call is straight forward,
it
just returns a simple table of data with no more than 50 rows. It's
a
third party web service so i have no control over it. The problem is
each call is on a per user basis and the client said if its succesful
it there could be 100,000 users. The frequency of each call could be
once a minute so that could be 100,000 calls a minute.


I'm designing the client app to make the requests and i'm thinking
each
request should be asynchronous but i'm not sure this is enough. I've
got
no experience of multi-threaded development so i'm not sure if this
would help (making the requests over many threads).


Also I'm concerned about bottlenecks. If the app I built was capable
of making 100000 requests a minute (possibly at the same time if its
multi-threaded) then would the network handle this load and also
could
the web service handle this many requests at once (i know the W3C
recommends only 2 client connections at once so that could be an
issue).


Anyway, any help or thoughts would be much apprecieated.


Thanks,
Alex

 

[Robbe Morris - [MVP] C#]

Does each user have their own single copy of the client app?

Aside from making each copy's user interface a little more
responsive while it waits for the web service response, it really
has very little to do with server load. The server is the bottleneck,
not your client app. Multi-threaded process on your client app
may cause more problems than it solves.

Are you sure your 3rd party web service provider is ready
for that much traffic?

 

[Norman Yuan]

It sounds almost like a "Denial-of-sevice attack". You stated you have no
control on the web service, but you certainly need to talk to the service
provider to see if they can handle such request load (a few thousands per
second).

 

[Jeremy]

How often does the data in the table change? Even if the clients poll once
a minute would they be getting new results each time?

I ask because it may be worth your while to make a "proxy web service".
Once that your clients can call, which would in turn call the real web
service but cache the data for 1 or so minutes. Only when the cache expires
would the proxy web service actually call the real web service and update
the cache. That way the real web service only gets called 1 time every
minute and there's only a database hit 1 time every minute. Your proxy web
service would get hit a lot but 99% of the time it would be dishing out
cached data, making it able to serve more hits. The bottle neck now becomes
your proxy web service, but if you found the load on that web service was
too high, you could put it on more than one server, splitting the load up.

 

[Peter Duniho]

[...]
I'm designing the client app to make the requests and i'm thinking
each
request should be asynchronous but i'm not sure this is enough. I've
got
no experience of multi-threaded development so i'm not sure if this
would help (making the requests over many threads).

I would use the async APIs available (either Socket or HttpWebRequest
class, using the Begin/EndXXX methods). These will use i/o completion
ports, which implicitly use a thread pool but which don't require you to
implement threading explicitly. Use of i/o completion ports is also
required for the most scalable i/o code.

Now, use of i/o completion ports (and in .NET, implicitly via the async
API) doesn't in and of itself guarantee scalability. There can be other
problems in the code that reduce performance. And conversely, depending
on the code it may not be necessary to use i/o completion ports in order
to achieve high bandwidth.

In particular, if you've got some well-defined small amount of data to
just send repeatedly on a variety of connections, it's possible you
could handle that just fine in a single thread. Network i/o is going to
be a LOT slower than your CPU bandwidth, and once you've completely
saturated your network connection, it doesn't matter how efficient the
rest of the code is, it can't go any faster.

That said:

Also I'm concerned about bottlenecks. If the app I built was capable
of making 100000 requests a minute (possibly at the same time if its
multi-threaded) then would the network handle this load and also
could
the web service handle this many requests at once (i know the W3C
recommends only 2 client connections at once so that could be an
issue).

It seems to me that your concerns are valid, at least to some extent.

As far as the network traffic goes, that really just depends on your
network. If you're doing this testing over a LAN, there's probably no
problem. For that matter, if you're doing it over an Internet
connection and the requests are very small, there's probably no problem.

But sure, if you're sending 100K of something out in a minute, that
something needs to be small enough to fit in your network bandwidth.
Hopefully that's the case here, but you're not specific about the size
of the requests, so it's hard to answer with a specific yes or no. As a
rough estimate, I calculate that on a relatively slow connection
(128Kbps, early-adopter cable modem upload speed), you can only do 100K
8-byte items in 60 seconds. Even a 1Mbps upload speed only gets you to
64-byte requests.

So it all depends, on your network speed and the size of the requests
(and those calculations are just rough estimates...actual network
performance and overhead does vary).

As far as the web service itself, I would be concerned as to whether you
are actually measuring what you think you're measuring. Depending on
the implementation, it could handle 100K requests from the same client
better than it can handle 100K requests from thousands of clients. So
at the very least, you may want to make sure that your client is
configured so that it _looks_ like multiple clients, from the web
service's point of view.

This sort of testing _is_ done, of course. So it's certainly possible.
You just need to be aware of the requirements for achieving the server
load you're looking for.

Pete

 

[Guest]

Thanks for all the replies. The app will be a windows service that simply
saves the data in a local SQL server. The data changes very frequently - its
call log data from a VOIP phone system and the offices that use them are on
the phone all day! i think the problem is how the web service has been
designed - i can only bring back the last 50 entries on a per user (phone)
basis so if i didn't poll frequently enough then i would miss some call logs.
This is unacceptable as the data is needed for all kind of management
reports as well as other areas of a web application.

I think i'll speak to the web service provider to see if i can get them to
add some new API's. I think they might if they knew i was writing a DOS app!!

Thanks again for all you help,
Alex