Bug in ThreadPool or HttpWebRequest?

[Dan Battagin]

Is there a known bug with the interaction between the HttpWebRequest and the
ThreadPool? I current spawn several HttpWebRequest's using
BeginGetResponse, and they work for a while, using worker threads from the
ThreadPool. However, eventually (relatively quickly) there become fewer and
fewer available worker threads in the pool, until there are 0 and a
System.InvalidOperationException occurs with the message: "There were not
enough free threads in the ThreadPool object to complete the operation."

It's as if the worker threads are never released by the HttpWebRequest,
which seems like a pretty serious bug.

Oh, and so far I only repro this on Win9x (ME to be precise, haven't tried
98 yet). The worker threads seem to be released fine by all of the NT
flavors.

Below I am including 2 things:
(a) The methods that work with the HttpWebRequest object
(b) The result of calling ThreadPool.GetAvailableThreads(out int
workerThreads, out int completionPortThreads) as I create each
HttpWebRequest (until the number of workers hits 0 at which point an
exception is thrown)

Thanks,
Dan

(a) basically I call these in a loop, such that no more than 8 are active
at any given time

public void BeginScrapeContent(ScraperConfig config) {
if (config == null) {
if (_config == null) {
throw new ArgumentNullException("config", "config or _config can not
be null.");
}
} else {
this._config = config;
}
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(this.Url);
request.BeginGetResponse(new AsyncCallback(EndRequestContent1), request);
}

private void EndRequestContent1(IAsyncResult ar) {
HttpWebRequest request = null;
try {
request = (HttpWebRequest)ar.AsyncState;
request.EndGetResponse(ar);
_content += new
StreamReader(request.GetResponse().GetResponseStream()).ReadToEnd();
_contentRetrievalSuccess1 = true;
} catch (Exception ex) {
_contentRetrievalSuccess1 = false;
Debug.WriteLine(ex.Message);
_failReason = ex is WebException ? ContentRetrievalFailure.NoResponse :
ContentRetrievalFailure.BadResponse;
} finally {
try {
request.GetResponse().Close();
} catch {
Debug.WriteLine("Error calling close");
}
}
}

(b) Here's the result, of in that loop where I call the BeginScrapeContent
method), checking ThreadPool.GetAvailableThreads each time I perform a new
BeginScrapeContent call (again, there are ever only a max of 8 of these at
any time). As you can see the number of worker threads drops to 0 quite
quickly.

wrkThrds: 25 cpThrds: 25
wrkThrds: 25 cpThrds: 25
wrkThrds: 24 cpThrds: 25
wrkThrds: 24 cpThrds: 25
wrkThrds: 23 cpThrds: 25
wrkThrds: 21 cpThrds: 25
wrkThrds: 19 cpThrds: 25
wrkThrds: 19 cpThrds: 25
wrkThrds: 19 cpThrds: 25
wrkThrds: 17 cpThrds: 25
wrkThrds: 17 cpThrds: 25
wrkThrds: 19 cpThrds: 25
wrkThrds: 17 cpThrds: 25
wrkThrds: 17 cpThrds: 25
wrkThrds: 18 cpThrds: 25
wrkThrds: 17 cpThrds: 25
wrkThrds: 18 cpThrds: 25
wrkThrds: 18 cpThrds: 25
wrkThrds: 17 cpThrds: 25
wrkThrds: 17 cpThrds: 25
wrkThrds: 18 cpThrds: 25
wrkThrds: 17 cpThrds: 25
wrkThrds: 18 cpThrds: 25
wrkThrds: 17 cpThrds: 25
wrkThrds: 17 cpThrds: 25
wrkThrds: 18 cpThrds: 25
wrkThrds: 16 cpThrds: 25
wrkThrds: 17 cpThrds: 25
wrkThrds: 15 cpThrds: 25
wrkThrds: 16 cpThrds: 25
wrkThrds: 16 cpThrds: 25
wrkThrds: 13 cpThrds: 25
wrkThrds: 13 cpThrds: 25
wrkThrds: 12 cpThrds: 25
wrkThrds: 12 cpThrds: 25
wrkThrds: 12 cpThrds: 25
wrkThrds: 12 cpThrds: 25
wrkThrds: 11 cpThrds: 25
wrkThrds: 12 cpThrds: 25
wrkThrds: 12 cpThrds: 25
wrkThrds: 11 cpThrds: 25
wrkThrds: 10 cpThrds: 25
wrkThrds: 9 cpThrds: 25
wrkThrds: 8 cpThrds: 25
wrkThrds: 9 cpThrds: 25
wrkThrds: 9 cpThrds: 25
wrkThrds: 8 cpThrds: 25
wrkThrds: 8 cpThrds: 25
wrkThrds: 7 cpThrds: 25
wrkThrds: 12 cpThrds: 25
wrkThrds: 10 cpThrds: 25
wrkThrds: 10 cpThrds: 25
wrkThrds: 7 cpThrds: 25
wrkThrds: 9 cpThrds: 25
wrkThrds: 8 cpThrds: 25
wrkThrds: 11 cpThrds: 25
wrkThrds: 13 cpThrds: 25
wrkThrds: 13 cpThrds: 25
wrkThrds: 11 cpThrds: 25
wrkThrds: 10 cpThrds: 25
wrkThrds: 11 cpThrds: 25
wrkThrds: 9 cpThrds: 25
wrkThrds: 12 cpThrds: 25
wrkThrds: 11 cpThrds: 25
wrkThrds: 10 cpThrds: 25
wrkThrds: 8 cpThrds: 25
wrkThrds: 7 cpThrds: 25
wrkThrds: 7 cpThrds: 25
wrkThrds: 8 cpThrds: 25
wrkThrds: 8 cpThrds: 25
wrkThrds: 8 cpThrds: 25
wrkThrds: 10 cpThrds: 25
wrkThrds: 14 cpThrds: 25
wrkThrds: 18 cpThrds: 25
wrkThrds: 19 cpThrds: 25
wrkThrds: 17 cpThrds: 25
wrkThrds: 17 cpThrds: 25
wrkThrds: 17 cpThrds: 25
wrkThrds: 15 cpThrds: 25
wrkThrds: 15 cpThrds: 25
wrkThrds: 14 cpThrds: 25
wrkThrds: 14 cpThrds: 25
wrkThrds: 14 cpThrds: 25
wrkThrds: 14 cpThrds: 25
wrkThrds: 11 cpThrds: 25
wrkThrds: 10 cpThrds: 25
wrkThrds: 10 cpThrds: 25
wrkThrds: 10 cpThrds: 25
wrkThrds: 11 cpThrds: 25
wrkThrds: 11 cpThrds: 25
wrkThrds: 11 cpThrds: 25
wrkThrds: 10 cpThrds: 25
wrkThrds: 10 cpThrds: 25
wrkThrds: 8 cpThrds: 25
wrkThrds: 9 cpThrds: 25
wrkThrds: 10 cpThrds: 25
wrkThrds: 5 cpThrds: 25
wrkThrds: 5 cpThrds: 25
wrkThrds: 8 cpThrds: 25
wrkThrds: 7 cpThrds: 25
wrkThrds: 7 cpThrds: 25
wrkThrds: 5 cpThrds: 25
wrkThrds: 6 cpThrds: 25
wrkThrds: 5 cpThrds: 25
wrkThrds: 4 cpThrds: 25
wrkThrds: 3 cpThrds: 25
wrkThrds: 3 cpThrds: 25
wrkThrds: 1 cpThrds: 25
wrkThrds: 0 cpThrds: 25

Exception Type: System.InvalidOperationException
Message: There were not enough free threads in the ThreadPool object to
complete the operation.
Stack Trace:
at System.Net.HttpWebRequest.BeginGetResponse(AsyncCallback callback,
Object state)
at TestApp.Retriever.BeginScrapeContent(ScraperConfig config)
at TestApp.Form1.RetrieveContentStart()
Source: System
Target Site: System.IAsyncResult BeginGetResponse(System.AsyncCallback,
System.Object)

 

[John Timney \(Microsoft MVP\)]

Your not the first to suffer from this bug.

You will need to manually count your invocations of threads and web
requests. Each web request will consume one thread from the pool, so each
method will use two pool calls. When the threaded method terminates deduct
two from your counter, one for the web request and one for the thread
method.

--
Regards

John Timney (Microsoft ASP.NET MVP)
----------------------------------------------
<shameless_author_plug>
Professional .NET for Java Developers with C#
ISBN:1-861007-91-4
Professional Windows Forms
ISBN: 1861005547
Professional JSP 2nd Edition
ISBN: 1861004958
Professional JSP
ISBN: 1861003625
Beginning JSP Web Development
ISBN: 1861002092
</shameless_author_plug>
----------------------------------------------

 

[Willy Denoyette [MVP]]

This is by design, you should handle the exception See Q815637.

Willy.

Dan Battagin wrote:
|| Is there a known bug with the interaction between the HttpWebRequest
|| and the ThreadPool? I current spawn several HttpWebRequest's using
|| BeginGetResponse, and they work for a while, using worker threads
|| from the ThreadPool. However, eventually (relatively quickly) there
|| become fewer and fewer available worker threads in the pool, until
|| there are 0 and a System.InvalidOperationException occurs with the
|| message: "There were not enough free threads in the ThreadPool
|| object to complete the operation."
||
|| It's as if the worker threads are never released by the
|| HttpWebRequest, which seems like a pretty serious bug.
||
|| Oh, and so far I only repro this on Win9x (ME to be precise, haven't
|| tried 98 yet). The worker threads seem to be released fine by all
|| of the NT flavors.
||
|| Below I am including 2 things:
|| (a) The methods that work with the HttpWebRequest object
|| (b) The result of calling ThreadPool.GetAvailableThreads(out int
|| workerThreads, out int completionPortThreads) as I create each
|| HttpWebRequest (until the number of workers hits 0 at which point an
|| exception is thrown)
||
|| Thanks,
|| Dan
||
|| (a) basically I call these in a loop, such that no more than 8 are
|| active at any given time
||
|| public void BeginScrapeContent(ScraperConfig config) {
|| if (config == null) {
|| if (_config == null) {
|| throw new ArgumentNullException("config", "config or _config
|| can not be null.");
|| }
|| } else {
|| this._config = config;
|| }
|| HttpWebRequest request =
|| (HttpWebRequest)WebRequest.Create(this.Url);
|| request.BeginGetResponse(new AsyncCallback(EndRequestContent1),
|| request); }
||
|| private void EndRequestContent1(IAsyncResult ar) {
|| HttpWebRequest request = null;
|| try {
|| request = (HttpWebRequest)ar.AsyncState;
|| request.EndGetResponse(ar);
|| _content += new
|| StreamReader(request.GetResponse().GetResponseStream()).ReadToEnd();
|| _contentRetrievalSuccess1 = true;
|| } catch (Exception ex) {
|| _contentRetrievalSuccess1 = false;
|| Debug.WriteLine(ex.Message);
|| _failReason = ex is WebException ?
|| ContentRetrievalFailure.NoResponse :
|| ContentRetrievalFailure.BadResponse; } finally {
|| try {
|| request.GetResponse().Close();
|| } catch {
|| Debug.WriteLine("Error calling close");
|| }
|| }
|| }
||
|| (b) Here's the result, of in that loop where I call the
|| BeginScrapeContent method), checking ThreadPool.GetAvailableThreads
|| each time I perform a new BeginScrapeContent call (again, there are
|| ever only a max of 8 of these at any time). As you can see the
|| number of worker threads drops to 0 quite quickly.
||
|| wrkThrds: 25 cpThrds: 25
|| wrkThrds: 25 cpThrds: 25
|| wrkThrds: 24 cpThrds: 25
|| wrkThrds: 24 cpThrds: 25
|| wrkThrds: 23 cpThrds: 25
|| wrkThrds: 21 cpThrds: 25
|| wrkThrds: 19 cpThrds: 25
|| wrkThrds: 19 cpThrds: 25
|| wrkThrds: 19 cpThrds: 25
|| wrkThrds: 17 cpThrds: 25
|| wrkThrds: 17 cpThrds: 25
|| wrkThrds: 19 cpThrds: 25
|| wrkThrds: 17 cpThrds: 25
|| wrkThrds: 17 cpThrds: 25
|| wrkThrds: 18 cpThrds: 25
|| wrkThrds: 17 cpThrds: 25
|| wrkThrds: 18 cpThrds: 25
|| wrkThrds: 18 cpThrds: 25
|| wrkThrds: 17 cpThrds: 25
|| wrkThrds: 17 cpThrds: 25
|| wrkThrds: 18 cpThrds: 25
|| wrkThrds: 17 cpThrds: 25
|| wrkThrds: 18 cpThrds: 25
|| wrkThrds: 17 cpThrds: 25
|| wrkThrds: 17 cpThrds: 25
|| wrkThrds: 18 cpThrds: 25
|| wrkThrds: 16 cpThrds: 25
|| wrkThrds: 17 cpThrds: 25
|| wrkThrds: 15 cpThrds: 25
|| wrkThrds: 16 cpThrds: 25
|| wrkThrds: 16 cpThrds: 25
|| wrkThrds: 13 cpThrds: 25
|| wrkThrds: 13 cpThrds: 25
|| wrkThrds: 12 cpThrds: 25
|| wrkThrds: 12 cpThrds: 25
|| wrkThrds: 12 cpThrds: 25
|| wrkThrds: 12 cpThrds: 25
|| wrkThrds: 11 cpThrds: 25
|| wrkThrds: 12 cpThrds: 25
|| wrkThrds: 12 cpThrds: 25
|| wrkThrds: 11 cpThrds: 25
|| wrkThrds: 10 cpThrds: 25
|| wrkThrds: 9 cpThrds: 25
|| wrkThrds: 8 cpThrds: 25
|| wrkThrds: 9 cpThrds: 25
|| wrkThrds: 9 cpThrds: 25
|| wrkThrds: 8 cpThrds: 25
|| wrkThrds: 8 cpThrds: 25
|| wrkThrds: 7 cpThrds: 25
|| wrkThrds: 12 cpThrds: 25
|| wrkThrds: 10 cpThrds: 25
|| wrkThrds: 10 cpThrds: 25
|| wrkThrds: 7 cpThrds: 25
|| wrkThrds: 9 cpThrds: 25
|| wrkThrds: 8 cpThrds: 25
|| wrkThrds: 11 cpThrds: 25
|| wrkThrds: 13 cpThrds: 25
|| wrkThrds: 13 cpThrds: 25
|| wrkThrds: 11 cpThrds: 25
|| wrkThrds: 10 cpThrds: 25
|| wrkThrds: 11 cpThrds: 25
|| wrkThrds: 9 cpThrds: 25
|| wrkThrds: 12 cpThrds: 25
|| wrkThrds: 11 cpThrds: 25
|| wrkThrds: 10 cpThrds: 25
|| wrkThrds: 8 cpThrds: 25
|| wrkThrds: 7 cpThrds: 25
|| wrkThrds: 7 cpThrds: 25
|| wrkThrds: 8 cpThrds: 25
|| wrkThrds: 8 cpThrds: 25
|| wrkThrds: 8 cpThrds: 25
|| wrkThrds: 10 cpThrds: 25
|| wrkThrds: 14 cpThrds: 25
|| wrkThrds: 18 cpThrds: 25
|| wrkThrds: 19 cpThrds: 25
|| wrkThrds: 17 cpThrds: 25
|| wrkThrds: 17 cpThrds: 25
|| wrkThrds: 17 cpThrds: 25
|| wrkThrds: 15 cpThrds: 25
|| wrkThrds: 15 cpThrds: 25
|| wrkThrds: 14 cpThrds: 25
|| wrkThrds: 14 cpThrds: 25
|| wrkThrds: 14 cpThrds: 25
|| wrkThrds: 14 cpThrds: 25
|| wrkThrds: 11 cpThrds: 25
|| wrkThrds: 10 cpThrds: 25
|| wrkThrds: 10 cpThrds: 25
|| wrkThrds: 10 cpThrds: 25
|| wrkThrds: 11 cpThrds: 25
|| wrkThrds: 11 cpThrds: 25
|| wrkThrds: 11 cpThrds: 25
|| wrkThrds: 10 cpThrds: 25
|| wrkThrds: 10 cpThrds: 25
|| wrkThrds: 8 cpThrds: 25
|| wrkThrds: 9 cpThrds: 25
|| wrkThrds: 10 cpThrds: 25
|| wrkThrds: 5 cpThrds: 25
|| wrkThrds: 5 cpThrds: 25
|| wrkThrds: 8 cpThrds: 25
|| wrkThrds: 7 cpThrds: 25
|| wrkThrds: 7 cpThrds: 25
|| wrkThrds: 5 cpThrds: 25
|| wrkThrds: 6 cpThrds: 25
|| wrkThrds: 5 cpThrds: 25
|| wrkThrds: 4 cpThrds: 25
|| wrkThrds: 3 cpThrds: 25
|| wrkThrds: 3 cpThrds: 25
|| wrkThrds: 1 cpThrds: 25
|| wrkThrds: 0 cpThrds: 25
||
|| Exception Type: System.InvalidOperationException
|| Message: There were not enough free threads in the ThreadPool
|| object to complete the operation.
|| Stack Trace:
|| at System.Net.HttpWebRequest.BeginGetResponse(AsyncCallback
|| callback, Object state)
|| at TestApp.Retriever.BeginScrapeContent(ScraperConfig config)
|| at TestApp.Form1.RetrieveContentStart()
|| Source: System
|| Target Site: System.IAsyncResult
|| BeginGetResponse(System.AsyncCallback, System.Object)

 

[Dan Battagin]

John,

Thanks for the reply. My question though: I think I am already doing this.
I've included a bit more of the code that I am using. This is the code that
calls the BeginScrapeContent below. Note how I only call that method if
there are 8 or fewer active requests.

It's as though, even though I am doing that, the worker threads are never
being released...I've also tried checking in the while loop for the number
of available worker threads and sleeping if there are less than 5, but that
doesn't work either (app just hangs)

Thoughts?

Thanks,
Dan

for (int i=0; i<items.Count; i++) {
if (_stop == false) {
Interlocked.Increment(ref _activeRequests);
while (_activeRequests > 8) {
Thread.Sleep(100);
}
s = new Scraper(items.ToString());
s.ContentRetrievalComplete += new
ContentRetrievalCompleteEventHandler(this.RetrieveScraperComplete);
s.BeginScrapeContent(config);
}
....
(and then in RetrieveScraperComplete)
....

Interlocked.Decrement(_activeRequests);

 

[Dan Battagin]

Willy, I could go for that, but I dont think it will work - the worker
threads never seem to free up, and once they get to 0 I'm hosed...

Do you have an example of what I should do? (Also, I replied to John's
response as well, you might want to check that out)

Thanks much for your time,
Dan

 

[Dave]

You could use your own threadpool class - then you could set the threadpool
limit as high as you want, and you would get a lot more control over
when/how the threads were terminating. If nothing else it might shed some
light on why the threadpool threads are not terminating. There are several
implementations of a threadpool object around - you should be able to google
one up, or search the MSFT .NET newsgroups for an earlier posting of same.

BTW: we use WebRequests and don't have this problem, so something else is
undoubtedly going on in your app.