Forcing ThreadPool threads to act like they are not background threads

[orekin]

Hi There

I have been programming C# for a couple of months and am trying to
master Threading.

I understand that ThreadPool uses background threads (see code example
in MSDN page titled 'ThreadPool Class [C#]') .... however I would like
to ensure that all my ThreadPool threads have completed before I exit
my Main function.

At the moment I am using this code in my main function:
ThreadPool.GetMaxThreads(out maxThreads,out
maxPortCompletionThreads);
do
{
Thread.Sleep(100);
ThreadPool.GetAvailableThreads(out workerThreads,out
portCompletionThreads);
}while(maxThreads!=workerThreads||maxPortCompletionThreads!=portCompletionThreads);

I use the above code right at the end of my main function, and it
seems to do the job of ensuring that all ThreadPool threads finish
before the main function completes (and consequently the application
finishes).

I am aware that I do not 'have to' use the ThreadPool - I could start
my own threads up then join the main thread to the own threads.
However I would prefer to stick to the ThreadPool if I can.

My question is simply 'is there a better way to ensure that all
ThreadPool threads complete before exiting the main function' ? The
code I am writing at the moment is part of a large project and I want
to make my contribution fairly standard (rather than have other
programmers look at my work and say 'what the heck is he doing here?!?
I have not seen this before!) ...

Thanks In Advance
Bill

 

[Dave]

You should use a WaitHandle or a locking mechanism to synchronize multiple threads.

private AutoResetEvent ev = new AutoResetEvent(false);
private delegate void ThreadPooledCallInvoker();

public void ThreadPooledCall()
{
// Todo: stuff

// Alert waiting threads we are complete
ev.Set();
}

public void DoStuff()
{
new ThreadPooledCallInvoker(ThreadPooledCall).BeginInvoke(null, null);

// wait till the background thread completes for up to 10 seconds
if (ev.WaitOne(10000, false))
{
// timeout expired!
}
}

 

[Richard Blewett [DevelopMentor]]

One small nit - you need to give yourself some mechanism for calling EndInvoke on the delegate instance you called BeginInvoke on (auch as passing an AsyncCallback)

Regards

Richard Blewett - DevelopMentor
http://www.dotnetconsult.co.uk/weblog
http://www.dotnetconsult.co.uk

You should use a WaitHandle or a locking mechanism to synchronize multiple threads.

private AutoResetEvent ev = new AutoResetEvent(false);
private delegate void ThreadPooledCallInvoker();

public void ThreadPooledCall()
{
// Todo: stuff

// Alert waiting threads we are complete
ev.Set();
}

public void DoStuff()
{
new ThreadPooledCallInvoker(ThreadPooledCall).BeginInvoke(null, null);

// wait till the background thread completes for up to 10 seconds
if (ev.WaitOne(10000, false))
{
// timeout expired!
}
}

 

[orekinbck]

Hi Dave & Richard

The Beta site is having some errors, so this might be the first time
this reply appears, or the Fourth!

Thanks for your response. I could get the code you supplied working
with one thread, but had problems using AutoResetEvent with more than
one worker thread. What do you think of this solution (you should be
able to cut paste and compile this code)?

using System;
using System.Threading;
using System.Diagnostics;

namespace AutoResetEvent_Examples
{
class MyMainClass
{
private delegate void ThreadPooledCallInvoker();

public static void ThreadPooledCall()
{
Debug.WriteLine("Hello from ThreadPooledCall!");
Thread.Sleep(500);
}

static void Main()
{
// Create the ThreadPooledCallInvoker delegates.
ThreadPooledCallInvoker dlgt = new
ThreadPooledCallInvoker(ThreadPooledCall);
ThreadPooledCallInvoker dlgt2 = new
ThreadPooledCallInvoker(ThreadPooledCall);

// Get the delegates going
IAsyncResult ar = dlgt.BeginInvoke(null,null);
IAsyncResult ar2 = dlgt2.BeginInvoke(null,null);
Thread.Sleep(0); //Allow Delegates to get started

//Create WaitHandleArray
WaitHandle[] myWaiters = new WaitHandle[2];
myWaiters[0] = ar.AsyncWaitHandle;
myWaiters[1] = ar2.AsyncWaitHandle;

if (!WaitHandle.WaitAll(myWaiters,1000,false))
{
//Timed Out
Debug.WriteLine("Threads Timed Out");
}
else
{
Debug.WriteLine("Threads Completed");
dlgt.EndInvoke(ar);
dlgt2.EndInvoke(ar2);
}
}
}
}

Thanks
Bill

 

[Dave]

One small nit - you need to give yourself some mechanism for calling EndInvoke on the delegate instance you called

BeginInvoke on (auch as passing an AsyncCallback)

Only if a return value is required. EndInvoke can be ignored otherwise.

MSDN:

If you need the return value from a process started with this method, call EndInvoke to get the value.

 

[Dave]

Your code doesn't work becaues you have misused WaitHandles. The purpose of a wait handle is to set a synchronized flag to let
another thread know that a process has been completed or that the application's "state" has changed.

The WaitHandles that you are using in your code example are never "set". This is why "Threads Timed Out" will always be written to
the output.

I revisited my code to figure out why you had trouble running this using multiple threads. It seems that I have made a mistake and
forgot a character (an important one at that):

The following lines are the context of the error:

if (ev.WaitOne(10000, false))
{
// timeout expired!
}

It should be changed to:

if (!ev.WaitOne(10000, false))
// WaitOne returns false if the specified time period elapsis before the handle is signaled
{
// timeout expired!
}


Try again using multiple threads and it should work just fine (with the correction). Here's an example:

The ouptut for the following code will be:

ThreadPooledCall: Call #0
Pooled: True
ThreadPooledCall: Call #1
Pooled: True
ThreadPooledCall: Call #2
Pooled: True
ThreadPooledCall: Call #3
Pooled: True
ThreadPooledCall: Call #4
Pooled: True
ThreadPooledCall: Call #5
Pooled: True
ThreadPooledCall: Call #6
Pooled: True
ThreadPooledCall: Call #7
Pooled: True
ThreadPooledCall: Call #8
Pooled: True
ThreadPooledCall: Call #9
Pooled: True
Finished.

Here's the code:

using System;
using System.Threading;
using System.Diagnostics;

namespace TestConsole
{
public class ThreadSyncTest
{
private AutoResetEvent ev = new AutoResetEvent(false);
private delegate void ThreadPooledCallInvoker(string Data);

public void ThreadPooledCall(string Data)
{
lock (this)
// synchronize access to the Debug class (it is not thread-safe)
{
Debug.WriteLine(string.Format("ThreadPooledCall: {0}", Data));
Debug.Indent();
Debug.WriteLine(string.Format("Pooled: {0}", Thread.CurrentThread.IsThreadPoolThread));
Debug.Unindent();
}

// Block the thread for 1 second
System.Threading.Thread.Sleep(1000);

// Alert waiting threads we are complete
ev.Set();
}


public void DoStuff()
{
for (int i = 0; i < 10; i++)
new ThreadPooledCallInvoker(ThreadPooledCall).BeginInvoke("Call #" + i, null, null);

for (int i = 0; i < 10; i++)
// wait till each background thread completes for up to 10 seconds
if (!ev.WaitOne(10000, false))
{
Debug.WriteLine("Timeout Expired.");
}

Debug.WriteLine("Finished.");
}
}
}




--
Dave Sexton
[email protected]
-----------------------------------------------------------------------

Hi Dave & Richard

The Beta site is having some errors, so this might be the first time
this reply appears, or the Fourth!

Thanks for your response. I could get the code you supplied working
with one thread, but had problems using AutoResetEvent with more than
one worker thread. What do you think of this solution (you should be
able to cut paste and compile this code)?

using System;
using System.Threading;
using System.Diagnostics;

namespace AutoResetEvent_Examples
{
class MyMainClass
{
private delegate void ThreadPooledCallInvoker();

public static void ThreadPooledCall()
{
Debug.WriteLine("Hello from ThreadPooledCall!");
Thread.Sleep(500);
}

static void Main()
{
// Create the ThreadPooledCallInvoker delegates.
ThreadPooledCallInvoker dlgt = new
ThreadPooledCallInvoker(ThreadPooledCall);
ThreadPooledCallInvoker dlgt2 = new
ThreadPooledCallInvoker(ThreadPooledCall);

// Get the delegates going
IAsyncResult ar = dlgt.BeginInvoke(null,null);
IAsyncResult ar2 = dlgt2.BeginInvoke(null,null);
Thread.Sleep(0); //Allow Delegates to get started

//Create WaitHandleArray
WaitHandle[] myWaiters = new WaitHandle[2];
myWaiters[0] = ar.AsyncWaitHandle;
myWaiters[1] = ar2.AsyncWaitHandle;

if (!WaitHandle.WaitAll(myWaiters,1000,false))
{
//Timed Out
Debug.WriteLine("Threads Timed Out");
}
else
{
Debug.WriteLine("Threads Completed");
dlgt.EndInvoke(ar);
dlgt2.EndInvoke(ar2);
}
}
}
}

Thanks
Bill

 

[Willy Denoyette [MVP]]

Only if a return value is required. EndInvoke can be ignored otherwise.

MSDN:

If you need the return value from a process started with this method, call
EndInvoke to get the value.

--

No, each BeginInvoke must be paired with a call to EndInvoke, otherwise the
CLR can possibly leak a handle. Only Control.BeginInvoke doesn't have to
call EndInvoke as it uses a different mechanism to dispatch the call on the
Control's thread.

From MSDN - Asynchronous Programming Overview:
CAUTION Always call EndInvoke after your asynchronous call completes.

Willy.

 

[Dave]

Thanks for clearing that up. It should be known, however, that the MSDN docs on this particular topic aren't exactly clear about
this.

This summary explains that EndInvoke is used to obtain a return value. It would have been nice if they mentioned also that it is
required to prevent memory leaks!

http://msdn.microsoft.com/library/d...s/cpguide/html/cpovrasynchronousdelegates.asp

I also searched for the "Caution" line that you mentioned and found it after reading through a few more docs that don't mention
anything about EndInvoke being a required call.

If it's really that big of an issue (memory leaks are!) then I think it should be documented better.

The site that I gave in my previous post was for ISynchronizeInvoke, which is used by WinForms Controls. I guess because they use
the MessageLoop that EndInvoke isn't required?

 

[orekinbck]

Thanks Dave, that makes sense.

Cheers
Bill

 

[Richard Blewett [DevelopMentor]]

Thjis issue was a disaster from start to finish. They forgot to tell anyone about this in 1.0 and so you see alot of sample code written using the "Fire and Forget" pattern. Te when 1.1 came out people noticed that the docs had changed (note: "docs had changed" not "it was shouted from the rooftops"). You will find many well respected book talk about the fire and forget pattern of async delegate invocation (Don Box's Essential .NET is one IIRC) because no-one knew it was an issue until 1.1.

Yep, ISynchronizeInvoke.BeginInvoke = Control.BeginInvoke

It was also tremendously useful to have the two method named the same thing - Control.BeginInvoke and Delegate.BeginInvoke - as the two method are often used in close proximity but have very different functions and usage patterns. I once asked someone from the WinForms team why then used BeginInvoke for Control and they said "we got there first" <sigh>

Regards

Richard Blewett - DevelopMentor
http://www.dotnetconsult.co.uk/weblog
http://www.dotnetconsult.co.uk

Thanks for clearing that up. It should be known, however, that the MSDN docs on this particular topic aren't exactly clear about
this.

This summary explains that EndInvoke is used to obtain a return value. It would have been nice if they mentioned also that it is
required to prevent memory leaks!

http://msdn.microsoft.com/library/d...s/cpguide/html/cpovrasynchronousdelegates.asp

I also searched for the "Caution" line that you mentioned and found it after reading through a few more docs that don't mention
anything about EndInvoke being a required call.

If it's really that big of an issue (memory leaks are!) then I think it should be documented better.

The site that I gave in my previous post was for ISynchronizeInvoke, which is used by WinForms Controls. I guess because they use
the MessageLoop that EndInvoke isn't required?

 

[Willy Denoyette [MVP]]

Thjis issue was a disaster from start to finish. They forgot to tell
anyone about this in 1.0 and so you see alot of sample code written using
the "Fire and Forget" pattern. Te when 1.1 came out people noticed that
the docs had changed (note: "docs had changed" not "it was shouted from
the rooftops"). You will find many well respected book talk about the fire
and forget pattern of async delegate invocation (Don Box's Essential .NET
is one IIRC) because no-one knew it was an issue until 1.1.

Yep, ISynchronizeInvoke.BeginInvoke = Control.BeginInvoke

It was also tremendously useful to have the two method named the same
thing - Control.BeginInvoke and Delegate.BeginInvoke - as the two method
are often used in close proximity but have very different functions and
usage patterns. I once asked someone from the WinForms team why then used
BeginInvoke for Control and they said "we got there first" <sigh>

Regards

Richard Blewett - DevelopMentor
http://www.dotnetconsult.co.uk/weblog
http://www.dotnetconsult.co.uk

Richard,

Note that the CLR team only discovered this after the facts (a leak), only
then was said they "reserved the right" to leak when failing to call
EndInvoke. IMO, for some time they considered it a bug/design flaw and were
looking for a solution, but finally gave up and did hand it back to the BCL
team who said he! these BeginInvoke/EndInvoke methods are compiler
generated, they are not our business and the compiler team said what can we
do about this? nothing - It's up to the user of the asynchronous delegate to
change it's design. So finally it was handed over to the technical writers
who wrote the Asynchronous Design pattern stuff in MSDN who simply added a
Caution ... (note that in v2.0 this has changed into Important ...).

This pattern is something I recognize and it makes me laugh when I think of
the many mistakes I (and other team mates) made in the past and still make
today, and the way we pushed them gently into another team. Software is
written by humans and humans are imperfect, that's why I still love this job
so much after all those years.

Willy.

 

[Dave]

Sounds like something should be pretty high up on Microsoft's TODO list for .NET.

Any word of this being fixed in 2.0? Is it not regarded as a *bug* anymore, and just a feature?

 

[Richard Blewett [DevelopMentor]]

This is now regarded as a feature - after all its the only way you'll know if an exception was thrown on the method. Btw: this becomes a bit neater with Anonymous method in v2.0

del.BeginInvoke(42, "Hello"
delegate
{
del.EndInvoke();
}, null);

Regards

Richard Blewett - DevelopMentor
http://www.dotnetconsult.co.uk/weblog
http://www.dotnetconsult.co.uk

Sounds like something should be pretty high up on Microsoft's TODO list for .NET.

Any word of this being fixed in 2.0? Is it not regarded as a *bug* anymore, and just a feature?

 

[Richard Blewett [DevelopMentor]]

Sorry, that should be:

IAsyncResult ar = del.BeginInvoke(42, "Hello"
delegate
{
del.EndInvoke(ar);
}, null);

Regards

Richard Blewett - DevelopMentor
http://www.dotnetconsult.co.uk/weblog
http://www.dotnetconsult.co.uk

This is now regarded as a feature - after all its the only way you'll know if an exception was thrown on the method. Btw: this becomes a bit neater with Anonymous method in v2.0

del.BeginInvoke(42, "Hello"
delegate
{
del.EndInvoke();
}, null);

Regards

Richard Blewett - DevelopMentor
http://www.dotnetconsult.co.uk/weblog
http://www.dotnetconsult.co.uk

Sounds like something should be pretty high up on Microsoft's TODO list for .NET.

Any word of this being fixed in 2.0? Is it not regarded as a *bug* anymore, and just a feature?



[microsoft.public.dotnet.languages.csharp]