threadpool: how can I make sure just an X amount of threads (CPU's)are used?

M

Matthijs de Z

Hi,

I've made a test project to see how the CPU usage will react based on
the example given on: http://msdn.microsoft.com/en-us/library/system.threading.threadpool.queueuserworkitem(VS.71).aspx
The code is below.

What I want to achieve is to be able to make sure that for a number of
task send to the threadpool (or any other solution), I will only use x
number of CPU’s. With the code below, it doesn’t matter how I set
ThreadPool.SetMaxThreads and ThreadPool.SetMinThreads, when I run the
program, all my CPU’s are consumed. I’ve tested this on a computer
with 2 and with 8 cores. Also the 8 core computer use all cores.

What do I need to do to get it working properly?
Kind regards,

Matthijs


// This example shows how to create an object containing task
// information, and pass that object to a task queued for
// execution by the thread pool.
using System;
using System.Threading;

// TaskInfo holds state information for a task that will be
// executed by a ThreadPool thread.
public class TaskInfo
{
// State information for the task. These members
// can be implemented as read-only properties, read/write
// properties with validation, and so on, as required.
public string Boilerplate;
public int Value;
public int CurrentACtiveThreads;

// Public constructor provides an easy way to supply all
// the information needed for the task.
public TaskInfo(string text, int number, int activeThreads)
{
Boilerplate = text;
Value = number;
CurrentACtiveThreads = activeThreads;
}
}

public class Example
{
public static void Main()
{
// Create an object containing the information needed
// for the task.

// Queue the task and data.
ThreadPool.SetMaxThreads(2, 1);
ThreadPool.SetMinThreads(1, 1);

int availableThread,completionPortThreads;

for (int i = 0; i < 100; i++)
{
ThreadPool.GetAvailableThreads(out availableThread, out
completionPortThreads);
TaskInfo ti = new TaskInfo("This report displays the
number",i,availableThread);
ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc),
ti);
Thread.Sleep(50);
}
Console.WriteLine("Main thread does some work, then sleeps.");

// If you comment out the Sleep, the main thread exits before
// the ThreadPool task has a chance to run. ThreadPool uses
// background threads, which do not keep the application
// running. (This is a simple example of a race condition.)

Thread.Sleep(100000);

Console.WriteLine("Main thread exits.");
}

// The thread procedure performs the independent task, in this
case
// formatting and printing a very simple report.
//
static void ThreadProc(Object stateInfo)
{
TaskInfo ti = (TaskInfo)stateInfo;
Console.WriteLine(ti.Boilerplate + " " + ti.Value +" " +
ti.CurrentACtiveThreads);
for (int i = 0; i < 10000000; i++)
{
Math.Sqrt(Math.Sin(i));
Math.Tan(Math.Sqrt(Math.Sin(i)));
}
}
}
 
P

Peter Duniho

Matthijs said:
Hi,

I've made a test project to see how the CPU usage will react based on
the example given on: http://msdn.microsoft.com/en-us/library/system.threading.threadpool.queueuserworkitem(VS.71).aspx
The code is below.

What I want to achieve is to be able to make sure that for a number of
task send to the threadpool (or any other solution), I will only use x
number of CPU’s. With the code below, it doesn’t matter how I set
ThreadPool.SetMaxThreads and ThreadPool.SetMinThreads, when I run the
program, all my CPU’s are consumed. I’ve tested this on a computer
with 2 and with 8 cores. Also the 8 core computer use all cores.

What do I need to do to get it working properly?

Either don't use ThreadPool (which will always try to run all of the
work items that have been queued), or don't queue more work items than
you want running at once.

You definitely do not want to mess around with SetMaxThreads(). The
ThreadPool is used not just by your own code, but by a variety of other
..NET components.

It won't have the effect you want anyway — as the documentation clearly
states, "You cannot set the number of worker threads or the number of
I/O completion threads to a number smaller than the number of processors
in the computer", so it's not a useful way to restrict how many CPU
cores are used at once — and it can have a serious negative effect on
the efficiency of the other components that use the ThreadPool class.

It is not very hard to set up your own queue to use as a sort of
"gatekeeper" to restrict how many work items you've actually queued at
once. In fact, I'm pretty sure I posted an example of doing that in
this newsgroup some time ago; Google Groups search can probably find the
code for you. Alternatively, I don't recall for sure, but I think maybe
Jon Skeet's "MiscUtil" library's custom thread pool implementation might
support what you want to do. You can find that library here:
http://www.yoda.arachsys.com/csharp/miscutil/

Pete
 
M

Matthijs de Z

It is not very hard to set up your own queue to use as a sort of
"gatekeeper" to restrict how many work items you've actually queued at
once.  In fact, I'm pretty sure I posted an example of doing that in
this newsgroup some time ago; Google Groups search can probably find the
code for you.  Alternatively, I don't recall for sure, but I think maybe
Jon Skeet's "MiscUtil" library's custom thread pool implementation might
support what you want to do.  You can find that library here:http://www..yoda.arachsys.com/csharp/miscutil/

Hi Peter,

thanks for your responds.
I've been googling around using
group:microsoft.public.dotnet.languages.csharp author:"peter duniho"
and some keywords, but it seems you're a very active newsgroup writer
(helping out people). I was not able to find the article you refer to.
Do you recall a subject for that, to narrow down my search?
Kind regards,

Matthijs
 
P

Peter Duniho

Matthijs said:
thanks for your responds.
I've been googling around using
group:microsoft.public.dotnet.languages.csharp author:"peter duniho"
and some keywords, but it seems you're a very active newsgroup writer
(helping out people). I was not able to find the article you refer to.
Do you recall a subject for that, to narrow down my search?
Kind regards,

I looked around, and found a couple of posts that might be helpful:
http://groups.google.com/group/microsoft.public.dotnet.languages.csharp/msg/2064ea48b0a9a1a4
http://groups.google.com/group/microsoft.public.dotnet.languages.csharp/msg/0d3b7aee7712b07e

The second example is much more complicated, but the portion of it that
deals with limiting concurrent tasks is probably more directly relevant
to your question, and isn't itself all that complicated. See the
"DownloadManager" class.

The first example actually accomplishes a very similar outcome, and by
virtue of using the BackgroundWorker winds up using the ThreadPool.
Note, however, that the particular code I posted is specifically
dependent on the BackgroundWorker being used in a context where there's
a SynchronizationContext to handle cross-thread invocations (e.g. a
Forms GUI thread, where Application.Run() has been called). Without
that, the code I posted before would need additional synchronization
dealing with the queue of tasks.

If you have questions about either of those, please feel free to ask.
In general, the basic idea is: keep a counter of how many concurrent
tasks you've already sent off for processing (by whatever means),
queuing any tasks added when the counter is already at the maximum, and
dequeuing a new task to process when a running task is completed.

There are lots of ways to accomplish that, but they all share that same
basic strategy.

Pete
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top