Dispatching tasks to multiple cores from a batch file

[adish]

Hi,

I need to execute many (say 20) time consuming tasks (each takes
about 1 hour to run).
At the moment I dispatch them serially one by one.
According to the TaskManager the process runs on 1 CPU at 50%
utilization.
The whole batch finishes running after about 20 hours.

I'm using a hyper-threaded XP machine and the env var.
NUMBER_OF_PROCESSORS=2.
I'd like to dispatch the tasks in parallel to each CPU so that my
processing time will be approx. 10hour.

How can I do this in a batch file script.

I know about the start command, but that does not give precise control
over the number of running processes, and can cause hanging or 3
processes to run in parallel. There is no option to wait.

Is there some way to loop on NUMBER_OF_PROCESSORS and
"waitMultipleThreads" until one of the tasks is done?

Please advise.
Thanks,
Adi

 

[Ridge Runner]

Why bother?

 

[Pegasus \(MVP\)]

Hi,

I need to execute many (say 20) time consuming tasks (each takes
about 1 hour to run).
At the moment I dispatch them serially one by one.
According to the TaskManager the process runs on 1 CPU at 50%
utilization.
The whole batch finishes running after about 20 hours.

I'm using a hyper-threaded XP machine and the env var.
NUMBER_OF_PROCESSORS=2.
I'd like to dispatch the tasks in parallel to each CPU so that my
processing time will be approx. 10hour.

How can I do this in a batch file script.

I know about the start command, but that does not give precise control
over the number of running processes, and can cause hanging or 3
processes to run in parallel. There is no option to wait.

Is there some way to loop on NUMBER_OF_PROCESSORS and
"waitMultipleThreads" until one of the tasks is done?

Please advise.
Thanks,
Adi

Try this:
@echo off
set Count=0
set AppName=MyApp.exe

:NextTask
set /a Count=%Count%+1
if %Count% GTR 20 goto :eof
start /b "My Application" "c:\Program Files\My Folder\%AppName%"

:Wait
ping localhost -n 60 > nul
tasklist | find /i "%AppName%" && goto Wait
goto NextTask

 

[Bob I]

You do realize that "Hyperthreading" in testing under best conditions
only increased throughput about 20%. Now on the other hand, if you had a
dual-core CPU, you may get maybe a 70% increase. But you aren't going to
see 10 hours, so don't get your hopes up.

 

[adish]

Well, a lot of the time is spent waiting for I/O (from the HDD), so
hopefully, I will see some speedup.
My 10 hour example, was just for brevity.
Also, I want this script to execute of multi-core machines too, where,
as you, I will see much more effect.
Thanks for the prompt reply,
Adi

 

[adish]

Hi,

Thanks for the prompt reply.
A few questions though:
1. tasklist returns "ERROR: The RPC server is unavailable." on my XP
machine. Is there some some service I need to start?
2. You are using ping as a "sleep" timer, right? Isn't there some
nicer way?
3. I don't see how the script takes into account the number of
processors. It seems that it will just spawn a process and when that's
finished it will call the next task. I could do that if I didn't use
start. Am I missing something?

Thanks,
Adi

 

[Bob I]

If the HD is the bottleneck, you won't see any improvement at all.

 

[Pegasus \(MVP\)]

See below.

Hi,

Thanks for the prompt reply.
A few questions though:
1. tasklist returns "ERROR: The RPC server is unavailable." on my XP
machine. Is there some some service I need to start?

*** I suggest you google for the string "The RPC server is unavailable"
*** to find out if this is of concern.

2. You are using ping as a "sleep" timer, right? Isn't there some
nicer way?

*** Why is ping not nice? It's native and it consumes few CPU
*** resources and it works on all Windows platforms. If you
*** don't like it then you can use one of the many sleep utilities.
*** There is even one in WinXP!

3. I don't see how the script takes into account the number of
processors. It seems that it will just spawn a process and when that's
finished it will call the next task. I could do that if I didn't use
start. Am I missing something?

*** It doesn't, and I don't know of any way to do it.

 

[adish]

Thanks.
I'll give it a shot.
Adi