Launch new thread vs new process for performing intensive databasetasks

[tombrogan3]

Hi, I'm writing a service which has to monitor a database looking for
new feeds (basically a few hundred thousand rows with a common id).

When it finds a new feed I can either:
1) Process it on the current thread (not ideal as the app must be seen
to be responding to new feeds straight away, and this processing can
take up to an hour).
2) Launch a new thread to process the feed.
3) Launch a new process to process the feed.

Are there any advantages / disadvantages to launching processes rather
than threads? Would the performance be better?

Obviously both threads / processes will access the same database, but
that's the only thing they have in common. There are no other shared
resources or values.

Cheers for any help,

Tom.

 

[Arne Vajhøj]

Hi, I'm writing a service which has to monitor a database looking for
new feeds (basically a few hundred thousand rows with a common id).

When it finds a new feed I can either:
1) Process it on the current thread (not ideal as the app must be seen
to be responding to new feeds straight away, and this processing can
take up to an hour).
2) Launch a new thread to process the feed.
3) Launch a new process to process the feed.

Are there any advantages / disadvantages to launching processes rather
than threads? Would the performance be better?

Obviously both threads / processes will access the same database, but
that's the only thing they have in common. There are no other shared
resources or values.

Thread creation is faster than process creation.

It is much easier to transfer data between threads than
between processes.

The only advantage of using a separate process I can think of is
that it can keep running after the starting process has exited.

Arne

 

[Jon Skeet [C# MVP]]

Thread creation is faster than process creation.

It is much easier to transfer data between threads than
between processes.

The only advantage of using a separate process I can think of is
that it can keep running after the starting process has exited.

It also adds an extra layer of isolation - if you have a crufty
database driver (perhaps including some native code) or something else
which might tear down the process, it's nice to be isolated from it.
Not often an issue, but I've seen some nasty drivers before...

Jon

 

[Ciaran O''Donnell]

If processing this stuff takes a hour then it sounds like a large amount of
data. I have found process spawning for this kind of work better in the past
as when the job is done, the process finishes and the memory is completely
release back to the OS. In my opinion, it is nice to know that when your
done, its all cleaned up.
If you do this on a new thread, you may still take up LOB heap space
especially if dealing with large strings, plus doing it on a thread will
possibly keep forcing garbage collections of your main heap which will slow
your app down.
Passing the information to the other process is pretty simple as command
line parameters and potentially getting information back from it by consuming
its standardoutput stream. You can start a new thread to launch the process,
add an event handler to DataReceived event for the standard output and then
wait for the new process to exit. Then act on whatever it did.