Does calling GC.Collect() only work from the main thread?

[Guest]

To take advantage of the hardware power on my company’s application server,
which has 4 CPUs and 8Gs of memory, I have created a multi-threaded
application that would simultaneously process multiple requests to generate
reports.

The main thread of the application would spawn several worker threads to
process requests and generate reports, and, at the same time, the main thread
would monitor the processing statuses of all worker threads.

To conserve memory, at the end of the class/method, which is associated with
the worker thread, GC.Collect() would be called after calling all the heavy
objects’ Dispose methods and setting them to null.

Strangely enough, just one request to generate a series of big reports could
raise OUT OF MEMORY errors.

So I converted the application to be single-threaded and process requests
one at a time. The application works fine and, at any given time, only 300 MB
of memory, at most, are used to process the very same request.

I could not help but conclude that when GC.Collect() is called from a worker
thread, no disposed memory would be re-claimed right away, unless it is
called from the main thread.

Is my conclusion correct? Or there is something else I do not know about GC
in .NET?

Thanks in advance for any help or advice.

Don Lin

 

[Henning Krause [MVP - Exchange]]

Hello,

you should generelly not call GC.Collect yourself. The GC will run
automatically, when necesssary.

If you feel that objects are not freed, you should use a Memory Profiler,
like the one from Scitech...

Best regards,
Henning Krause

 

[Guest]

In both versions of the applcaition, I call GC.Collect() immediately after
one report has been generated successfully.

When multi-threaded version of the applcation is being executed, I could see
the usage of memory just keep growing and growing. But when single-threaded
version of the application is runing, I could see the usage of memory
decrease a lot immediately after a report has sbeen generated successfully.

 

[Guest]

Unfortunately, without calling GC.Collect specifically in the code, the OUT
OF MEMORY would, often time, crash the applcaition process entirely, leaving
the application no way to catch the errors and act accordingly.

ActiveReport is used to generate reports in PDF and XLS, and most of the
reports contain 2000+ pages.

I think that this might be the type of application, which might need to call
GC.Collect to make the memory footprint as small as possible.

Thanks.

Don Lin

 

[Chris Mullins [MVP]]

You've probably got a number of things going on.

1 - You really shouldn't be calling GC.Collect yourself. I know you've heard
this from other people, and probably thought about it a whole lot, but you
really shouldn't be doing it. In several years of building .Net server
applications, I've only come across 1 place where calling GC.Collect has
been appropiate.

2 - 8G of memory doesn't do you any good in .Net. From your post, I'm going
to assume you're running x86 code. This means your code only gets about 1.3
gigs of memory to play with before it hits OOM. You don't get any more than
that, in x86 land. (Does the /3GB switch do anything with the CLR? I don't
think it does...) For more memory, you've got to switch over to x64 or
Itanium.

3 - What CLR are you running? The Server CLR or the Workstation CLR? You'll
get better performance out of the Server CLR, but that means your heap will
be divided up between 4 processors, which means each individual managed heap
will only be 300 megs or so. Before you answer, "I don't know." go look this
up in Google....

4 - Are you doing I/O of some sort? Lots of SQL queries? Network I/O? File
I/O? COM Interpop Calls? I would guess you're doing something. The problem
with this is that you're almost certainly (inadvertantly) pinning chunks of
memory all over the place, thereby creating heap fragmentation. This makes
you run out of memory very quickly, even though it looks like you're only
using 200 or 300 megs of memory.

To debug this, you need to:
1 - Break out a memory profiler and poke around. I like Scitech's memory
profiler (free trial, cheap license, awesome product), but there are a
number of other options.

2 - When your get OOM, use ADPlus to generate a Dump file. Load the Dump
into WinDBG + Son of Strike, and look for heap fragmentation. Details on how
to do this at: http://www.coversant.net/Default.aspx?tabid=88&EntryID=28

FWIW, GC.Collect can be called from any thread. The places where I call it
for heap compaction prior to Network I/O buffer allocation is all
multi-threaded and works just fine...

 

[Henning Krause [MVP - Exchange]]

Hello,

have you monitored the process with a memory profiler?

Henning

 

[Henning Krause [MVP - Exchange]]

Hello,

2 - 8G of memory doesn't do you any good in .Net. From your post, I'm
going to assume you're running x86 code. This means your code only gets
about 1.3 gigs of memory to play with before it hits OOM. You don't get
any more than that, in x86 land. (Does the /3GB switch do anything with
the CLR? I don't think it does...) For more memory, you've got to switch
over to x64 or Itanium.

Should work, if you patch the exe file (there is a program which does this
in the Visual Studio C++ tools).

Best regards,
Henning Krause

 

[Guest]

The reason that forcing a garbage collection makes any difference at
all, is most likely that it starts finalizing objects that you haven't
disposed of properly.

Every class that implements the IDisposable interface has a Dispose
method that you should call when you are done using the object. If you
don't, the object has to wait for a garbage collection to identify it as
an object that needs to be finalized. Then a background thread will
finalize the objects one at a time so that they can bee freed by a later
garbage collection.

The finalizer is only a backup for the Dispose method, that is used if
the program fails to dispose iof it properly. It's much more expensive
to free objects that has to be finalized. If you call Dispose, it turns
the object into a managed resource that easily can be freed by the
garbage collector.

(For database connections and datareaders, you can either call Dispose
or Close, as they do the same work. This is pointed out in the
documentation.)