Is this a true statement?

[Chris]

If I use Perfmon to look at the number of GC Handles for my application, and
it stays about the same over several hours, I don't have an issue with
freeing up unmanaged resources in my app. Is that true?

I have an application that is using up memory. Using Perfmon, # Bytes in
all heaps stays around 2.5MB after running for hours. TaskMgr shows my
process to be using ~75MB and growing steadily. If the memory isn't being
used by any of the .Net heaps, and I'm not increasing the number of GC
Handles, where the heck is the memory being consumed?

Could someone please take pity on me and point me toward the, probably
simple, explanation for this? I keep thinking I understand the way garbage
collection works, and the promotion from gen0 to gen1 to gen2, but I must be
missing something fundamental here.

TIA,
Chris

 

[Larry Brasfield]

If I use Perfmon to look at the number of GC Handles for my application, and
it stays about the same over several hours, I don't have an issue with
freeing up unmanaged resources in my app. Is that true?

I do not see any logical way to equate 'GC Handles'
to 'unmanaged resources'. The number of each is, in
general, independent. So, no, your assertion is not true.

I have an application that is using up memory. Using Perfmon, # Bytes in
all heaps stays around 2.5MB after running for hours. TaskMgr shows my
process to be using ~75MB and growing steadily. If the memory isn't being
used by any of the .Net heaps, and I'm not increasing the number of GC
Handles, where the heck is the memory being consumed?

Maybe you are leaking OS handles.

Could someone please take pity on me and point me toward the, probably
simple, explanation for this? I keep thinking I understand the way garbage
collection works, and the promotion from gen0 to gen1 to gen2, but I must be
missing something fundamental here.

Perhaps you have assumed that worries about resource
leaks are history, now that a GC system is here. Sorry
to be the bearer of bad news, but 't'aint so. From your
description, I can divine that you are using some kind of
resource other than CLR memory, since that is pretty
much the only resource that is (almost) automatically
managed without programmer effort.

My bet would be that, when you get this fixed, a few
more finalizers, IDisposable implementation, and C#
'using' constructs will be in place.

TIA,
Chris

Good luck. Let us know what you find.

 

[Bob Grommes]

GC Handles doesn't relate to unamanged resources.

However, TaskManager just shows the working set, which can be misleading.
In other words it shows memory that Windows hasn't felt any need to reclaim
as yet, but this doesn't necessarily mean it's not been released by your
app.

Do you notice the working set as reported by TaskManager going down when you
minimize the app? If the memory consumption continues to rise overall over
time even when you periodically minimize and restore the application window,
then you may not be releasing unmanaged resources.

--Bob

 

[Willy Denoyette [MVP]]

If the managed heaps are NOT growing (did you look at the LH heap!!), the
memory increase is due to the growing of the non managed portion.
This part of the process memory is taken by all code modules and data, that
is:
1 - all loaded modules (managed and unmanaged DLL's)
2 - all memory allocated using the Marshal class unmanaged allocators like
Marshal.AllocHGlobal and Marshal,AllocCoTaskMem, Marshal.Copy etc.
3 - all memory allocated using native OS allocators from unmanaged code.
4 - all memory used by your thread stacks (up to 1MB per thread).

So if your unmanaged memory consumption keeps growing, check for COM interop
leaks or PInvoke leaks. Release your COM objects when done with it. When
using PInvoke make sure you don't have to free unmanaged memory, and don't
use PInvoke to call undocumented functions or functions that possibly
allocate without any means for the caller to de-allocate.
If you are using Marhal class allocators make sure you free the allocated
memory when done.
Keep the number of threads (manual and threadpool) running (or waiting) as
low as possible, make sure some of them aren't blocked waiting for an event
that won't happen.

Willy.

 

[Chris]

I do not see any logical way to equate 'GC Handles'
to 'unmanaged resources'. The number of each is, in
general, independent. So, no, your assertion is not true.

Thanks Larry. I'm just going by what I get when I hit "Explain" in Perfmon
for the # GC Handles counter. Here is what it says...

"This counter displays the current number of GC Handles in use. GCHandles
are handles to resources external to the CLR and the managed environment.
Handles occupy small amounts of memory in the GCHeap but potentially
expensive unmanaged resources."

-Chris

 

[Chris]

GC Handles doesn't relate to unamanged resources.

However, TaskManager just shows the working set, which can be misleading.
In other words it shows memory that Windows hasn't felt any need to reclaim
as yet, but this doesn't necessarily mean it's not been released by your
app.

Do you notice the working set as reported by TaskManager going down when you
minimize the app? If the memory consumption continues to rise overall over
time even when you periodically minimize and restore the application window,
then you may not be releasing unmanaged resources.

Hi Bob,

My app is running as a Windows Service. It will continue to consume memory
until the system is unusable.

-Chris

 

[Chris]

If the managed heaps are NOT growing (did you look at the LH heap!!), the
memory increase is due to the growing of the non managed portion.

Yep, I did check the large object heap and it stays very small. I came to
the same conclusion on the memory being used by unmanaged code and just
wanted to run it by others to be sure.

This part of the process memory is taken by all code modules and data, that
is:
1 - all loaded modules (managed and unmanaged DLL's)
2 - all memory allocated using the Marshal class unmanaged allocators like
Marshal.AllocHGlobal and Marshal,AllocCoTaskMem, Marshal.Copy etc.
3 - all memory allocated using native OS allocators from unmanaged code.
4 - all memory used by your thread stacks (up to 1MB per thread).

So if your unmanaged memory consumption keeps growing, check for COM interop
leaks or PInvoke leaks. Release your COM objects when done with it. When
using PInvoke make sure you don't have to free unmanaged memory, and don't
use PInvoke to call undocumented functions or functions that possibly
allocate without any means for the caller to de-allocate.
If you are using Marhal class allocators make sure you free the allocated
memory when done.
Keep the number of threads (manual and threadpool) running (or waiting) as
low as possible, make sure some of them aren't blocked waiting for an event
that won't happen.

Thanks for all the above, Willy. My app is actually fairly simple. It uses
an FTP client library to check a remote site every couple minutes and see if
a certain file has been updated. If it has, I download it, parse it, and
write the info to a database. I don't directly interact with any unmanaged
code, but I know the DB libraries and IO calls do.

Most of the classes in this app have actually been running in a Production
environment 24/7 for months now. I recently added some functionality, did a
little refactoring, and changed the DB drivers I use to communicate with our
Sybase DB. I suspect my change from using ODBC drivers to a ADO .Net Data
Provider from Sybase, is where my problem lives, but I can't figure out what
I'm doing wrong. It looks like I'm calling dispose on all my commands,
connections and data adapters. I do allow a few tables to live, and be
recreated with new, each time I process the downloaded file.

-Chris

 

[Willy Denoyette [MVP]]

Yep, I did check the large object heap and it stays very small. I came to
the same conclusion on the memory being used by unmanaged code and just
wanted to run it by others to be sure.


Thanks for all the above, Willy. My app is actually fairly simple. It
uses
an FTP client library to check a remote site every couple minutes and see
if
a certain file has been updated. If it has, I download it, parse it, and
write the info to a database. I don't directly interact with any
unmanaged
code, but I know the DB libraries and IO calls do.

Most of the classes in this app have actually been running in a Production
environment 24/7 for months now. I recently added some functionality, did
a
little refactoring, and changed the DB drivers I use to communicate with
our
Sybase DB. I suspect my change from using ODBC drivers to a ADO .Net Data
Provider from Sybase, is where my problem lives, but I can't figure out
what
I'm doing wrong. It looks like I'm calling dispose on all my commands,
connections and data adapters. I do allow a few tables to live, and be
recreated with new, each time I process the downloaded file.

-Chris

Chris,

If you suspect the Sysbase provider, couldn't you write a simple testcase
that exercises the same DB calls against a test DB using the same provider?
That way it's much simpler to debug and watch your memory consumption using
native tools like windbg and VADump.

Willy.

 

[Chris]

Chris,

If you suspect the Sysbase provider, couldn't you write a simple testcase
that exercises the same DB calls against a test DB using the same provider?
That way it's much simpler to debug and watch your memory consumption using
native tools like windbg and VADump.

Willy.

Yes, I think that's what I need to do now. I'm such a newbie that I didn't
know about vadump and windbg though. I've been giving myself a crash course
today but I still have a whole lot of reading to do. You wouldn't happen to
know how to translate the stats below into meaningful class names would you?
The 33652 pages in private Heap 3 would appear to be my problem, but I
haven't figured out yet where to go from here.

Thanks again for your help with this.

-Chris

Heap Working Set Contributions
237 pages from Process Heap (class 0x00000000)
0x00130000 - 0x00230000 215 pages
0x08050000 - 0x08150000 22 pages
0 pages from UNKNOWN Heap 0 (class 0x00008000)
0x00230000 - 0x00240000 0 pages
0 pages from Private Heap 1 (class 0x00001000)
0x002F0000 - 0x00300000 0 pages
3 pages from Private Heap 2 (class 0x00001000)
0x00510000 - 0x00520000 3 pages
33652 pages from Private Heap 3 (class 0x00001000)
0x00520000 - 0x00530000 1 pages
0x06890000 - 0x06990000 232 pages
0x086D0000 - 0x088D0000 511 pages
0x089D0000 - 0x08DD0000 1023 pages
0x08ED0000 - 0x096D0000 2047 pages
0x09BD0000 - 0x0ABD0000 4095 pages
0x0BED0000 - 0x0DED0000 8190 pages
0x11010000 - 0x15010000 16383 pages
0x17910000 - 0x1F910000 1170 pages
3 pages from Private Heap 4 (class 0x00001000)
0x06D40000 - 0x06D50000 3 pages
1 pages from Private Heap 5 (class 0x00001000)
0x07520000 - 0x07560000 1 pages
3 pages from Private Heap 6 (class 0x00001000)
0x07560000 - 0x07660000 3 pages
6 pages from Private Heap 7 (class 0x00001000)
0x07EA0000 - 0x07EE0000 6 pages
4 pages from Private Heap 8 (class 0x00001000)
0x07EE0000 - 0x07F20000 4 pages
1 pages from Private Heap 9 (class 0x00001000)
0x07F30000 - 0x07F40000 1 pages
9 pages from Private Heap 10 (class 0x00001000)
0x08040000 - 0x08050000 9 pages
2 pages from Private Heap 11 (class 0x00001000)
0x08150000 - 0x08190000 2 pages

 

[cshaw]

If I use Perfmon to look at the number of GC Handles for my application, and
it stays about the same over several hours, I don't have an issue with
freeing up unmanaged resources in my app. Is that true?

I have an application that is using up memory. Using Perfmon, # Bytes in
all heaps stays around 2.5MB after running for hours. TaskMgr shows my
process to be using ~75MB and growing steadily. If the memory isn't being
used by any of the .Net heaps, and I'm not increasing the number of GC
Handles, where the heck is the memory being consumed?

Could someone please take pity on me and point me toward the, probably
simple, explanation for this? I keep thinking I understand the way garbage
collection works, and the promotion from gen0 to gen1 to gen2, but I must be
missing something fundamental here.

Looks like I finally got this beat, after spending my entire weekend on it.
The DataReader implementation, in the Sybase ADO .Net Data Provider,
apparently doesn't clean up after itself if you supply a commandbehavior
when constructing it. I had no problem with what I was doing with the
OdbcDataReader but started seeing this memory leak when I moved it to the
Sybase specific version. Took me a LONG time to decide to try the default
constructor though.

-Chris