GetTotalMemory, GlobalMemoryStatus

[Daniel Moth]

Hi

Hopefully someone can help me clarify some memory issues and in particular
the difference between GetTotalMemory & GlobalMemoryStatus...

Apparently GetTotalMemory(false) retrieves the number of bytes currently
thought to be allocated. I assume the bytes returned reflect only the
managed bytes(objects) that *my* managed code has allocated, is that
correct? In other words if I have PInvoke calls that allocate memory that
would not be included, right? Is there anything else GetTotalMemory excludes
that one would assume is included?

I observe a large difference between what GetTotalMemory returns and what
GlobalMemoryStatus reports (taking in account the memory used by the system.
No other apps running). I am guessing this is because the GlobalMemoryStatus
includes the JITed code as used memory whereas GetTotalMemory doesn't; is
that correct? Still the discrepancy is large so there must be other factors
that I am not aware of... For example when GetTotalMemory returns 446KB
GlobalMemoryStatus (or indeed the memory control panel or even the remote
system info from eVC) shows 9072KB. What extra bytes is it including?

It seems even the simplest CF app (8KB exe) consumes over 1MB of memory. Is
that right or is there some flaw in my approach? Also GC.Collect affect the
results of GetTotalMemory but not the results of GlobalMemoryStatus... is
that in line with your observations?

Note that when the above results are observed, I then leave the unit/app
running for at least 1 hour (in idle mode) and the results remain the
same... I have also used the slider to effectively force a memory load of
over 95%...

Cheers
Daniel

 

[Paul G. Tobey [eMVP]]

There are *always* several applications running and they use a fair amount
of memory. The kernel (nk.exe), the windowing system (gwes.exe), the device
manager (device.exe), the file system manager (filesys.exe), etc. are all in
there. As I recall, the RAM filesystem is allocated by the file system
manager. It's possible that the memory that you have allocated to that is
contributing a significant fraction of what you are seeing allocated.

Paul T.

 

[Daniel Moth]

OK, sorry, when I said "no other apps" I meant no apps of mine (thought the
"taking in account memory used by the system" bit covered that)...

In any case, if we open explorer pointing to the folder where the exe
resides and then check the memory in use (from the memory panel) we get a
value A. If we then launch our exe and go back to the panel we see a value
B. If C=B-A, isn't it fair to say that C is the amount of bytes used by our
app/process?

If not then please let me know what I am missing?

If it is, then (some of) my original questions were:
1) Why is C=9072KB when GetTotalMemory(false) returns 446KB
2) How come the simplest CF app (8KB exe) shows C=over 1MB
3) After a GC.Collect, GetTotalMemory diminishes but C remains constant.
Why?

Cheers
Daniel

PS
I am not positive I know what you mean by this:

If you mean the size of my binaries they are 1MB in total so a long way to
almost 9MB isn't it?

 

[Paul G. Tobey [eMVP]]

I would expect that GetTotalMemory( false ) would return the total dynamic
(and maybe static), *data* memory allocations for your process, but it might
easily not include memory allocated by the run-time environment for things
like the JIT compiler, the code itself for your program, as well as the
system class code, etc. That is, in C++, C would include the amount of RAM
dedicated to the actual code for the loaded classes, while GetTotalMemory(
false ) would likely only include those *data* items actually allocated by
your code via new().

My comment about RAM filesystem, etc. was covered by your actions to get a
value for A.

Paul T.

 

[Daniel Moth]

Hmm... interesting...

I suspect you are right about GetTotalMemory(false) returning the data items
only (which is what my leading question of my original email was trying to
get confirmation to)...

Since for the simple (8KB exe) CF app, the value C=1MB then we could deduct
that the JIT compiler and other standard memory allocations needed by the
runtime are covered by 1MB [1]

Based on [1] we are still missing (in my specific scenario) over 7MB, which
so far we think accounts for the code (both own (1MB of binaries) and
Microsoft's). It would be useful at this point to get definite confirmation
of this plus to understand the relationship between number of bytes of C#/VB
code versus number of bytes taken at runtime...

I still find [1] itself a huge jump.

My other question remains un-tackled.Not only we should be able to observe the same reduction that GetTotalMemory
reports but since a collection may include pitching of jitted bytes I would
definitely expect it to affect C (the number of bytes returned by
GlobalMemoryStatus).

Cheers
Daniel

 

[Paul G. Tobey [eMVP]]

I can advance a theory about why GC doesn't return the memory to the system,
but I don't know for sure. It would be interesting to see if allocating a
lot of things to the point where the GC grabs a significant amount of extra
RAM from the system and then releasing those allocations would return it to
the system. It may just be that the GC grabs a significant amount of RAM by
default (or at certain allocation sizes or something), and continues to
manage that memory itself (I'm sure that would make allocations faster). It
seems like there would be some way to force release of everything not
actually containing an active allocation, but maybe not. C++ does something
like this in the run-time library, by the way. When you do new enough, the
run-time will need more memory, so it will do a LocalAlloc() to grab some
more and then sub-allocate that for the new requests. However, unless you
deallocate *all* of those suballocations in that one block (which you don't
have any visibility into), it will never do a LocalFree()...

Paul T.

Hmm... interesting...

I suspect you are right about GetTotalMemory(false) returning the data items
only (which is what my leading question of my original email was trying to
get confirmation to)...

Since for the simple (8KB exe) CF app, the value C=1MB then we could deduct
that the JIT compiler and other standard memory allocations needed by the
runtime are covered by 1MB [1]

Based on [1] we are still missing (in my specific scenario) over 7MB, which
so far we think accounts for the code (both own (1MB of binaries) and
Microsoft's). It would be useful at this point to get definite confirmation
of this plus to understand the relationship between number of bytes of C#/VB
code versus number of bytes taken at runtime...

I still find [1] itself a huge jump.

My other question remains un-tackled.Not only we should be able to observe the same reduction that GetTotalMemory
reports but since a collection may include pitching of jitted bytes I would
definitely expect it to affect C (the number of bytes returned by
GlobalMemoryStatus).

Cheers
Daniel

I would expect that GetTotalMemory( false ) would return the total dynamic
(and maybe static), *data* memory allocations for your process, but it might
easily not include memory allocated by the run-time environment for things
like the JIT compiler, the code itself for your program, as well as the
system class code, etc. That is, in C++, C would include the amount of RAM
dedicated to the actual code for the loaded classes, while GetTotalMemory(
false ) would likely only include those *data* items actually allocated by
your code via new().

My comment about RAM filesystem, etc. was covered by your actions to get a
value for A.

Paul T.

(thought
the

get

a by
our

way

to are
all that

 

[Chris Tacke, eMVP]

Even a simple managed app must launch the EE and load many of the CF base
class libraries, so 1MB is no surprise to me. Since they're in the GAC,
launching a *second* simple app will probably only use about another 8k.

--
Chris Tacke, eMVP
Co-Founder and Advisory Board Member
www.OpenNETCF.org
---
Windows CE Product Manager
Applied Data Systems
www.applieddata.net

 

[Daniel Moth]

Nice theory... That would explain a few things... I wonder if, after
grabbing a significant amount of memory, it grabs a further significant
amount when the first one fills up and *then* never frees either until *all*
of those suballocations are GC-able...

Hopefully some MS person will jump in to confirm the theories/speculations
that this thread contains...

In the mean time thanks for your insights... I'll dig further and post any
new findings (in the form of questions no doubt

Cheers
Daniel

I can advance a theory about why GC doesn't return the memory to the system,
but I don't know for sure. It would be interesting to see if allocating a
lot of things to the point where the GC grabs a significant amount of extra
RAM from the system and then releasing those allocations would return it to
the system. It may just be that the GC grabs a significant amount of RAM by
default (or at certain allocation sizes or something), and continues to
manage that memory itself (I'm sure that would make allocations faster). It
seems like there would be some way to force release of everything not
actually containing an active allocation, but maybe not. C++ does something
like this in the run-time library, by the way. When you do new enough, the
run-time will need more memory, so it will do a LocalAlloc() to grab some
more and then sub-allocate that for the new requests. However, unless you
deallocate *all* of those suballocations in that one block (which you don't
have any visibility into), it will never do a LocalFree()...

Paul T.

Hmm... interesting...

I suspect you are right about GetTotalMemory(false) returning the data items
only (which is what my leading question of my original email was trying to
get confirmation to)...

Since for the simple (8KB exe) CF app, the value C=1MB then we could deduct
that the JIT compiler and other standard memory allocations needed by the
runtime are covered by 1MB [1]

Based on [1] we are still missing (in my specific scenario) over 7MB, which
so far we think accounts for the code (both own (1MB of binaries) and
Microsoft's). It would be useful at this point to get definite confirmation
of this plus to understand the relationship between number of bytes of C#/VB
code versus number of bytes taken at runtime...

I still find [1] itself a huge jump.

My other question remains un-tackled.

3) After a GC.Collect, GetTotalMemory diminishes but C remains constant.
Why?

Not only we should be able to observe the same reduction that GetTotalMemory
reports but since a collection may include pitching of jitted bytes I would
definitely expect it to affect C (the number of bytes returned by
GlobalMemoryStatus).

Cheers
Daniel

I would expect that GetTotalMemory( false ) would return the total dynamic
(and maybe static), *data* memory allocations for your process, but it might
easily not include memory allocated by the run-time environment for things
like the JIT compiler, the code itself for your program, as well as the
system class code, etc. That is, in C++, C would include the amount

of
RAM

dedicated to the actual code for the loaded classes, while GetTotalMemory(
false ) would likely only include those *data* items actually

allocated

by

get

a get to
that way only
the remain
the memory
load

 

[Daniel Moth]

Hi Chris

Although I only have one managed app running in the scenario I am trying to
debug, I was surprised with your comment about the GAC since I understand
the GAC is a mechanism for "footprint reuse" rather than runtime memory
reuse...

So, for info, I run the simple .NETcf app and every instance does indeed
consume this extra 1MB asI expected that I personally do find large but that
is just a difference of perception (in reality what I find large is the
7-9MB my real app grabs of which my data is under 1/2MB)

Cheers
Daniel

PS
The data from my unit/platform
A = Memory in use with explorer running pointing to folder with CF app
B = Memory in use after launching app
C = Memory in use after launching second instance of app
D = Memory in use after closing second app
E = Memory in use after closing first app

A = 3832
B = 4988
C = 6152
D = 4992
E = 3840

 

[Chris Tacke, eMVP]

Now that's interesting and unexpected.

 

[Alex Feinman [MVP]]

Aren't we forgetting about the heap?