Sharon said:
Ok, that's good.
But what about the Windows Task Manager? Can I or can't I learn how much
memeory the application is consumimg? is it the 'Meme Usage' column? is
the
'Meme Usage' + 'VM Size' columns?
No you can't simply add these two to calculate the amount of memory your
application is consuming, and you can't actually calculate the exact amount
of memory taken by a process using the performance counters but you can get
an estimate, but before we go and have a look at it, I would suggest you to
forget about Taskman and use perfmon.exe instead. The reason is that taskman
only tells you part of the story and the names of the counters are a bit
confusing; what is called "Mem Usage" in taskman is the "Process -
Workingset" size in perfmon, and "VM Size" is perfmon's "Process - Private
bytes" counter, the values of these counters in taskman are correct but they
might be misleading.
The "WorkingSet" of a process is the amount of RAM actually occupied by the
pages of a process, however, while this includes pages private to the
process it also consists of pages shared with other processes (the shared
pages).
The "Private bytes" represents the total of the pages, private to the
process, that have been committed (and possibly modified) by the program and
as such have been reserved on the paging file (Private bytes == Page File
bytes in perfmon). Part of it can be in the WS while the remaining pages are
effectively paged-out. Note that the GC heap memory is strictly private and
as such part of "private bytes", also the space taken by the assemblies
loaded and the JIT'd code is part of the "private bytes".
So, while it's obvious that you can't add the WS and Private bytes to
calculate the real amount of memory ACTUALLY committed by the process,
because you don't know exactly how much private bytes are part of the WS,
you can get a pretty good idea about your actual memory consumption by
looking at the "Virtual Bytes" counter.
But there is more, even if you have sufficient VM (that is FREE RAM + FREE
page file space) to accomdate the 2GB maximum of user space (or 3GB on 32
bit windows with PAE), it doesn't mean that you can (nor should) allocate
all free process space for managed objects, the reason is 'memory
fragmentation'.
When the CLR is loaded and ready to execute program code, you are left with
~1.6 GB free VM space (your milleage may vary depending on OS and CLR
version) and you can use almost all of it for small objects (<85Kb), but
with Large Objects things might go wrong because this 1.6Gb space is
fragmented. For instance while it might be possible to create 16 instances
of a say an array of 100MB , it's just impossible to create 2 objects of
800MB or 3 objects of 500MB. Note that here I'm talking about a simple pure
managed application, things might get worse when you have to deal with
unmanaged code in your application (COM or PInvoke interop). Unmanaged code
can also allocate memory from the free (process) heap space and this
allocation can further fragment the heap such that there might be less space
left in a contigious block to store large objects.
Conclusion, while it's hard to calculate the exact amount of memory consumed
by a process, it's nearly impossible to say how much memory there is left in
a contiguous block, and that's the point that troubles a lot of people, just
because they don't know/care about the hidden cost associated with some of
the container classes in a highly virtualized frameworks like (but not
limited to) .NET and Java, they just create Dataset with tons tables and
rows comming from SQL like stores and they are supprised that they run out
of memory (not to mention the disastrous performance because of paging).
So while might be important to keep an eye at your memory consumption, the
question to know exactly how much there is free might be moot in the
presence of large objects. So I would suggest you apply a good design that
is based mainly on small classes and watch out for things like ArrayLists
and (worse) highly sophisticated things like DataSets (they have a large
hidden cost!) that get larger than a few megabyte. Watch out for Interop
scenarios, be careful with third party ActiveX controls or .NET controls ,
most of them are simple AX wrappers and some are real crap some are simply
not usable in your scenario (STA objects in a windows service or webservice
for instance). Be careful using PInvoke interop to call third party (free
stuff or public domain), some libraries assume they can allocate all memory
they want not to mention their weird base addresses (the address where they
load in VAS) so they further fragment the available memory.
Willy.