Hashtable and ArrayList with bad memory usage?

[Matthias Kwiedor]

Hi@all!

I have a app (c#) where i load up a external dll (managed code from c#)
with a small arraylist and hashtable in two routines (about 40000 objects
in each arraylist and hashtable).

If i load the dll everything is fine and when i load then the objects into
an Hashtable or Arraylist of the main program it freezes 5MB each which i
don't get free. The weird think is, when i reload the Tables a second time
on a new local variable block it freeze no more memory!?!

I load the dll everytime i need it and dispose them with a custom dispose
destructor after i have load the vars into the new one!

After trying some thinks i heard about a arraylist bug which freeze some
memory and so i changed the List to a string[] - but there was no change -
allways freeze up the 5mb after going up from 19mb to 50mb and then free
most of them to 25mb! (loading both ArrayList and Hashtable ends in 30MB
Memory Usage)

Someone have a idea?


Thanx



Matthias

 

[Jon Skeet [C# MVP]]

I have a app (c#) where i load up a external dll (managed code from c#)
with a small arraylist and hashtable in two routines (about 40000 objects
in each arraylist and hashtable).

If i load the dll everything is fine and when i load then the objects into
an Hashtable or Arraylist of the main program it freezes 5MB each which i
don't get free. The weird think is, when i reload the Tables a second time
on a new local variable block it freeze no more memory!?!

I load the dll everytime i need it and dispose them with a custom dispose
destructor after i have load the vars into the new one!

After trying some thinks i heard about a arraylist bug which freeze some
memory and so i changed the List to a string[] - but there was no change -
allways freeze up the 5mb after going up from 19mb to 50mb and then free
most of them to 25mb! (loading both ArrayList and Hashtable ends in 30MB
Memory Usage)

Someone have a idea?

I don't really know what you mean by "freezing" memory. Could you write
a short but complete program which demonstrates the program? That would
give us a better idea of what you mean.

See http://www.pobox.com/~skeet/csharp/complete.html

 

[Matthias Kwiedor]

Hi Jon,

thanks for your reply!

Here is a code snippet:


dll:
--------------------
using System;
using System.Collections;

namespace dll
{
public class Categorylist
{
private bool disposed = false;

ArrayList a = new ArrayList();
Hashtable h = new Hashtable();

public Categorylist()
{
//
// TODO: Fügen Sie hier die Konstruktorlogik hinzu
//
}

~Categorylist()
{
Dispose(false);
}

public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}

protected virtual void Dispose(bool disposeManagedResources)
{
if (!this.disposed)
{
if (disposeManagedResources)
{
a.Clear();
h.Clear();
GC.Collect();
GC.SuppressFinalize(this);
}
disposed=true;
}
}

public Hashtable CategoryList()
{
h.Clear();


// Fill the Hashtable with datas (20000 datasets)

return (h);
}


public ArrayList CategoryListArray()
{
a.Clear();

// Fill the ArrayList with datas (20000 datasets)

return (a);
}
}
}


main:
----------------
Click on a button (for the ArrayList):

dll.Categorylist catlist = new dll.Categorylist();

ArryList strCat = catlist.CategoryListArray();
MessageBox.Show(strCat.Count.ToString());
strCat = null;
catlist.Dispose();
GC.Collect();



Click on a button (for the Hashtable):

dll.Categorylist catlist = new dll.Categorylist();

Hashtable hashCat = catlist.CategoryList();
MessageBox.Show(hashCat.Count.ToString());
hashCat = null;
catlist.Dispose();
GC.Collect();

----------------------------------------------------------------------

The Application just uses two buttons to fetch the sample data. After
starting the app it needs about 15MB. When pressing one button it raises
to 45MB for holding the Datas into strCat or hashCat, then after setting
the local values to null and dispose the catlist dll it release memory,
but only to 20MB (pressing the first button) or 25MB (pressing both
buttons). When you press every button several times it hold a memory
usage of 25MB with a GC Memory Usage of about 3MB - that makes a diff of
about 7MB which is somewhere. Hope this code helps to understand the
problem. BTW it makes no diference if i dispose the dll or not!


Thanks



Matthias

 

[Jon Skeet [C# MVP]]

The Application just uses two buttons to fetch the sample data. After
starting the app it needs about 15MB. When pressing one button it raises
to 45MB for holding the Datas into strCat or hashCat, then after setting
the local values to null and dispose the catlist dll it release memory,
but only to 20MB (pressing the first button) or 25MB (pressing both
buttons). When you press every button several times it hold a memory
usage of 25MB with a GC Memory Usage of about 3MB - that makes a diff of
about 7MB which is somewhere. Hope this code helps to understand the
problem. BTW it makes no diference if i dispose the dll or not!

How are you measuring memory usage, here? Bear in mind that the CLR
will hold on to more memory than it is actually using, so that it can
allocate more quickly rather than having to ask the OS again.

I see no reason for you to implement IDisposable in your class, by the
way.

Given that your code includes:

// Fill the ArrayList with datas (20000 datasets)

it's rather hard to reproduce your problem. (The fact that you're
talking about a GUI app without giving us all the code for it, or
indeed as far as I can see really needing a GUI app at all doesn't
help.)

If the first paragraph of this reply doesn't help, please reread the
"short and complete programs" web page - if you could post code which
really fulfills everything on there, it would help a lot.

 

[Matthias Kwiedor]

Hi Jon,

sorry for posting parts of the code who doesn't help. I put the whole test
code into one server where you can download it

http://81.89.197.50/graphics/code.zip

I see the 10MB used at the Taskmanager where the App jumps from 16MB to
26MB of memory usage. Maybe i don't understand something at this point

But its scary for a user to see that a application uses 50 to 60MB of
memory while the GC of the Application just uses 6 to 7MB!


Thanks again




Matthias

 

[Jon Skeet [C# MVP]]

sorry for posting parts of the code who doesn't help. I put the whole test
code into one server where you can download it

http://81.89.197.50/graphics/code.zip

I see the 10MB used at the Taskmanager where the App jumps from 16MB to
26MB of memory usage. Maybe i don't understand something at this point

But its scary for a user to see that a application uses 50 to 60MB of
memory while the GC of the Application just uses 6 to 7MB!

The problem there is that your DLL has *loads* of string literals in.
Those won't get garbage collected, but will be reused. That's why after
the first time of each operation, you don't see the memory going up. If
you read those values in from a file instead of having them embedded in
the DLL as literals, I suspect you wouldn't see nearly as much memory
being used (after the collection has been garbage collected,
obviously).

One thing to note though - my results don't actually show nearly as bad
memory results as yours do: even after loading everything in and going
through several times, the memory before/after click is about 3.2Mb.

Task manager shows 28Mb, but that's not too bad - a lot of that will be
available for any more allocations the app needs to perform.

 

[Matthias Kwiedor]

The problem there is that your DLL has *loads* of string literals in.
Those won't get garbage collected, but will be reused. That's why
after the first time of each operation, you don't see the memory going
up. If you read those values in from a file instead of having them
embedded in the DLL as literals, I suspect you wouldn't see nearly as
much memory being used (after the collection has been garbage
collected, obviously).

One thing to note though - my results don't actually show nearly as
bad memory results as yours do: even after loading everything in and
going through several times, the memory before/after click is about
3.2Mb.

Task manager shows 28Mb, but that's not too bad - a lot of that will
be available for any more allocations the app needs to perform.

Hi Jon,

glad to hear that there is no brain-bug in there I thought i didn't
understand some basic thinks!

I know that this is a huge dataset and thought about to put them into
one file - i will try the performance there for reading the file and put
them into the array or hash.

I must say you are right about the performance - after loading it the
first time (needs about 2 second) it works very fast (as i know from web
caching). Also confirm the 28MB memory usage (after start about 18MB).
After that i installed VMWare and created a Win XP Enviroment with 128MB
of Memory and my main App (which is a little bit bigger) drops from 60mb
to just 30mb of memory usage after opening several programs who eats
memory.

So i come to my final that .net uses alot of memory if the system has
alot of them (and free) and gives the memory free if other applications
need them. Thats nice to give a hand to performance over memory if you
have, but it's maybe a little bit scary to some people if they see that
you App eats 70MB of memory


Thanks for all your work and help



Matthias

 

[Adam D]

I have a app (c#) where i load up a external dll (managed code from
c#) with a small arraylist and hashtable in two routines (about 40000
objects in each arraylist and hashtable).

If i load the dll everything is fine and when i load then the objects
into an Hashtable or Arraylist of the main program it freezes 5MB
each which i don't get free. The weird think is, when i reload the
Tables a second time on a new local variable block it freeze no more
memory!?!

I load the dll everytime i need it and dispose them with a custom
dispose destructor after i have load the vars into the new one!

After trying some thinks i heard about a arraylist bug which freeze
some memory and so i changed the List to a string[] - but there was
no change - allways freeze up the 5mb after going up from 19mb to
50mb and then free most of them to 25mb! (loading both ArrayList and
Hashtable ends in 30MB Memory Usage)

Someone have a idea?

I don't really know what you mean by "freezing" memory. Could you
write a short but complete program which demonstrates the program?
That would give us a better idea of what you mean.

See http://www.pobox.com/~skeet/csharp/complete.html

Don't rely on the Garbage Collector to collect what and when you want it.
Try something similar in C++ (unmanaged of course). You will see
something more along the lines of what you are expecting.