Is this a bug in .NET garbage collection?

[Guest]

Thanks, Chris. I think it might make sense to try ArrayList... I will try it.
This post is helpful.

Yang.

Hi Yang

You might be right that we need to redesign the project becasue .NET does
not have the ability to handle huge arrays correctly. If I interpret your
saying about redesign, I would say you are telling us NOT to use huge arrays.

I don't think I ever said .NET doesn't handle huge arrays correctly
What I mean is, the way you should design your managed application, with respect to memory allocations, is very different than unmanaged. In C++ you can allocate a huge array,
and as soon as you're done with it delete it, and the memory is freed. Obviously with a GC you can't do that, and lage objects are handled differently, since it's so expensive to
create, deallocate, re-create, etc.

Instead of a huge contiguous chunk of memory, is it possible for you to use an ArrayList, or some other dynamic structure? That way it won't all be allocated into the LOH, and be
collected earlier, possibly in pieces. It's hard to give concrete advice without seeing your application, but in general you want to avoid huge memory allocations in .NET because
it limits what the GC can do with it (mainly for perf reasons). See Rico Mariani's blog for more on GC and Perf: http://weblogs.asp.net/ricom/

I have describle the algorithm about how to handle this situation ideally in
my previous posts. You just simply do not want to admit I am right and you
just do not want to say .NET needs improvement - this is the part really
confused me...

Of course .NET needs improvement, that's why I have a job
But I think your algorithm doesn't take into account the other issues involved in a general garbage collector's implementation. There's an excellent book on garbage collection
technology by Jones and Lins that I would recommend, if you're interested in learning more.

-Chris

--

This posting is provided "AS IS" with no warranties, and confers no rights. Use of included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm

Note: For the benefit of the community-at-large, all responses to this message are best directed to the newsgroup/thread from which they originated.

 

[Guest]

I have found running your own force garbage collection thread in the
background can help even though it will take up a little CPU. You can play
around with the sleep time.

static void Main()
{
Thread GCThread = new System.Threading.Thread(new ThreadStart(RunGC));
GCThread.Start();
Application.Run(new Form1());
GCThread.Abort();
}

static private void RunGC()
{
do
{
GC.Collect();
Thread.Sleep(new TimeSpan(0, 0, 0, 10));
} while (1 != 0);
}

private void button1_Click(object sender, System.EventArgs e)
{
byte[] aaa = new byte[300000000];
for(int i=0; i<aaa.Length; i++) aaa = 10;
}

 

[Jon Skeet [C# MVP]]

I have found running your own force garbage collection thread in the
background can help even though it will take up a little CPU. You can play
around with the sleep time.

The garbage collector doesn't really run in a background thread - it
needs to stop *all* managed threads while it runs.

I would personally advise against forcing garbage collections in almost
all situations.

 

[Chris Lyon [MSFT]]

JT,

I can't stress enough that what you suggested is a BAD IDEA. Not only are you hurting performance, but you're messing with the GC's own collection scheme. What is
happening by continuously calling GC.Collect(), is that you're forcing promotion of objects from generation 0 to generation 2, that would otherwise be collected.

The GC tunes itself to your memory allocation patterns and determines the best times to Collect to minimize performance hits, and maximize memory usage. There are very
few times where GC.Collect() should be called in production code.

For more information about GC collection perf hits, see Rico Mariani's blog entry:
http://weblogs.asp.net/ricom/archive/2003/12/04/41281.aspx

-Chris


--------------------

I have found running your own force garbage collection thread in the
background can help even though it will take up a little CPU. You can play
around with the sleep time.

static void Main()
{
Thread GCThread = new System.Threading.Thread(new ThreadStart(RunGC));
GCThread.Start();
Application.Run(new Form1());
GCThread.Abort();
}

static private void RunGC()
{
do
{
GC.Collect();
Thread.Sleep(new TimeSpan(0, 0, 0, 10));
} while (1 != 0);
}

private void button1_Click(object sender, System.EventArgs e)
{
byte[] aaa = new byte[300000000];
for(int i=0; i<aaa.Length; i++) aaa = 10;
}

I found a very strange behavior when I write a C# windows application. If I
allocate a huge chunk of memory by an array, the memory will never be
released by .NET.

The problem can be demostrated as follows:

1. Just create the simplest windows form project and add a button and
handler like this:

private void button1_Click(object sender, System.EventArgs e)
{
byte[] aaa = new byte[300000000];
for(int i=0; i<aaa.Length; i++)
aaa = 10;
}

2. After executing the above code, I observed the memory in Task Manager.
The Commit Charge Total jumped to 300M and the MEM Usage shows the same
thing. The problem is whatever you do the memory usage can never drop back. I
have tried to open many other applications and NOTHING can make the memory
get released.

3. The variable aaa has gone out of scope, so it does not make any sense
that the .NET still holds the memory. Now the performance of the whole system
is downgraded.

4. The only thing I can make the memory release to Windows is to call
GC.Collect(). Is this the only way to release that memory? Do I suppose to do
this?

Can anybody confirm if this is a bug in .NET? Appearantly it does not make
sense to hold the memory FOREVER even if nobody uses it.

Thanks,

Yang.

--

This posting is provided "AS IS" with no warranties, and confers no rights. Use of included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm

Note: For the benefit of the community-at-large, all responses to this message are best directed to the newsgroup/thread from which they originated.

 

[Guest]

Yes I know maximize memory usage is the problem...Joking

JT,

I can't stress enough that what you suggested is a BAD IDEA. Not only are you hurting performance, but you're messing with the GC's own collection scheme. What is
happening by continuously calling GC.Collect(), is that you're forcing promotion of objects from generation 0 to generation 2, that would otherwise be collected.

The GC tunes itself to your memory allocation patterns and determines the best times to Collect to minimize performance hits, and maximize memory usage. There are very
few times where GC.Collect() should be called in production code.

For more information about GC collection perf hits, see Rico Mariani's blog entry:
http://weblogs.asp.net/ricom/archive/2003/12/04/41281.aspx

-Chris


--------------------

I have found running your own force garbage collection thread in the
background can help even though it will take up a little CPU. You can play
around with the sleep time.

static void Main()
{
Thread GCThread = new System.Threading.Thread(new ThreadStart(RunGC));
GCThread.Start();
Application.Run(new Form1());
GCThread.Abort();
}

static private void RunGC()
{
do
{
GC.Collect();
Thread.Sleep(new TimeSpan(0, 0, 0, 10));
} while (1 != 0);
}

private void button1_Click(object sender, System.EventArgs e)
{
byte[] aaa = new byte[300000000];
for(int i=0; i<aaa.Length; i++) aaa = 10;
}

I found a very strange behavior when I write a C# windows application. If I
allocate a huge chunk of memory by an array, the memory will never be
released by .NET.

The problem can be demostrated as follows:

1. Just create the simplest windows form project and add a button and
handler like this:

private void button1_Click(object sender, System.EventArgs e)
{
byte[] aaa = new byte[300000000];
for(int i=0; i<aaa.Length; i++)
aaa = 10;
}

2. After executing the above code, I observed the memory in Task Manager.
The Commit Charge Total jumped to 300M and the MEM Usage shows the same
thing. The problem is whatever you do the memory usage can never drop back. I
have tried to open many other applications and NOTHING can make the memory
get released.

3. The variable aaa has gone out of scope, so it does not make any sense
that the .NET still holds the memory. Now the performance of the whole system
is downgraded.

4. The only thing I can make the memory release to Windows is to call
GC.Collect(). Is this the only way to release that memory? Do I suppose to do
this?

Can anybody confirm if this is a bug in .NET? Appearantly it does not make
sense to hold the memory FOREVER even if nobody uses it.

Thanks,

Yang.

--

This posting is provided "AS IS" with no warranties, and confers no rights. Use of included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm

Note: For the benefit of the community-at-large, all responses to this message are best directed to the newsgroup/thread from which they originated.