Memory Leak Caused by Using DataTable.Select

[Tristan]

Hi,

I have a DataTable object and need a subset of the rows at runtime so
I perform a DataTable.Select on it to retrieve an array of DataRows. I
am using this over the DataView/Filter approach as it performs
slighltly better in my application.

The problem is that the returned DataRow array does get removed from
memory. I have tried setting all the elements to null, setting the
array variable to null, wrapping the array variable in an IDisposable
class, calling GC.Collect etc. etc. but it still does not get removed
from memory.

Example code below:

DataRow[ ] dr = myDataTable.Select("ProductType = 'Book');

// Perform business logic based on contents of array

dr = null;
GC.Collect(); // Memory used up by dr variable not freed.


Any help appreciated.

Thanks

Tristan.

 

[Cor Ligthert[MVP]]

Tristan,

Be aware that the by you created array is only an array of references.

Therefore setting that to null/nothing means only that the references are
set to null/nothing.

The datarows stay in memory as long as it has a reference to something that
exists or that it self is referencing to something that still exist. (That
is the reason garbaging is not as easy as it looks).

Cor

 

[Cor Ligthert[MVP]]

Tristan,

I forgot to write. A dataset reflects a database.

Cor

 

[Miha Markic]

Hi Tristan,

But how do you know that the array of references doesn't get away?
The only reliable way of knowing that is by using a memory profiler.
Anyway, there is no leak whatsoever in your case.

 

[Cor Ligthert [MVP]]

Wrong message, sorry had to by a message after this (in time)

 

[Tristan]

Hi Cor / Miha

Thanks for the replies. I realise that the array is only holding an
array of references which is why I'm suprised there is a noticable
increase in memory consumption. It seems to be the fact that the array
exists that is what's holding up the additional memory. This
additional memory is on top of that used by the underlying datatable.
I have looked in the CLR Profiler and the Array is still there with
it's own memory. Even when just one row was present in the array
(selected from a table with 60,000 rows) an extra 2MB is getting used
up by calling DataTable.Select, on top of the 16MB that the DataTable
holds. I have a test app which allows me to click a button to display
the total memory being used by the app. Before I do the
DataTable.Select I get 16MB and 18MB after. The test app does nothing
more than read data into a datatable and then peform a
DataTable.Select. No matter what I do, I can't get the memory back
down to 16MB from 18MB after I have finished with the Array returned
from the DataTable.Select.

Thanks
Tristan.

 

[Cor Ligthert [MVP]]

Tristan,

I wrote if it uses a reference or when it is referenced. Are you sure that
this array is not referenced by something as by instance a datasource of a
control which is placed global. Then setting that datasource to null can
probably help you?

Cor

 

[Miha Markic]

Hi Tristan,

I am pretty sure that you are not using/reading CLR Profiler correctly.
Can't help you there since I never used it.

 

[Tristan]

Cor,

Here is simply what my test app is doing, this is all it does as I
have isolated it from my main application. As you can see the DataRow
array doesn't even contain any rows in this case and it is local to
the function it is created in:

private DataTable _dataTable = null;
private void PopulateData()
{
// Connect and setup command (simple select statement
// to retrieve table with 60,000 rows)

SqlDataAdapter adapter = new SqlDataAdapter();
adapter.SelectCommand = connection;

adapter.Fill(_dataTable);
}

private void GetSpecificRows()
{
DataRow[] rows = _dataTable.Select("1 = 2");
rows = null;
GC.Collect();
}

private void ShowMemory
{
long mem = GC.GetTotalMemory(false) / 1048576;
MessageBox.Show("Memory: " + mem.ToString());
}

private void RunTest()
{
PopulateData();

ShowMemory(); // Shows 16MB

GetSpecificRows();

ShowMemory(); // Shows 18MB
}

Thanks

Tristan.

 

[Miha Markic]

Did you read what GetTotalMemory actually returns?
"Return Value
A number that is the *best available approximation* of the number of bytes
currently allocated in managed memory."
Again, if you want to check for memory leaks you should know what you are
after...

 

[Cor Ligthert[MVP]]

Tristan,

Can you remove that GC and set a Thread.Sleep(60000) in it before that test
to see what happens while the program comes in an iddle state?

Cor

 

[Tristan]

Cor,

I have removed the GC.Collect and my test function now looks like:

private void RunTest()
{
PopulateData();

ShowMemory(); // Shows 16MB

GetSpecificRows();

Thread.Sleep(60000);

ShowMemory(); // Shows 18MB
}

Hope that I understood you correctly. It still shows 18MB upon
completion. Unfortunatey I can't see what the total memory is whilst
inside the Thread.Sleep as in this Thread state, evaluations can't be
performed but I'd be suprised if it went down to 16MB and then back up
to 18MB.

Thanks
Tristan.

 

[Patrice]

AFAIK it doesn't mean anything. The idea behind a GC is that you don't care
about releasing memory immediately. For example your app may want to reuse
the memory you previously feed few minutes earlier...

You could start by a general GC article such as
http://msdn.microsoft.com/msdnmag/issues/1200/GCI2/. A more significant test
would be likely to loop. Either it finally fails and you have a problem. Or
the GC will clean up when the memory pressure starts to be too high (you
also have a method on the GC that should be able to increase artificially
the memory pressure for a test)...