Pls HELP!- DataSet Inherited object not finalizing

  • Thread starter Thread starter MuZZy
  • Start date Start date
Willy said:
Well it depends what you are doing in your finalizer, if all you are doing
is decrement a shared variable it's a waste of resources (finalizers are
expensive to run), if you are releasing unmanaged resources YOU OWN, it's
something that should be done in your Dispose method.
But re-registering will certainly NOT release the resources held by the
DataSet and it's contained objects, that's taken care of by the GC not the
Finalizer!!!
In finalizer i destroy unmanaged c++ objects of types STRMAP, STRPAIRVECTOR, etc. which are STL
types defines.

Again, i know i should do it in Dispose(), but that's not an option now as fixing it in all places
woul dtake me more than a year with hundreds of classes using this derived from DataSet class.

So i have to work with what i have unfotunately...
 
Willy said:
What exactly do you mean by it's working?
Are the memory leaks gone?

Willy.

Well, under "it's done!" i meant that objects get finalized, i see it because the static counter
decreases.
But no, memory leaks aren't gone.. and that seems to be a much bigger issue...

Oh man, my brain is on fire!
 
MuZZy said:
In finalizer i destroy unmanaged c++ objects of types STRMAP,
STRPAIRVECTOR, etc. which are STL types defines.

Again, i know i should do it in Dispose(), but that's not an option now as
fixing it in all places woul dtake me more than a year with hundreds of
classes using this derived from DataSet class.

So i have to work with what i have unfotunately...

I told you before and I suggest you again;

- forget the finalizer, your problem is not related to this!!
Using the performance manager, watch the Gen0, 1, 2 and large heap (LH)
allocations (CLR memory counters), when done with the DataSet (extended) and
you remove all tables from the DS, you will see that all counters drop to a
reasonable low level after a GC run (you can try by calling GC.Collect),
however if you fail to remove the DataTables there is nothing to collect as
the DataTables still hold a reference to the DS and the DS still holds
references to the contained tables.

Willy.
 
Willy said:
I told you before and I suggest you again;

- forget the finalizer, your problem is not related to this!!
Using the performance manager, watch the Gen0, 1, 2 and large heap (LH)
allocations (CLR memory counters), when done with the DataSet (extended) and
you remove all tables from the DS, you will see that all counters drop to a
reasonable low level after a GC run (you can try by calling GC.Collect),
however if you fail to remove the DataTables there is nothing to collect as
the DataTables still hold a reference to the DS and the DS still holds
references to the contained tables.

Your last statement about DataTable refs and DataSet refs is very interesting.
Could you give me a bit more explanation on what you mean?

Do you mean that if i don't clear DataSetExtended->Tables->Clear() it won't be GC'd?
Even if i do DataSetExtended = null; ?
Why? The obect is dereferenced, Tables are contained in it, reference is internal...

AM I MISSING SOMETHING??? I know i am :( Pls HELP!!!
I'm going nuts with app, frankly :(
 
Willy Denoyette said:
I told you before and I suggest you again;

- forget the finalizer, your problem is not related to this!!

It is, if they were relying on the finalizer to release unmanaged
resources.
Using the performance manager, watch the Gen0, 1, 2 and large heap (LH)
allocations (CLR memory counters), when done with the DataSet (extended) and
you remove all tables from the DS, you will see that all counters drop to a
reasonable low level after a GC run (you can try by calling GC.Collect),
however if you fail to remove the DataTables there is nothing to collect as
the DataTables still hold a reference to the DS and the DS still holds
references to the contained tables.

I don't think that's the problem, as he's now seeing the DataSets being
finalized.
 
See inline ***

Willy.

Jon Skeet said:
It is, if they were relying on the finalizer to release unmanaged
resources.
*** Sure, I didn't say the finalizer was useless, I said it won't release
the DataSet objects.
I don't think that's the problem, as he's now seeing the DataSets being
finalized.

*** No, he's not seeing the DataSet's being released, he's seeing the
finalizer being called (check it's reply dated 23:53 on this thread).
 
Your last statement about DataTable refs and DataSet refs is very
interesting.
Could you give me a bit more explanation on what you mean?

Do you mean that if i don't clear DataSetExtended->Tables->Clear() it
won't be GC'd?
Even if i do DataSetExtended = null; ?
Why? The obect is dereferenced, Tables are contained in it, reference is
internal...

AM I MISSING SOMETHING??? I know i am :( Pls HELP!!!
I'm going nuts with app, frankly :(

I guess, I can better illustrate this by means of a sample.
Compile following code, and run it from the console, start perfmon and watch
the GC counters (CLR memory object).
You will see that as long as the DataSet contains tables, it won't get
collected. Don't forget that the actual size of a DataSet is the size of
it's contained table(s) data.

// Begin
using System;
using System.Data;
class Tester
{
static void Main()
{
using(DataSet custDS = new DataSet("Test"))
{
Console.WriteLine("Start perfmon, select counters and press <Enter> when
done");
Console.ReadLine(); // this gives you the opportunity to start permon
and select the GC counters.
DataTable dt = Build(custDS);
// Check - does the table belongs to a dataset?
if (dt.DataSet != null) // NOTE this property is read-only and holds a
reference to the containing DataSet!!!!
{
Console.WriteLine("Detach all tables");
// Remove tables from the collection,
// comment following statement and watch the perf counters...
dt.DataSet.Tables.Clear();
// or remove a single table from the collection
// dt.DataSet.Tables.Remove (dt);
// Check again...
if (dt.DataSet == null)
Console.WriteLine("DS reference cleared, press <Enter> to force GC");
}
Console.ReadLine();
// Force a collection (only for demo purpose)
GC.Collect();
}
Console.ReadLine();
}
// Fill a table with data and add it to the DataSet
private static DataTable Build(DataSet ds){
DataTable dt = ds.Tables.Add("MyTable");
DataColumn dcol;
DataRow drow;
dcol = new DataColumn();
dcol.DataType = System.Type.GetType("System.Int32");
dcol.ColumnName = "id";
dt.Columns.Add(dcol);
dcol = new DataColumn();
dcol.DataType = Type.GetType("System.String");
dcol.ColumnName = "item";
dt.Columns.Add(dcol);
// Add some data
for(int i = 0; i < 50000; i++){
drow = dt.NewRow();
drow["id"] = i;
drow["item"] = "item " + i;
dt.Rows.Add(drow);
}
return dt;
}
}
// End code

Willy.
 
Willy Denoyette said:
See inline ***

Willy.


*** Sure, I didn't say the finalizer was useless, I said it won't release
the DataSet objects.

he's not trying to release DataSet in the finalizer. He has a class derived
from DataSet. In this derived class, he has a bunch of unmanaged STL objects
that needs to be released. but because DataSet is supressing the
finalization, the finalizer that attempt to release these STL objects in the
derived class wasn't called. the sample code he supplied that simply
decrements a counter was just a sample to demonstrate his problem. that is
not what he's doing in his real code.
I don't think that's the problem, as he's now seeing the DataSets being
finalized.

*** No, he's not seeing the DataSet's being released, he's seeing the
finalizer being called (check it's reply dated 23:53 on this thread).
 
if what you are saying is true, then this modified sample should not collect
because the circular reference between DataSet and DataTable, and clearly
that's not the case.

All GC cares about is if the object is reachable from a root object.
DataSet and DataTable can reference each other all they want, and if it's not
reachable from the root, it's garbage and will be collected.

using System;
using System.Data;

namespace TestNamespace
{
public class Test :DataSet
{
public static int i = 0;
public Test():base()
{
i++;
GC.ReRegisterForFinalize(this);
}
~Test(){i--;}
}

public class MainClass
{
[STAThread]
public static void Main(string[] args)
{
Console.WriteLine("Generating Garbage:");
for (int i = 0 ; i < 10 ; i ++) {
Test t = new Test();
t.Tables.Add();
}

Console.WriteLine("Collecting Garbage: " + Test.i.ToString());

GC.Collect();
GC.WaitForPendingFinalizers();
Console.WriteLine("Objects not finalized: "+ Test.i.ToString());

Console.ReadLine();
}
}
}
 
Your test is flawed.

your using() block is keeping the reference to DataSet around. that's why
detaching DataTable forces the collection on it, without detachment causes
the DataTable not being collected because it's still reachable through a
rooted object (DataSet)

Willy Denoyette said:
Your last statement about DataTable refs and DataSet refs is very
interesting.
Could you give me a bit more explanation on what you mean?

Do you mean that if i don't clear DataSetExtended->Tables->Clear() it
won't be GC'd?
Even if i do DataSetExtended = null; ?
Why? The obect is dereferenced, Tables are contained in it, reference is
internal...

AM I MISSING SOMETHING??? I know i am :( Pls HELP!!!
I'm going nuts with app, frankly :(

I guess, I can better illustrate this by means of a sample.
Compile following code, and run it from the console, start perfmon and watch
the GC counters (CLR memory object).
You will see that as long as the DataSet contains tables, it won't get
collected. Don't forget that the actual size of a DataSet is the size of
it's contained table(s) data.

// Begin
using System;
using System.Data;
class Tester
{
static void Main()
{
using(DataSet custDS = new DataSet("Test"))
{
Console.WriteLine("Start perfmon, select counters and press <Enter> when
done");
Console.ReadLine(); // this gives you the opportunity to start permon
and select the GC counters.
DataTable dt = Build(custDS);
// Check - does the table belongs to a dataset?
if (dt.DataSet != null) // NOTE this property is read-only and holds a
reference to the containing DataSet!!!!
{
Console.WriteLine("Detach all tables");
// Remove tables from the collection,
// comment following statement and watch the perf counters...
dt.DataSet.Tables.Clear();
// or remove a single table from the collection
// dt.DataSet.Tables.Remove (dt);
// Check again...
if (dt.DataSet == null)
Console.WriteLine("DS reference cleared, press <Enter> to force GC");
}
Console.ReadLine();
// Force a collection (only for demo purpose)
GC.Collect();
}
Console.ReadLine();
}
// Fill a table with data and add it to the DataSet
private static DataTable Build(DataSet ds){
DataTable dt = ds.Tables.Add("MyTable");
DataColumn dcol;
DataRow drow;
dcol = new DataColumn();
dcol.DataType = System.Type.GetType("System.Int32");
dcol.ColumnName = "id";
dt.Columns.Add(dcol);
dcol = new DataColumn();
dcol.DataType = Type.GetType("System.String");
dcol.ColumnName = "item";
dt.Columns.Add(dcol);
// Add some data
for(int i = 0; i < 50000; i++){
drow = dt.NewRow();
drow["id"] = i;
drow["item"] = "item " + i;
dt.Rows.Add(drow);
}
return dt;
}
}
// End code

Willy.
 
I'm watching perfmon and i see that CLR Gen 2 increases dramatically though egenartion 0 and 1 heaps
oscillate around one value
 
Daniel Jin said:
Your test is flawed.

your using() block is keeping the reference to DataSet around. that's why
detaching DataTable forces the collection on it, without detachment causes
the DataTable not being collected because it's still reachable through a
rooted object (DataSet)

My bad, sorry for this, I should have mentioned this explicitly, but who
tells the they aren't using the same construct. Note that OP said he wasn't
the author of the application, so I only wanted to illustrate (on purpose):
1. the effect of keeping a reference to the DataSet instance, be it in a
scoped using block or something else, and..
2. the effect of releasing the contained table objects on the managed memory
consumption, without the need to release the DataSet object instance.

I did this only to show what OP has to look for and to illustrate how the
performance counters can be used to watch the behavior.

Willy.
 
Daniel Jin said:
he's not trying to release DataSet in the finalizer. He has a class
derived
from DataSet. In this derived class, he has a bunch of unmanaged STL
objects
that needs to be released. but because DataSet is supressing the
finalization, the finalizer that attempt to release these STL objects in
the
derived class wasn't called. the sample code he supplied that simply
decrements a counter was just a sample to demonstrate his problem. that
is
not what he's doing in his real code.

Sure, but until now he didn't show any real code, and running the finalizer
did not solve his memory leak issue as he replied in another posting.
Problem with OP is that this is not the first (and not the last, actualy he
started a new one) thread on this same issue. In a previous thread he
explicitly said that when running the code through a profiler, it was clear
that the memory was taken by a DataSet, that means it was managed memory so
running the finalizer would not release this (unless some way or another a
reference was held by the unmanaged part).
Another thing I fail to see ( without seeing some code) is why ( and how )
the managed part is freeing unmanaged STL objects.

Willy.
 
Willy said:
Sure, but until now he didn't show any real code, and running the finalizer
did not solve his memory leak issue as he replied in another posting.
Problem with OP is that this is not the first (and not the last, actualy he
started a new one) thread on this same issue. In a previous thread he
explicitly said that when running the code through a profiler, it was clear
that the memory was taken by a DataSet, that means it was managed memory so
running the finalizer would not release this (unless some way or another a
reference was held by the unmanaged part).
Another thing I fail to see ( without seeing some code) is why ( and how )
the managed part is freeing unmanaged STL objects.

Willy.


Ok, here is the real piece of code:

// In .h file here is the part of class declaration
public __gc class CResSet : public DataSet
{
public:
CResSet(CResSet *rs);
~CResSet();
STRMAP *m_smTables;
STRPAIRVECTOR *m_vp;
STRSET *m_ssPopulated;
<...>
}


// In .cpp file, here is the finalizer releasing unmanaged memory

// constructor
TnrData::CResSet::CResSet()
{
<..>
m_vp = new STRPAIRVECTOR();
m_ssPopulated = new STRSET();
m_smTables = new STRMAP();
<..>
}

// finalizer
TnrData::CResSet::~CResSet()
{
delete m_smTables; // THESE ARE
delete m_vp; // UNMANAGED STL
delete m_ssPopulated; // OBJECTS

Counter--;
for (int i = 0 ; i < Numbers->Count; i++)
{
Object *o = Numbers->get_Item(i);
int iii = Convert::ToInt32(o);
if (iii == Num)
Numbers->Remove(o);
}
}

So did you say that i can't free unmanaged objects in managed code?
Could you please exlplain?

Thank you
Andrey
 
MuZZy said:
Ok, here is the real piece of code:

// In .h file here is the part of class declaration
public __gc class CResSet : public DataSet
{
public:
CResSet(CResSet *rs);
~CResSet();
STRMAP *m_smTables;
STRPAIRVECTOR *m_vp;
STRSET *m_ssPopulated;
<...>
}


// In .cpp file, here is the finalizer releasing unmanaged memory

// constructor
TnrData::CResSet::CResSet()
{
<..>
m_vp = new STRPAIRVECTOR();
m_ssPopulated = new STRSET(); m_smTables = new STRMAP();
<..>
}

// finalizer
TnrData::CResSet::~CResSet()
{
delete m_smTables; // THESE ARE
delete m_vp; // UNMANAGED STL
delete m_ssPopulated; // OBJECTS

Counter--;
for (int i = 0 ; i < Numbers->Count; i++)
{
Object *o = Numbers->get_Item(i);
int iii = Convert::ToInt32(o);
if (iii == Num)
Numbers->Remove(o);
}
}

So did you say that i can't free unmanaged objects in managed code?
Could you please exlplain?

Thank you
Andrey

You must be kidding, now you are talking about managed C++ wrapping
unmanaged objects.
Yes, you can destroy the objects exactly the way you are doing, but I have
some doubts about the remainder in the destructor.
Where is Num and Numbers coming from, are they members of this class?
Willy.
 
Willy said:
You must be kidding, now you are talking about managed C++ wrapping
unmanaged objects.
My apology! I made a wrong assumption that in this thread we are talking mostly about managed code,
i also made a mistake originally in the sample post, didn't put __gc for class...

I'm lacking knowledge about managed c++ as i didn't have any experience with it (i had some with
regular C++ of course) up until last week.

See, the project i'm taking part in consists of 5 layers - UI is in C# and all others (like business
layer etc) are in c++. So that\s where my confusion is coming form.
Yes, you can destroy the objects exactly the way you are doing, but I have
some doubts about the remainder in the destructor.
Where is Num and Numbers coming from, are they members of this class?
Willy.

yes, "Numbers" is a static ArrayList member and Num is instance's int member

And thank you for your help!
 
Back
Top