Multithreading, DataTables, and Garbage Collection

[Jenny K]

Hi everyone,

I've got a console application that grabs data into 3 datatables
asynchronously, and writes each datatable's data to a textfile. The problem
is that the data returned by each sproc could easily hit up to 100,000
records each. Since the datatable doesn't implement IDisposable, would it
be prudent of me to set the datatable reference to null and immediately
force a garbage collection to release the memory used by the datatable? I
read somewhere this practice is discouraged - but the exe can consume up to
200 MB RAM unless I force the GC.

Here's some of my code. Any comments?. This is my first attempt at
multithreading so I can use all the help I can get!
Thanks,
Jenny
p.s. please respond to the group.

public class MainClass
private static void DoStuff{
/*create the delegates*/
MainClass.GetData1Delegate d1 = new GetData1Delegate (getData1);
MainClass.GetData2Delegate d2 = new GetData2Delegate (getData2);
MainClass.GetData3Delegate d3 = new GetData3Delegate (getData3);

//call the delegates and pass params and add 2 null params (callback and
asyncstate)
System.IAsyncResult ar1 = d1.BeginInvoke(dt1,batDate,null,null);
System.IAsyncResult ar2 = d2.BeginInvoke(dt2,batDate,null,null);
System.IAsyncResult ar3 = d3.BeginInvoke(dt3,batDate,null,null);

bool asyncsComplete = false;
bool d1Ended = false;
bool d2Ended = false;
bool d3Ended = false;

while(!asyncsComplete){ //Poll for completion every 1000 milliseconds
if(ar1.IsCompleted && !d1Ended){
d1.EndInvoke(ar1);
//write dt1 to text file
dt1 = null;
System.GC.Collect();
d1Ended = true;
}
if(ar2.IsCompleted && !d2Ended){
d2.EndInvoke(ar2);
//write dt2 to text file
dt2 = null;
System.GC.Collect();
d2Ended = true;
}
if(ar3.IsCompleted && !d3Ended){
d3.EndInvoke(ar3);
//write dt3 to text file
dt3 = null;
System.GC.Collect();
d3Ended = true;
}
asyncsComplete = (d1Ended && d2Ended && d3Ended);
Thread.Sleep(1000);
}

} //end DoStuff

/* Data Getters */
public delegate void GetData1Delegate(DataTable dt1, string batDate);
public static void GetData1(DataTable dt1, string batDate){
//run sproc and fill dt1
}

public delegate void GetData2Delegate(DataTable dt2, string batDate);
public static void GetData2(DataTable dt2, string batDate){
//run sproc and fill dt2
}

public delegate void GetData3Delegate(DataTable dt3, string batDate);
public static void GetData3(DataTable dt3, string batDate){
//run sproc and fill dt3
}


} //end class

 

[Jenny K]

hmmm....no replies?
I guess I'm on my own on this one.

Regards,
J

 

[Ryan Pedersen]

I am just learning about this stuff myself so I hesitate to jump in.
Please take this post with a grain of salt. I am not an expert with
threading but I might be able to help a little.

It seems to me that you are doing a lot of work that doesn't need to
be done. The purpose of the callback is so that when there is work to
be done the method specified will be called, right? Here you are
firing up each event but then you are looping to monitor for the
results?

Rather than looping you should be able to use a ManualResetEvent this
doesn't consume the resources that looping does. It just puts the
calling thread (the one that started the async calls) in a
waiting/suspeded state. Then when one of the async calls is done you
can call Set() on the ManualResetEvent that will bring the calling
thread back to life. If you need to wait on all of the trheads before
bringing the main thread back to life I think that you can use the
ManualResetEvent for that too.

Some or all of this post may be wrong. Like I said I am not an expert
but perhaps what I have said above will help you find the right
answer. I would also recommend the book Inside C#. It has an excellent
chapter on threading and a sperate chapter on deligates. Very well
writing and easy to understand.

Best regards,
Ryan Pedersen