Application Eating up Memory

[news]

Hi,

I have a C# application (Visual Studio 2005) that invloves reading a large amount of data from SQL server (in this case SQL 2005). Here is what I am trying to do.

//declare a data set obeject
DataSet accountRecord = new DataSet();
DataSet oDS = new DataSet();

//query database useing the OdbcConnection connection

//loop thru all the records

//insert a new row in another database. assuming the connection exists
odr = oDS.Tables[0].NewRow();

odr[field name] = value;

//update dataset
oDS.Tables[0].Rows.Add(odr);

//save data in table.
oAdapter.Update(oDS);

//end loop

When the internal loop is being executed the memory usage goes on increasing till I get an System.OutOfMemory exception. Can you tell me the cause of the memory increasing in the for loop. Please let me know if you need more information.

Sincerely,
Saurabh

 

[Peter Bromberg [C# MVP]]

Couple of ideas here. You don't have complete code in your sample, so it's hard to tell what exactly is going on.

1. Why ODBC? If you are talking to SQL Server, the SQLClient class and SqlConnection object will be more efficient.

2. It seems that what you are doing is adding a row to a Datatable and using the DataAdapter Update method to insert a new row in a table. Why not just do a direct insert with ExecuteNonQuery? It's more efficient.

3. Make sure you observe best practices by opening your connection immediately before use and closing it immediately after. This returns the connection object to the connection pool, and is most likely the cause of your problems.

-Peter
Hi,

I have a C# application (Visual Studio 2005) that invloves reading a large amount of data from SQL server (in this case SQL 2005). Here is what I am trying to do.

//declare a data set obeject
DataSet accountRecord = new DataSet();
DataSet oDS = new DataSet();

//query database useing the OdbcConnection connection

//loop thru all the records

//insert a new row in another database. assuming the connection exists
odr = oDS.Tables[0].NewRow();

odr[field name] = value;

//update dataset
oDS.Tables[0].Rows.Add(odr);

//save data in table.
oAdapter.Update(oDS);

//end loop

When the internal loop is being executed the memory usage goes on increasing till I get an System.OutOfMemory exception. Can you tell me the cause of the memory increasing in the for loop. Please let me know if you need more information.

Sincerely,
Saurabh

 

[Arne Vajhøj]

I have a C# application (Visual Studio 2005) that invloves reading a
large amount of data from SQL server (in this case SQL 2005). Here is
what I am trying to do.

//declare a data set obeject
DataSet accountRecord = new DataSet();
DataSet oDS = new DataSet();

//query database useing the OdbcConnection connection

//loop thru all the records

//insert a new row in another database. assuming the connection exists
odr = oDS.Tables[0].NewRow();

odr[field name] = value;


//update dataset
oDS.Tables[0].Rows.Add(odr);

//save data in table.
oAdapter.Update(oDS);

//end loop

When the internal loop is being executed the memory usage goes on
increasing till I get an System.OutOfMemory exception. Can you tell me
the cause of the memory increasing in the for loop. Please let me know
if you need more information.

DataSet and "large amount of data" does not match well.

A DataSet has all data in memory, so if data are in many GB size, then
you will run out of virtual memory on a 32 bit Windows and out
of pagefile on a 64 bit Windows.

Arne