XmlDataDocument\DataSet Memory Footprint, Feedback appreciated.

D

Derrick

I am reading in xml files that equate to sql tables, via XmlDataDocument,
and then operating on the DataSet. With the most simple app that just loads
the xml doc, I see the memory footprint of the app grow to roughly 5x the
size of the xml document after the xml doc is read. Can anyone comment on
this? Is this expected?

Below are two samples, the first hits northwind, and creates a 19meg
(roughly, on my machine) xml file. The second simply reads it in, with a
pause after reading so you can inspect the memory usage in task manager.

Any suggestions as to how I can get the footprint down much closer to the
size of the xml file would be greatly appreciated.

Thanks in advance!!

Derrick




Creation of sample xml file:
using System;
using System.Data;
using System.Data.SqlClient;
using System.Xml;
namespace DataExtract
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
string forXml = " FOR XML AUTO, XMLDATA";
SqlConnection mySqlConnection = new
SqlConnection("server=127.0.0.1;database=northwind;uid=sa;pwd=admin");
XmlTextReader myXmlReader = null;
SqlCommand mySqlCmd = new SqlCommand();
DataSet myDataSet = new DataSet();
mySqlConnection.Open();
mySqlCmd.Connection = mySqlConnection;
mySqlCmd.CommandText = "select * from orders cross join customers" + forXml;
myXmlReader = (XmlTextReader)mySqlCmd.ExecuteXmlReader();
myDataSet.ReadXml(myXmlReader, XmlReadMode.Fragment);
myXmlReader.Close();
myDataSet.WriteXml("c:\\lotsa_data.xml");
myDataSet.WriteXmlSchema("c:\\lotsa_data.xsd");
myDataSet.Dispose();
mySqlConnection.Close();
}
}
}

The reader:

using System.Data;
using System.Xml;
using System.IO;
namespace DataRead
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
Stream xsdStream = new StreamReader("c:\\lotsa_data.xsd").BaseStream;
Stream xmlStream = new StreamReader("c:\\lotsa_data.xml").BaseStream;
XmlDataDocument myXmlDoc = new XmlDataDocument();
myXmlDoc.DataSet.ReadXmlSchema(xsdStream);
myXmlDoc.Load(xmlStream);
Console.WriteLine("Loaded...");
Console.ReadLine();
}
}
}
 
D

Derrick

Thanks, downloaded and tried it out, it basically looks like XML objects
(elements and attributes) are half the bag, strings the other half. Is
there a way to read in the XML as simple data, toss the xml associated
objects out, and end up with just native types in the DataSet tables?

The code that reads the xml file into a dataset is all core C#...

Thanks again!


Sherif ElMetainy said:
Hello

I suggest you use a memory profiler to track where your memory is consumed.
Here is a good one
http://www.scitech.se/memprofiler/
This software tells you the datatypes of allocated objects

Best regards
Sherif

Derrick said:
I am reading in xml files that equate to sql tables, via XmlDataDocument,
and then operating on the DataSet. With the most simple app that just loads
the xml doc, I see the memory footprint of the app grow to roughly 5x the
size of the xml document after the xml doc is read. Can anyone comment on
this? Is this expected?

Below are two samples, the first hits northwind, and creates a 19meg
(roughly, on my machine) xml file. The second simply reads it in, with a
pause after reading so you can inspect the memory usage in task manager.

Any suggestions as to how I can get the footprint down much closer to the
size of the xml file would be greatly appreciated.

Thanks in advance!!

Derrick




Creation of sample xml file:
using System;
using System.Data;
using System.Data.SqlClient;
using System.Xml;
namespace DataExtract
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
string forXml = " FOR XML AUTO, XMLDATA";
SqlConnection mySqlConnection = new
SqlConnection("server=127.0.0.1;database=northwind;uid=sa;pwd=admin");
XmlTextReader myXmlReader = null;
SqlCommand mySqlCmd = new SqlCommand();
DataSet myDataSet = new DataSet();
mySqlConnection.Open();
mySqlCmd.Connection = mySqlConnection;
mySqlCmd.CommandText = "select * from orders cross join customers" + forXml;
myXmlReader = (XmlTextReader)mySqlCmd.ExecuteXmlReader();
myDataSet.ReadXml(myXmlReader, XmlReadMode.Fragment);
myXmlReader.Close();
myDataSet.WriteXml("c:\\lotsa_data.xml");
myDataSet.WriteXmlSchema("c:\\lotsa_data.xsd");
myDataSet.Dispose();
mySqlConnection.Close();
}
}
}

The reader:

using System.Data;
using System.Xml;
using System.IO;
namespace DataRead
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
Stream xsdStream = new StreamReader("c:\\lotsa_data.xsd").BaseStream;
Stream xmlStream = new StreamReader("c:\\lotsa_data.xml").BaseStream;
XmlDataDocument myXmlDoc = new XmlDataDocument();
myXmlDoc.DataSet.ReadXmlSchema(xsdStream);
myXmlDoc.Load(xmlStream);
Console.WriteLine("Loaded...");
Console.ReadLine();
}
}
}
Click to expand...
Click to expand...
 
S

Sherif ElMetainy

Hello

You can set the references to all xml objects to null, so that they are not
referenced anymore by your code.
Then they will be collected in the next garbage collection.

You can also force garbage collection to run immediately by calling
System.GC.Collect(), but many people thinks its better that you let the CLR
do garbage collection when it wants.

Best regards,
Sherif

Derrick said:
Thanks, downloaded and tried it out, it basically looks like XML objects
(elements and attributes) are half the bag, strings the other half. Is
there a way to read in the XML as simple data, toss the xml associated
objects out, and end up with just native types in the DataSet tables?

The code that reads the xml file into a dataset is all core C#...

Thanks again!


Sherif ElMetainy said:
Hello

I suggest you use a memory profiler to track where your memory is consumed.
Here is a good one
http://www.scitech.se/memprofiler/
This software tells you the datatypes of allocated objects

Best regards
Sherif
Click to expand...
comment
on
Click to expand...
with
a
pause after reading so you can inspect the memory usage in task manager.

Any suggestions as to how I can get the footprint down much closer to the
size of the xml file would be greatly appreciated.

Thanks in advance!!

Derrick




Creation of sample xml file:
using System;
using System.Data;
using System.Data.SqlClient;
using System.Xml;
namespace DataExtract
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
string forXml = " FOR XML AUTO, XMLDATA";
SqlConnection mySqlConnection = new
SqlConnection("server=127.0.0.1;database=northwind;uid=sa;pwd=admin");
XmlTextReader myXmlReader = null;
SqlCommand mySqlCmd = new SqlCommand();
DataSet myDataSet = new DataSet();
mySqlConnection.Open();
mySqlCmd.Connection = mySqlConnection;
mySqlCmd.CommandText = "select * from orders cross join customers" + forXml;
myXmlReader = (XmlTextReader)mySqlCmd.ExecuteXmlReader();
myDataSet.ReadXml(myXmlReader, XmlReadMode.Fragment);
myXmlReader.Close();
myDataSet.WriteXml("c:\\lotsa_data.xml");
myDataSet.WriteXmlSchema("c:\\lotsa_data.xsd");
myDataSet.Dispose();
mySqlConnection.Close();
}
}
}

The reader:

using System.Data;
using System.Xml;
using System.IO;
namespace DataRead
{
class Class1
{
[STAThread]
static void Main(string[] args)
{
Stream xsdStream = new StreamReader("c:\\lotsa_data.xsd").BaseStream;
Stream xmlStream = new StreamReader("c:\\lotsa_data.xml").BaseStream;
XmlDataDocument myXmlDoc = new XmlDataDocument();
myXmlDoc.DataSet.ReadXmlSchema(xsdStream);
myXmlDoc.Load(xmlStream);
Console.WriteLine("Loaded...");
Console.ReadLine();
}
}
}
Click to expand...
Click to expand...
Click to expand...
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Cannot find out missing brace error } 4
Problem with MemoryStream 2
Simple database class error 5
I can reproduce this Memory leak! 5
Inserting rows in dataset bound to XmlDataDocument - Framework bug? 3
convert recordset to dataset 0
a more OO approach. 2
XMLDataDocument - improper Dataset row insert into XML hierarchy 0

Top