Getting out of memory when DataSet.GetXml()

[OverTheTop]

When the dataset is too large it thows OutOfMemory Exception although
that there is enough momory on the web server, is this a bug in .net
or something must be configured?! The Dataset contains near 50 000
rows. How can I escape this. Please help?

 

[Miha Markic [MVP C#]]

..net handles memory a bit differently and might use more memory than the
size of dataset file when loading.
Did you try adding memory? How much memory do your server have and how large
is the xml file?

 

[Rad [Visual C# MVP]]

When the dataset is too large it thows OutOfMemory Exception although
that there is enough momory on the web server, is this a bug in .net
or something must be configured?! The Dataset contains near 50 000
rows. How can I escape this. Please help?

Well ... for starters I would try to see if I can get the size of the
dataset down. Is it possible to load less data?

 

[Miha Markic [MVP C#]]

The other way would be to deal with it as an XML document.

 

[OverTheTop]

Hi Miha

the web server has 1.5 GB free memory...but when the exception was
trown the memory on the server used was 150 MB...I don't know why the
server didn't use all his memory available...may be .net is playing
stupied...I don't know what is the size of the
dataset...sorry...Monday on work I'll check

Miha Markic [MVP C#] íàïèñà:

.net handles memory a bit differently and might use more memory than the
size of dataset file when loading.
Did you try adding memory? How much memory do your server have and how large
is the xml file?

--
Miha Markic [MVP C#, INETA Country Leader for Slovenia]
RightHand .NET consulting & development www.rthand.com
Blog: http://cs.rthand.com/blogs/blog_with_righthand/

When the dataset is too large it thows OutOfMemory Exception although
that there is enough momory on the web server, is this a bug in .net
or something must be configured?! The Dataset contains near 50 000
rows. How can I escape this. Please help?

 

[OverTheTop]

Hi Rad

Because of the application there is no way of loading less data...

Rad [Visual C# MVP] íàïèñà:

 

[OverTheTop]

Yes I was thinking of something like this...first use
DataSet.WriteXml(tempFile) and then use a XmlDocument to load the
tempFile and get the xml string representation of the DataSet. Is that
what you suggest? Is this gonna solve this issue?



Miha Markic [MVP C#] íàïèñà:

The other way would be to deal with it as an XML document.

--
Miha Markic [MVP C#, INETA Country Leader for Slovenia]
RightHand .NET consulting & development www.rthand.com
Blog: http://cs.rthand.com/blogs/blog_with_righthand/

When the dataset is too large it thows OutOfMemory Exception although
that there is enough momory on the web server, is this a bug in .net
or something must be configured?! The Dataset contains near 50 000
rows. How can I escape this. Please help?

 

[Miha Markic [MVP C#]]

Hm, that's odd.
The problem when you see much memory available and you get outofmemory
exception might be caused by memory fragmentation.
Or perhpas there is a bug somewhere, hard to tell.

--
Miha Markic [MVP C#, INETA Country Leader for Slovenia]
RightHand .NET consulting & development www.rthand.com
Blog: http://cs.rthand.com/blogs/blog_with_righthand/

Hi Miha

the web server has 1.5 GB free memory...but when the exception was
trown the memory on the server used was 150 MB...I don't know why the
server didn't use all his memory available...may be .net is playing
stupied...I don't know what is the size of the
dataset...sorry...Monday on work I'll check

Miha Markic [MVP C#] íàïèñà:

.net handles memory a bit differently and might use more memory than the
size of dataset file when loading.
Did you try adding memory? How much memory do your server have and how
large
is the xml file?

--
Miha Markic [MVP C#, INETA Country Leader for Slovenia]
RightHand .NET consulting & development www.rthand.com
Blog: http://cs.rthand.com/blogs/blog_with_righthand/

When the dataset is too large it thows OutOfMemory Exception although
that there is enough momory on the web server, is this a bug in .net
or something must be configured?! The Dataset contains near 50 000
rows. How can I escape this. Please help?

 

[Miha Markic [MVP C#]]

Yes I was thinking of something like this...first use
DataSet.WriteXml(tempFile) and then use a XmlDocument to load the
tempFile and get the xml string representation of the DataSet. Is that
what you suggest? Is this gonna solve this issue?

Yes, or even less memory problematic if you use XmlReader that doesn't load
everything - it is just some sort of cursor.

 

[OverTheTop]

Yes I was thinking of something like this...first use
DataSet.WriteXml(tempFile) and then use a XmlDocument to load the
tempFile and get the xml string representation of the DataSet. Is that
what you suggest? Is this gonna solve this issue?

Yes, or even less memory problematic if you use XmlReader that doesn't load
everything - it is just some sort of cursor.

this how i do it now

string xmlDataSet = null;

string tempFile = Path.GetTempFileName();

FileInfo fi = new FileInfo(tempFile);

if(fi.Exists) {

ds.WriteXml(tempFile);

using(StreamReader sr = fi.OpenText()) {

xmlDataSet = sr.ReadToEnd();
sr.Close();
}

fi.Delete();


About sr.ReadToEnd() method...can also throw outofmemory exception,
but can I escape this, because String in .Net is stored in the heap
(in memory) so if the hardware can't hold it, noone can. Corrent me if
I am wrong? So is this the best solution I can think of?