Dataset.ReadXml: M/c hangs when trying to read 37mb xml string.

[Guest]

Hi All,
I am trying to read a 37 mb xml from a url into a stream and then store it
in a string variable "_xml". When I try to pass this string as a StringReader
object to the ReadXml method, my m/c hangs and I have to kill the process.
Is there any upper limit on the size of data which can be passed to the
Dataset's ReadXml method? If yes, can anyone point me to the documentation or
help me resolve this issue.

Here is the piece of code:

WebClient wc = new WebClient();
Stream stream = wc.OpenRead(_url); //Url returns 37mb xml data.
StreamReader sReader = new StreamReader(stream);
string _xml = sReader.ReadToEnd();
_ds = new DataSet();

_ds.ReadXml(new StringReader(_xml), XmlReadMode.InferSchema);

Thanks in advance.
Savvy

 

[Patrice]

For now I woud try to get the file locally to see if the problem is in
getting the file over http or loading it in the DS. Also you'll be then able
to modify this file to see how it performs when loaded in the DataSet...

You could also create a dummy table wiht numerous records and save/restore
the DS to see if you can overcome 37 Mb. How much dio you wait ? Could it be
just slow ?

Good luck.

 

[Guest]

Hi,
The url in this case is a xml file present on my own m/c in a virtual
directory. The file gets loaded over http pretty fast and gets stuck on this
line of code:
_ds.ReadXml(new StringReader(_xml), XmlReadMode.InferSchema);

I waited for > 10 mins. for the xml string to load in the dataset but the
debugger doesn't proceed to the next line of code and the m/c hangs.

Thanks,

 

[Patrice]

What if you load a smaller version of this file ? Does it work ?

 

[Guest]

It works with upto 2 MB Xml file (smaller version of the 37mb xml file) and
takes around 2.5 minutes to load it in dataset.

Any other inputs??

--Savvy

 

[Patrice]

If the time is linear it should take about 45 minutes then ? I would wait
more than 10 minutes, launching this at luch time or in the evening. Also I
didn't noticed at first but this method will likely have a huge memory
comsumption.

For now I would say it is just really slow and you have to wait more...

Another option would be likely to get the file on disk and read from there.

I tried to create a dummy datatable with 10 string fields with 40 characters
each. It is around 30 Mb and it takes only 7 s to load....

Good luck.