Out of Memory Exception

G

Guest

----Begin : Background description----
My Application reads flat file and based on the record number in the flat
file, it populates data to the on-fly Datatables of 23 different types. The
finally output expected is XML.
The Application runs on 2GB RAM server. The input flat file size can be upto
800MB.
----End : Background description----

----Begin : Technical description----
Visual Studio 2003, .Net 1.1
Console application in C#
----End : Technical description----

----Begin : Problem description----
When the input file is of size 500MB or more, the application started
throwing Out of Memory exception.

Any idea to resolve this problem guys?
----End : Problem description----

Thanks in advance

With regards
Venkat
 
G

Guest

Hi,
Even I too agree. But in this case each table records are related to each
other.

For eg. Table 7, 8, 9 10, 11, 12 and 13 had parent child relation so as
other tabless too have such relation.... In this case to do?

Thanks

With regards
Venkat

Miha Markic said:
Hi,

You should read file line by line (or a piece by piece) and write xml
directly using XmlWriter perhaps.
Sou don't keep all those MBs in memory.

--
Miha Markic [MVP C#]
RightHand .NET consulting & development www.rthand.com
Blog: http://cs.rthand.com/blogs/blog_with_righthand/

So y
Venkatachalam said:
----Begin : Background description----
My Application reads flat file and based on the record number in the flat
file, it populates data to the on-fly Datatables of 23 different types.
The
finally output expected is XML.
The Application runs on 2GB RAM server. The input flat file size can be
upto
800MB.
----End : Background description----

----Begin : Technical description----
Visual Studio 2003, .Net 1.1
Console application in C#
----End : Technical description----

----Begin : Problem description----
When the input file is of size 500MB or more, the application started
throwing Out of Memory exception.

Any idea to resolve this problem guys?
----End : Problem description----

Thanks in advance

With regards
Venkat
 
M

Miha Markic [MVP C#]

Hi,

Venkatachalam said:
Hi,
Even I too agree. But in this case each table records are related to
each
other.

For eg. Table 7, 8, 9 10, 11, 12 and 13 had parent child relation so as
other tabless too have such relation.... In this case to do?

I would simulate a database - go forth and back through file to load just
necessary records.
Perhaps you might create an index of records in text file (you would index
id and their position in file).
You might also create more complex solutions - depends on the structure of
text file and the way you want to do it.
 
M

Miha Markic [MVP C#]

Venkatachalam said:
Don't you think that will degrade the performance?

Actually performance is infitively better: working vs non-working ;-)
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top