Working with large txt files using streams

[Guest]

hi all,

I am getting an error when i process 1 MB txt file using StreamReader and i
am reading char by char using streamReader.Read() method at some point it is
throwing an exception System.ComponentModel.win32Exception:The handle is
invalid.
Any help for this problem or any kind of informaiton to process large files
the file size can be 10 MB.

thanks in advance
Syed

 

[Morten Wennevik]

Hi Syed,

Just a thought, is the text file UTF8 encoded? If not, specify another encoding when creating the StreamReader.

 

[Guest]

Hi Morten,

The text file is having arabic data, but i tested the streamReader with
UTF8 encoding and still the same error occurs.

Thanks

 

[Morten Wennevik]

Hi Syed,

If you don't specify an encoding when creating the StreamReader, it will try to decode the file as UTF8. Try specifying Encoding.Unicode or Encoding.Default. If both fail I'm afraid I'm out of ideas and you might want to start a new thread with your question.

Hi Morten,

The text file is having arabic data, but i tested the streamReader with
UTF8 encoding and still the same error occurs.

Thanks

 

[William Stacey [MVP]]

hmm. 1MB is not large, so don't think that is the issue. Could you post a
small but complete code sample that shows the error?

 

[Guest]

Hi William ,
i am sending u the text file data and code to process the text data.
Note : All this processing is done in windows service and checked with all
the encodings (Default,UTF8,unicode). i really don't known where i am going
wrong.
------------------------------------------------------
V|173848|1|4001938833|14250316|dsfsdfs|sdfsdfsdfsd|1000174076|0|sdfsdfsdfsdf|6674544|2330310451|14260601|sdfsdf||TUR|3|1|||asdsd|1||4001938833|16-03-1425||||asdas|sdfsdf||sdfds|Umit|Muhriz||Cicekli|TUR|asdas|1982/11/22||1|asdasd|1|140946|2005/06/30|asdasd|2006/06/29|1|||233|Umit Muhriz Cicekli||||||
V|173848|1|4001938833|14250316|dsfsdfs|sdfsdfsdfsd|1000174076|0|sdfsdfsdfsdf|6674544|2330310451|14260601|sdfsdf||TUR|3|1|||asdsd|1||4001938833|16-03-1425||||asdas|sdfsdf||sdfds|Umit|Muhriz||Cicekli|TUR|asdas|1982/11/22||1|asdasd|1|140946|2005/06/30|asdasd|2006/06/29|1|||233|Umit Muhriz Cicekli||||||
V|173848|1|4001938833|14250316|dsfsdfs|sdfsdfsdfsd|1000174076|0|sdfsdfsdfsdf|6674544|2330310451|14260601|sdfsdf||TUR|3|1|||asdsd|1||4001938833|16-03-1425||||asdas|sdfsdf||sdfds|Umit|Muhriz||Cicekli|TUR|asdas|1982/11/22||1|asdasd|1|140946|2005/06/30|asdasd|2006/06/29|1|||233|Umit Muhriz Cicekli||||||
V|173848|1|4001938833|14250316|dsfsdfs|sdfsdfsdfsd|1000174076|0|sdfsdfsdfsdf|6674544|2330310451|14260601|sdfsdf||TUR|3|1|||asdsd|1||4001938833|16-03-1425||||asdas|sdfsdf||sdfds|Umit|Muhriz||Cicekli|TUR|asdas|1982/11/22||1|asdasd|1|140946|2005/06/30|asdasd|2006/06/29|1|||233|Umit Muhriz Cicekli||||||
V|173848|1|4001938833|14250316|dsfsdfs|sdfsdfsdfsd|1000174076|0|sdfsdfsdfsdf|6674544|2330310451|14260601|sdfsdf||TUR|3|1|||asdsd|1||4001938833|16-03-1425||||asdas|sdfsdf||sdfds|Umit|Muhriz||Cicekli|TUR|asdas|1982/11/22||1|asdasd|1|140946|2005/06/30|asdasd|2006/06/29|1|||233|Umit Muhriz Cicekli|||||||

-------------------------------------------------------
// Reading the text file using stream reader
//Open the file using stream reader

ObjStreamReader = new StreamReader(e.FullPath,System.Text.Encoding.UTF8);

char c = ' ';
while (ObjStreamReader.Peek() > 0)
{
// Reading the line char by char
c = (char) ObjStreamReader.Read();
if (c != '\n')
{
strLine += c;
}
else
{
// building the Xml by splitting the strLine and
sending to the queue.
}
//Going to next line
} // end of while loop



thanks
syed

 

[Jon Skeet [C# MVP]]

i am sending u the text file data and code to process the text data.
Note : All this processing is done in windows service and checked with all
the encodings (Default,UTF8,unicode). i really don't known where i am going
wrong.

Could you post a short but complete program which demonstrates the
problem?

See http://www.pobox.com/~skeet/csharp/complete.html for details of
what I mean by that.