Process File in Chunks - StreamReader

[das]

I posted this on ado.net, which seems like the wrong group - so here it
goes!

Hi All,
I want to process a file in chunks, meaning I have a big file of
50K rows. I want to process 5K rows at a time.

I am using StreamReader object and looping through each row and process

it, and as I process it I create an XML file and send it to DB to be
processed.


while(oSR.Peek() > -1)
{
// 1. Create Xml from these lines


}


// 2. Call DB Functions.

How do I read only certain number of lines at a time, complete my steps

1 and 2 and come back and read another 5K lines and so on....


any ideas gurus?
thanks

 

[Guest]

This is kind of hokey, and untested, but should give the idea:


StringBuilder sb=null;
while(oSR.Peek() > -1)
{
sb=new StringBuilder();
for(int i=0;i<5000;i++)
{
if(oSr.Peek()>0-1)
sb.Append(oSR.ReadLine());

}
ProcessMyXml(sb.ToString()

}

-- where ProcessMyXml accepts the string containing the 5000 lines,
you can Split ('\r') on the newline to get into a string[] array for
processing.
and Split again on each row if it has delimited info in it.

Peter

 

[das]

thank you, I will test this and reply back.