Craig, since you are only displaying a progress bar, then you want an average
measurement. I would highly recommend using the method I show below,
with some form of cut-off. For example:
(Read 16k worth of data, 4 times through the loop). Then:
float fudge = 1.05f;
totalLines = (int) (averagedLines * (FileLength / 16k) * fudge);
If you are reading the total number of lines first, then you are already
processing the entire file. There are fast ways to do this (as I've shown
below)
and slow ways, but you need a way that doesn't force you to read the entire
file,
and instead guess at the total number of lines.
--
Justin Rogers
DigiTec Web Consultants, LLC.
Blog:
http://weblogs.asp.net/justin_rogers
Craig Bumpstead said:
Thanks everybody for the advice,
The files that I have been reading are about 1 to 3 Gb in size.
So as you could imagine that the ReadLine takes some time to complete.
I wanted the amount of lines in a file so that I could then use it for
the calc. of the progress bar.
Cheers,
Craig
least,
but
creates a very large amount of extra baggage. ReadToEnd() creates one huge
string. Split repackages that data into a string for every single line. Big
memory
waste at this point. Since Split will return a string array with even the last
line you
should just need a Length call.
Here is a more performant version for large files that uses a sharing
FileStream.
I've also included an updated version of the ReadToEnd method. You can easily
add some timing code in and create a rather large file that demonstrates the
first
method being faster and more memory efficient.
using System;
using System.IO;
public class LineCount {
private static byte[] lineBuffer = new byte[4196]; // 4K
private static void Main(string[] args) {
int lines = 0;
using(FileStream fs = new FileStream(args[0], FileMode.Open,
FileAccess.Read, FileShare.Read, lineBuffer.Length)) {
int bufferRead = 0;
while( (bufferRead = fs.Read(lineBuffer, 0,
lineBuffer.Length))
0 ) {
for(int i = 0; i < bufferRead; i++) {
if ( lineBuffer == 0xD ) {
lines++;
}
}
}
fs.Close();
}
lines++;
Console.WriteLine(lines);
StreamReader sw = new StreamReader(args[0]);
lines = sw.ReadToEnd().Split((char)13).Length;
Console.WriteLine(lines);
}
}
