How to only read the last 100 bytes from a Binary file?

[Guest]

Hi,

Is it possible to read a file in reverse and only get the last 100 bytes in
the file without reading the whole file from the begining? I have to get info
from files that are in the last 100 bytes of the file and some of these files
are 600Mb -1 GB in size. I am getting "outofMemory.." exceptions on the
largest files and the other files take "forever" to get the last 100 bytes.

This is the code I have currently that works with smaller files:

Dim fileData() As Byte
FilePath = "C:\Temp\1DD04336CA1A45CB81A89167048D2594.abk"
Try
Dim oFile As System.IO.FileInfo
oFile = New System.IO.FileInfo(FilePath)
Dim oFileStream As System.IO.FileStream = oFile.Open
_(IO.FileMode.Open, IO.FileAccess.Read)
Dim lBytes As Long = oFileStream.Length
ReDim fileData(lBytes)
Dim StartReadAt As Long = lBytes - 100
Dim StopReadAt As Long = 100
oFileStream.Read(fileData, StartReadAt, StopReadAt)
oFileStream.Close()
Catch ex As Exception
MsgBox(ex.Message)
End Try

The problem with this code is an Array is created in the size of the file
resulting in upper bounds of maybe 200 000 000, so this is time consuming to
get the last 100 bytes.

Any idéas?
Thanks /Thomas

 

[Michael D. Ober]

Try seeking to oFileStream.Length - 100. Then read from there to the end of
the file. The array size only needs to be 100 bytes. It may take the OS a
while to do the seek, but you won't get the out of memory errors.

Mike Ober.

 

[Guest]

Ahh, thanks for the "Eye opener"
I used the binaryreader instead and set the "cursor" to the End -100
position and step to the end.

This code worked for me:

Dim fileData(100) As Byte
Dim oFileStream As New FileStream(FilePath, FileMode.Open,
FileAccess.Read)
Dim oBinaryReader As New BinaryReader(oFileStream)
Dim lBytes As Long = oFileStream.Length
oBinaryReader.BaseStream.Position = lBytes - 100
Dim fdi As Integer = 0
For i As Long = 0 To 99
fileData(i) = oBinaryReader.ReadByte()
Next i
oBinaryReader.Close()
oFileStream.Close()

Thanx
/Thomas

 

[Michael D. Ober]

Out of curiosity, is there a performance difference when seeking to end -100
on a 1K file vs a 10Gb file. I know the old FAT file system would take a
lot longer for the seek on the larger file. Also, why not use ReadBytes and
avoid your read loop?

Dim oBinaryReader As New BinaryReader(New FileStream(FilePath,
FileMode.Open, FileAccess.Read))
oBinaryReader.BaseStream.Position = oBinaryReader.BaseStream.Length - 100
Dim fileData(100) As Byte = oBinaryReader.ReadBytes(100)
oBinaryReader.Close() ' Close both the binary reader and the base
stream

Mike.

 

[Guest]

Regardless of the file size it takes about 156 milliseconds to get the last
100 bytes.
Measured with a 100kb file and a 656MB file over the network. This is very
nice since I have to read hundreds of files on a regular basis.

I get this error when trying youre example with ReadBytes(100) "Explicit
initialization is not permitted for arrays declared with explicit bounds."
But the idéa is good and I'll look further into it.

Thanks again
/Thomas

 

[Guest]

Got it, Dim as Byte array:

Dim oFileStream As New FileStream(FilePath, FileMode.Open, FileAccess.Read)
Dim oBinaryReader As New BinaryReader(oFileStream)
Dim lBytes As Long = oFileStream.Length
oBinaryReader.BaseStream.Position = lBytes - 100
Dim fileData As Byte() = oBinaryReader.ReadBytes(100)
oBinaryReader.Close()
oFileStream.Close()

Did you mean that oBinaryReader.Close automatically closes oFileStream.Close?

 

[Michael D. Ober]

According to the documentation, BinaryReader.Close() closes both the binary
reader and the underlying stream. That's why you can use a single Binary
Reader and access it's BaseStream property to get the file length and set
your desired position.

Mike.