How to save and read very big Array Value to/from file in VB.NET?

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

str_Array() 'strings
I need save it to file and next time i can read from file.

--------------------------------------------------------
--------------------------------------------------------
Save Function
I use
str_FromAndToFile=join(str_Array,"VBTAB")
--------------------------------------------------------
Read Function
I use
str_Array=Split(str_FromAndToFile,"VBTAB")
--------------------------------------------------------
--------------------------------------------------------

The problem is more str_Array() huge ,the code run more slowly.

some cost value
(C1.7-512M)
str_Array(i) Read Cost Save Cost.
100000 1.2s 1.5s
1000000 2s 13s
....

Actually I need i=1'000'000'000. :-(
Could anyone help me to improve it or give me some efficiency source code?

THX!

(e-mail address removed)
 
oncelovecoffee,
In addition to the other comments, I would iterate over the array & write
each element as a line.

To read it back I would simply read each line.

I would consider using an ArrayList as they automatically expand as needed.

Something like:

Imports System.IO
Imports System.Text

Dim str_Array As New ArrayList

Dim input As New StreamReader("myfile.in", Encoding.Default)
Dim line As String = input.ReadLine()
Do Until line Is Nothing
str_Array.Add(line)
line = input.ReadLine()
Loop


Dim output As New StreamWriter("myfile.out", False,
Encoding.Default)

For Each line As String In str_Array
output.WriteLine(line)
Next


Actually I need i=1'000'000'000. :-(
Click to expand...
However if you really have a billion lines, you may want to consider
upgrading to .NET 64 bit edition, as 32 bit .NET only supports 2G of memory.

http://lab.msdn.microsoft.com/vs2005/

Unfortunately the 64 bit edition is only available as Beta 1 currently, you
will need to wait until sometime in 2005 for the release.

Alternatively consider reading only the line you need to work on and write
it.


Dim input As New StreamReader("myfile.in", Encoding.Default)
Dim output As New StreamWriter("myfile.out", False,
Encoding.Default)
Dim line As String = input.ReadLine()
Do Until line Is Nothing
' do something with line
output.WriteLine(line)
line = input.ReadLine()
Loop

Hope this helps
Jay
 
At last ,I use the policy "Used, Record it",if one of the array member's value is default .then I don't touched it:-)
:EXAMPLE
0 XXX
1 XXX
2 DEFAULT
3 DEFALUT
..
..
..
999999999 XXX
1000000000 DEFALUT

i record

0 XXX
1 XXX
..
..
999999999 XXX

it reduce the size of file and save time to write file.
the worst conditation is I still should record them all if every value changed
:-(

And good news is 1000 times Rnd test shows about 30% of array members's value changed (AVG). :-)
 
Back
Top