writing multi dimensional array of data to text file

[daqmaneng]

does anyone have any suggestions for writing a two dimensional array of
data to a file.

Currently, I have an array of 5 columns, with 1000 elements per array.
I know that I can use a for next loop to go through each data point in
the array and use the streamwriter class for each individual data point
and write each point individually to the file.

I would like to simplify this by simply writing the entire array to
file.

My array could be as big as 100,000 elements, so I'd like to find a
more efficient method,.

thanks

 

[AMDRIT]

Take a look at
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter and see if
that will help you out. Or did you need for that data to be human readable?

 

[daqmaneng]

Ultimately it will need to be presented in a text file, and may likely
even need to be exported to Excel. If I need to save it as a binary
first and then convert to text (or numeric), so be it.

I'm really suprised that there is not a method to invoke the writing of
an entire array to a text file. Seems like it would be a common need
for programmers.

 

[AMDRIT]

The question there is portability and reusability. Presemably, when you
write out to file for later resue you would use the same program to read it
back in. When the need arises that some other program is expected to
reconstruct the data (i.e. the human factor) then you have to be a bit more
explicit.

Is it for an IBM, then you have to export in EBSIDC
Is it for a PC, the ASCII will prolly suit you fine.
Does it need to go over the web, then perhaps XML is the key for you.
The tried and true way is as a simple Byte(), of course who can read that?

While the need may be ever present in a programmer's world, don't sell
yourself short that the solution is prewrapped. It is bad enough that a lot
of functionality is prewrapped and the libraries are bloated to the hilt.
But because we are programmers, we know how to create our own utilities to
accomplish our tasks as needed.

Rather than use an array, perhaps you can use an arraylist, or even better
system.collection.collectionbase? Nest your data as needed then provide a
nice little tostring() method or serialize(filename) method. Another option
is to pass the Array into a custom array serializer, to minimize the bloat
while you are working with the data.

 

[daqmaneng]

Thanks for the suggestions.

I'm involed in a project for data acquisition, where the data is coming
into my application at high rates (can be up to 1MgHz, in chunks of a
few thousand samplesas single data types). Generally, the end user
needs to store the data to a text file or excel. As excel cannot keep
up with the higher data transfer rates (as well as excessive file
sizes), the text file is the preffered saving method. The end user
only needs to access the data via a text file format.

I'll certainly investigate some of your recommendations for a solution.

Thanks again, it is very apprecaited.