I would like to read a text file (fixed length formaated) really fast and
store the data into an Access database (2003).
Using the streamreader and reading line by line, separating the line into
string just takes to long.
When I Import the file with access manually goes fast. But how to use this
fromout a C# programme
The fastest way I can think of is ugly, but perhaps worth investigating. If
your lines of text are all exactly the same length, and the fields within
the line are in exactly the same places, you could read the entire file into
an array of char, and use a struct with explicit layout to get the fields.
For instance, if your lines have fields A, B, C, and D starting at positions
0, 8, 12, and 27, with a total line length of 60 characters, then you could
do something like this:
[StructLayoutAttribute(LayoutKind.Explicit)]
struct RecordParser {
[FieldOffsetAttribute(0)] char[8] A;
[FieldOffsetAttribute(8)] char[4] B;
[FieldOffsetAttribute(12)] char[15] C;
[FieldOffsetAttribute(27)] char[33] D;
}
....
public unsafe void ParseFile() {
char[] theFile = ...; // do whatever you need to do to read the file
into a char[] buffer
int lineLength = 60;
RecordParser* record;
for(int RecordPosition=0; RecordPosition < theFile.Length;
RecordPosition+=lineLength) {
record = (RecordParser *) (&(theFile[RecordPosition]));
// do something useful with record->A, record->B, etc
}
}
Depending on the actual types you're trying to extract out of your lines of
text, you may have to do some additional work "do something useful" stage to
parse the individual fields into the proper types (e.g. converting "120.17"
to an actual float or double value, trim trailing whitespace, etc.)
Note that because we're using unmanaged pointers to coerce raw memory
(slices of the char array) into acting like structs, we're violating the
Common Language Runtime's type system and therefore we must mark the
ParseFile method as "unsafe" to satisfy the compiler that we know what we're
doing (for that matter, you may also need to mark the RecordParser struct as
"unsafe" as well; I'm not sure about that offhand). Also, you must add the
"/unsafe" command line switch to your compilation command.
For a much more thorough treatment of this type of memory access from C#,
see chapter 10 of Don Box's excellent book "Essential .NET Volume 1".
