Serialization

[colin]

Hi,
I need to transfer data structures to/from a fixed file format,
some of the format is quite easy its just a binary store of
char/int/.../floats etc.
however some of the fields are stored as variable size arrays of structures
etc
and some individual fields are stored as comressed.
some of the structures are nested quite deep.

also some of the feilds are not stored the same way,
ie references to objects are stored via an index in a table.

whats the best way to do this in c# ?

ive worked out various ways I can do this,
such as a class type for each data item and code to write each field to the
file
in order but there are so many structures I realy need a way of specifyng
how each field is stored in the file.

ive been looking at the serialization functionality,
but im not too clear on this so far,
or if its going to be any easier to do.
being able to just set an attribute for each
member would be the most convenient way to do it
but ive only found the attribute to not serialize a member,
rather than the opposite wich would be handier
is there any custom attributes that can be used ?


Colin =^.^=

 

[Andrew Faust]

It sounds like your structure is too complex for the native serialization
capabilities in .Net to handle. However, you can always build your own
serialization formatters.
http://msdn.microsoft.com/msdnmag/issues/04/10/AdvancedSerialization/

However, if you build your own serialization formatter you're still stuck
writing the code to read and write this file manually.

ive worked out various ways I can do this,
such as a class type for each data item and code to write each field to
the file
in order but there are so many structures I realy need a way of specifyng
how each field is stored in the file.

Unfortunately, this may well be the best method. You can structure it in
such a way to make it manageable, though. You could have each class
implement an interface similar to this:

void ReadFromStream(Stream stream);
void WriteToStream(Stream stream);

Then have your main class call the Read or Write for each class type as
it's needed. This way you can keep the code broken up in to manageable
chunks.

 

[colin]

Hi, thanks, yes ive been reading up on it,
the ms help is either so skimpy on explnations or the code examples so long
its realy
hard to pick out the info I want,
however it seems to be geared for either xml or binary dumping.

I have been looking further at the Type info wich seems versatile,
you can list all the members of a structure at run time,
you could then serialise each one by type etc,
ive also looked into custom attributes,
these seem realy versatile too.

I could have atributes for each member I want to serialize
wich specifies the storage format etc,
this seems very versatile,
the attributes are simply put against the structure members,
the structure is otherwise totaly unafected.
also the structure itself defines the members for serialization
wich removes the posibilty of missing a discrepancy in a function

Im not sure yet if the getmemberlist() function gives al the members in
order,
but it would be posible to put positional information in the attributes.

the only down side is the speed of reading some of the files are 40mb+
it would be possible to actualy have the program create code wich
could either be compiled at run time or put into the project itself.
this is perhaps rather fiddly though.

but at least I can try it the slow way first

unless theres any better ways, il sleep on it a bit first.
im sure someones done this before,
I have some c++ code to folow but this is hard to folow and hard to debug
as all the references are indexes/pointers etc,
and heavy use of macros.

heres some info on the file types ..
http://wiki.beyondunreal.com/wiki/Package_File_Format
looks fairly easy to start with ... then
http://wiki.beyondunreal.com/wiki/Package_File_Format/Data_Details
and theres a lot of info ive had to gues at

Colin =^.^=

It sounds like your structure is too complex for the native serialization
capabilities in .Net to handle. However, you can always build your own
serialization formatters.
http://msdn.microsoft.com/msdnmag/issues/04/10/AdvancedSerialization/

However, if you build your own serialization formatter you're still stuck
writing the code to read and write this file manually.

ive worked out various ways I can do this,
such as a class type for each data item and code to write each field to
the file
in order but there are so many structures I realy need a way of specifyng
how each field is stored in the file.

Unfortunately, this may well be the best method. You can structure it in
such a way to make it manageable, though. You could have each class
implement an interface similar to this:

void ReadFromStream(Stream stream);
void WriteToStream(Stream stream);

Then have your main class call the Read or Write for each class type as
it's needed. This way you can keep the code broken up in to manageable
chunks.

--
Andrew Faust
andrew[at]andrewfaust.com
http://www.andrewfaust.com

Hi,
I need to transfer data structures to/from a fixed file format,
some of the format is quite easy its just a binary store of
char/int/.../floats etc.
however some of the fields are stored as variable size arrays of
structures etc
and some individual fields are stored as comressed.
some of the structures are nested quite deep.

also some of the feilds are not stored the same way,
ie references to objects are stored via an index in a table.

whats the best way to do this in c# ?

ive worked out various ways I can do this,
such as a class type for each data item and code to write each field to
the file
in order but there are so many structures I realy need a way of specifyng
how each field is stored in the file.

ive been looking at the serialization functionality,
but im not too clear on this so far,
or if its going to be any easier to do.
being able to just set an attribute for each
member would be the most convenient way to do it
but ive only found the attribute to not serialize a member,
rather than the opposite wich would be handier
is there any custom attributes that can be used ?


Colin =^.^=