serialisation issues

[Dan]

I seem to find serilisation strangely difficult, i do the following:

Create a memory stream
Create a binary formatter
Read my object into the memorystream by using the binaryformatter serialise
method
then i send the memorystream.getbuffer over the socket.

At the client side receiving that info:

i create a memory stream
create a binary formatter
write my received bytes into the memory stream
deserialise the memorystream using binaryformatter.deserialise to an object

.....at this deserialisation point it all breaks, so it seems to serialise
with no problems.

In order to test why this was i, in the same method, serialised an object
then immediately deserialised it and still had an error which i woul dhave
thought eliminates the issue of incomplete data over a socket.

Really stuck on this one, i have read tuts on the net all of which match my
code, what am i doing wrong?

 

[Marc Gravell]

a: what error?

b: when testing locally (serialize and then deserialize) did you reset the
stream position?

c: when serializing remotely, does the second project reference the exact
same object? typically you would need a shared common assembly (just
declaring the serialized objects) which both projects reference; having the
same cs file in both does not suffice. Alternatively, surrogates can be
used.

d: if none of the above helps, have you any small (but complete) snippet
that illustrates it?

Marc

 

[Dan]

Hi Marc,

Thank you for your reply.

a: presently i am catching the error and posting it in a message box to show
the trace, but it only tells me the line it is on, how do get the exception
to tell me the usual EndOfStream etc type of error (bit of a newbie question
i know) To be honest i have been very slack on excpetion handling in the
past which i have had to change on this project where i am catching
everything.

b: No i did not and the error there (without catching) was endof
stream.........i am feeling a 'duh' situation here. I didnt realise you had
to reset the position (i am very new to working with streams) . How do i
reset the memorystream position? And i presume from your point that when a
stream is read it will always need resetting?.....actually this is making
sense to how it knows you have reached the end of the stream, if it doesnt
restart each time that would explain how you can keep waiting until a stream
is correctly filled as long as it is initially set to the expected size!
Eureka, am i right??

c: The second project does use a cs file which matches and not a common
assembly! I did not realise that would be a problem since it is the same cs.
Could you tell me how i can set to projects to use the same assembly so that
my server side and client side share the exact same files?

I wont post a code snippet as based on your points my code is fine (others
have seen it and agreed) but the fundamentals and concepts have been
implemented wrong by myself. Look forward to hearing from you in regards to
the above.

 

[Marc Gravell]

a: if you are catching an exception, just do .GetType() on the exception

b: no: you shouln't normally monkey with the position - however, you are
dowing a write-then-read: after writing an object the stream-cursor stays at
the END of the stream awaiting more input; however, to read that same object
you need to go back to the START of the stream to read the data you just
wrote; stream.Position = 0 should do the job; this is only for this debug
test - there should be no need to do that at the receiver since this stream
will initialise at the START. Also: you shouldn't assume that streams are
always seekable. MemoryStream *is* which is handy for your test, but many
aren't.
So yes, eureka... although normally you dont set a stream "to the expected
size" - you simply keep reading until it refuses to give you any more
(although a check of how much you expect is always sensible).

c: Create a new project; put your cs in there; build it. Remove the cs from
both the other two. Add a reference from both the "main" projects to the new
assembly. Build 'em. Test. That's the short version (sorry, I'm in a hurry)
;-p Best to strong-name the other assembly as well.
The problem here is that a type is scoped by it's assembly, so *even if the
namespaces are identical* MyClass in Assembly1 is treated as different to
MyClass in Assembly2.

d: Yeah - I suspect that the above will get you running; if it still doesn't
play ball, post some code.

Marc

 

[Dan]

Excellent! Really explained it all and when i sorted my error messages
suddenly my probelms became clear.

My receive end was doing this:

MemoryStream ms = new MemoryStream ();



ms.Write(recvData,0,bytesReceived);

ms.Position = 0;


BinaryFormatter b = new BinaryFormatter();

someObject myObj = (someObject) b.Deserialize( ms );

As you can see i added the ms.Position=0; Which fixed it and put in the
assembled versions. Above i write the bytes received over the socket into
the stream and then deserialise the stream. After you said it goes to the
end it became clear.

I take it there is a way of doing the above without setting the position to
0? Either way it works a treat!

Thanks!!!

 

[Marc Gravell]

Cheers for that - appreciated; although to be honest I'm just trying to
repay the kind help from others on this list in the past; "pay it forward"
etc.

As for the question: well, it depends on what you are doing ;-p

It isn't clear in you code where recvData and bytesReceived have come from;
I suspect that they have come from a call to Read on the input socket
stream, in which case you could ditch MemoryStream completely and
deserialize directly from the original stream. Depending on how you are
obtaining recvData, your code may not be the safest option:

Firstly (not the real reason: just a gentle warm up) - what if your object
is bigger than your buffer? You have written one block (bytesReceived
long) - but what if your buffer is 512 bytes but the object takes 2000? Your
source isn't there, so for all I know you have looped until Read returns 0,
but that isn't what the text describes.

Secondly (more interestingly) note that any call to stream.Read is only
guaranteed to return "some" data (if still open) - not necessarily the
amount you asked for even if the stream has not finished; that is why I said
that normally you just keep reading until either the stream ends or you have
read what you are interested in. When you call Deserialize it will do this
for you - i.e. it it tries to deserialize a 2000 byte object, it will keep
calling Read until *either* it has got 2000 bytes *or* stream.Read returns 0
(meaning the end of the stream), in which case it will throw an EOF.

What you have written is safe *only* if you have copied enough data to the
memory stream; but how much is enough? For any non-trivial object it is hard
to tell unless you pass this figure directly in advance - but adding this
would be duplication as the serialized form of the object already includes
this data; personally, I'd try and deserialize directly from the origianl
(input socket) stream.

Does that help some more?

Marc

 

[Marc Gravell]

Additional treatise on streams:

Generally, one of the concepts of using a stream is that you don't need to
have the entire data before you can start processing it; by copying to a
memory-stream you are essentially treating the data as a byte-array;
possibly convenient, but for large-scale use that can hugely increase the
amount of data in memory at any one time. Conversely, by reading directly
from the source stream, you are (possibly - depends on timing, buffering,
etc) reading the data *as it arrives* from the source.

The simplest example here is a file; treating this as a memory-stream you
have to load the entire file into memory *first* - but treating it as a
file-stream you can read the data off the disk as you need it. Some streams
never end: it would take about 2 minutes to write a RandomStream that
returns random data forever (using Random.GetBytes) - and this is still
perfectly valid.

Reading from the original stream is *generally* going to be more efficient -
you are then essentially in "fire hose" mode; you have one chance to read
data as it arrives, and then it is gone (forever). Sometimes code needs two
passes on data to fully process it, which might involve copying it to a
second (seekable) stream (or some other buffer), but this should be the
exception.

Marc