Serializing Objects Across A NetworkStream

[Guest]

I've got a basic "chat" infrastructure that can send objects across the wire, but I'm running into a few issues with trying to send multiple messages and things behaving differently in debug and release...

Is there anyone out there that considers themselves a threading / sockets / serialization guru? Not sure if it's appropriate to post an entire project here and the issues discussion is probably a bit too complex for newsgroup back and forth, but if someone thinks they can help, I can send the code via email.

I suspect there might be a simpler way to accomplish my goal, but I've not got enough experience in this realm...

If you're up for the challenge, please let me know.

Thanks.

Jerry

 

[Nicholas Paldino [.NET/C# MVP]]

Jerry,

I wouldn't recommend posting the whole project. I can't say that many
people would want to wander through it all.

If you can consolidate your errors into a sample program that displays
the error, that would help.

As for issues discussion about what you are doing, that's pretty much
the whole point of the newsgroups, no? I would say try us.

Hope this helps.


--
- Nicholas Paldino [.NET/C# MVP]
- (e-mail address removed)

I've got a basic "chat" infrastructure that can send objects across the
wire, but I'm running into a few issues with trying to send multiple
messages and things behaving differently in debug and release...

Is there anyone out there that considers themselves a threading / sockets /
serialization guru? Not sure if it's appropriate to post an entire project
here and the issues discussion is probably a bit too complex for newsgroup
back and forth, but if someone thinks they can help, I can send the code via
email.

I suspect there might be a simpler way to accomplish my goal, but I've not
got enough experience in this realm...

If you're up for the challenge, please let me know.

Thanks.

Jerry

 

[Guest]

I just figured the postings might get too hard to follow... Not a lack of
confidence in the community. )

I can't really consolidate the project. There's nothing else in it right
now except the communication code. I suppose I could try to explain my
thought process and post the whole deal... It's not all that big, just
totally uncommented and is quite a mess as I've been trying to figure this
out for several days and keep commenting out my old code - for reference. I
figured it's be easier to explain what I was thinking via email. If people
keep chiming in in the newsgroup, the thread gets hard to follow.

I'll try to clean it up a bit and post it...

Thanks.

Jerry

 

[Peter Huang [MSFT]]

Hi Jerry,

Based on my research, the problem seems to be relative with how you parse
the buffer and show them.

Actually TCP protocol is stream-like protocol.
We should consider it all the communication between the TCPClient is a
stream.
e.g.
if ClientA call 5 five times NetStream.Write and the TCPClient is connect
to Client B.
The Client may call 4 or 6 times NetStream.Read to get all the data sent by
Client A.
So in your scenario, we can depend on the Enqueue operation to put the
whole byte array into the queue.

Here is my test to show this scenario.

1. Please change your code as below.
static int i = 0;
private void GetStreamObject(IAsyncResult ar)
{

Debug.WriteLine(++i);
byte[] buffer;
int bytes = 0;

//Extract state object from AsyncResult and get a local pointer to the
NetworkStream.
ObjectInfo oi = (ObjectInfo)ar.AsyncState;
NetworkStream ns = oi.networkStream;

lock(ns)
{
//Read the send buffer.
bytes = ns.EndRead(ar);
//Debug.WriteLine(bytes);

//Create a local buffer the size of the received data.
buffer = new byte[bytes];
//Copy the received data into the appropriate sized buffer.
Buffer.BlockCopy(oi.buffer, 0, buffer, 0, bytes);
StreamWriter sw = new StreamWriter(@"C:\buffer.txt",true);
sw.WriteLine("*");
foreach(byte b in buffer)
sw.Write(b);
sw.WriteLine("#");
sw.Close();
//Analyze what we have.
AnalyzeBytes(buffer);

//Create a new buffer for the next read.
//Probably don't really need this.
oi.buffer = new byte[BUFFER_SIZE];

//Start another async read.
AsyncCallback GetStreamCallback = new AsyncCallback(GetStreamObject);
tcpClient.GetStream().BeginRead(oi.buffer, 0, oi.buffer.Length,
GetStreamCallback, oi);
}
}

I Start the your program and use 5 in the for loop to send 5 objects in one
button click.

The result is that.
1. The Debug.WriteLine(++i); show that the method GetStreamObject is only
called 4 times including the time we click the connect button.
2. I write the buffer to a file as attachment.

We will find that *# include the data we get from one receive.

We will find that the data in one receive *# maybe include two and half
objects' data but not whole object.
Also you may try to find the pattern "01000255" in the result below, you
will find that it occur five times exclude the first connect message.
Which means that actually we have received all the data with you reproduce
sample.

So I think the key point in your scenario is to design another method to
parse the whole buffer stream during the connected time, but not break the
stream into the items in the Queue. Because the item(byte array) may not be
a whole complete object).

Anyway your idea that use a header to show following objects size is a good
idea and that is what we will use network programming.

e.g. We must define the size of the Header which show what is the size of
the object.
Header---Date----Header----Data....

We should define the Header Size e.g. 16 bytes.

Please note the above data may not be in the four receive operations.
Maybe just one time receive, we will got all the data.

What we should do is.
Read 16bytes from the buffer we received, and know the size of the object
and then read the size to get object. And then accordind our design here,
we should read another 16bytes to read the data....





*
4010001000255255255255100000001220007477101115115971031016710897115115443286
1011141151051111106149464846505254494651495650564432671171081161171141016111
0101117116114971084432801179810810599751011218411110710111061110117108108510
0017679710910110884971081074677101115115971031013000137710111511597103101781
1710998101114137710111511597103101831161141051101031177101115115971031018412
1112101014830679710910110884971081074677101115115971031014377101115115971031
0184121112101115200020000000630002067111110110101991161011003211611132115101
1141181011144652522552552553067971091011088497108107467710111511597103101437
7101115115971031018412111210111510007118971081171019595082000000011#
*
21100#
*
0100025525525525510000000122000747710111511597103101671089711511544328610111
4115105111110614946484650525449465149565056443267117108116117114101611101011
1711611497108443280117981081059975101121841111071011106111011710810851000176
7971091011088497108107467710111511597103101300013771011151159710310178117109
9810111413771011151159710310183116114105110103117710111511597103101841211121
0101483067971091011088497108107467710111511597103101437710111511597103101841
2111210111520002000100063000197525225525525530679710910110884971081074677101
1151159710310143771011151159710310184121112101115100071189710811710195950820
0010001121100010002552552552551000000012200074771011151159710310167108971151
1544328610111411510511111061494648465052544946514956505644326711710811611711
4101611101011171161149710844328011798108105997510112184111107101110611101171
0810851000176797109101108849710810746771011151159710310130001377101115115971
0310178117109981011141377101115115971031018311611410511010311771011151159710
3101841211121010148306797109101108849710810746771011151159710310143771011151
1597103101841211121011152000200010006300019752522552552553067971091011088497
1081074677101115115971031014377101115115971031018412111210111510007118971081
1710195950820001000112110001000255255255255100000001220007477101115115971031
0167108971151154432861011141151051111106149464846505254494651495650564432671
1710811611711410161110101117116114971084432801179810810599751011218411110710
1110611101171081085100017679710910110884971081074677101115115971031013000137
7101115115971031017811710998101114137710111511597103101831161141051101031177
1011151159710310184121112101014830679710910110884971081074677101115115971031
0143771011151159710310184121112101115200020001000630001975252255255255306797
1091011088497108107467710111511597103101437710111511597103101841211121011151
0007118971081171019595082000100011211000100025525525525510000000122000747710
1115115971031016710897115115443286101114115105111110614946484650525449465149
5650564432671171081161171141016111010111711611497108443280117981081059975101
1218411110710111061110117108108510001767971091011088497108107467710111511597
1031013000137710111511597103101781171099810111413771011151159710310183116114
1051101031177101115115971031018412111210101483067971091011088497108107467710
1#
*
1151159710310143771011151159710310184121112101115200020001000630001975252255
2552553067971091011088497108107467710111511597103101437710111511597103101841
2111210111510007118971081171019595082000100011211000100025525525525510000000
1220007477101115115971031016710897115115443286101114115105111110614946484650
5254494651495650564432671171081161171141016111010111711611497108443280117981
0810599751011218411110710111061110117108108510001767971091011088497108107467
7101115115971031013000137710111511597103101781171099810111413771011151159710
3101831161141051101031177101115115971031018412111210101483067971091011088497
1081074677101115115971031014377101115115971031018412111210111520002000100063
0001975252255255255306797109101108849710810746771011151159710310143771011151
15971031018412111210111510007118971081171019595082000100011#



Best regards,

Peter Huang

Microsoft Online Community Support
==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscriptions/managednewsgroups/default.aspx#notif
ications.
Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscriptions/support/default.aspx.
==================================================
(This posting is provided "AS IS", with no warranties, and confers no
rights.)

 

[Guest]

Peter - You're brilliant. Simple debugging technique of writing out the
bytes, etc. made the whole process more apparent to me. At one point or
another I had counters and such everywhere but in the right places. You're
insight is very much appreciated.

So my next question is would it be considered an appropriate solution to
just create thread that loops continuously looking at the byte queue and
raising an event everytim eit receives a full object? They seem to do that
in all the TcpListener examples, but I don't know if that's really the best
way to go about it.

And when I raise an event, is the thread that raises the event blocked until
the event recipeint is done processing it?

Thanks, again.

Jerry

Hi Jerry,

Based on my research, the problem seems to be relative with how you parse
the buffer and show them.

Actually TCP protocol is stream-like protocol.
We should consider it all the communication between the TCPClient is a
stream.
e.g.
if ClientA call 5 five times NetStream.Write and the TCPClient is connect
to Client B.
The Client may call 4 or 6 times NetStream.Read to get all the data sent
by
Client A.
So in your scenario, we can depend on the Enqueue operation to put the
whole byte array into the queue.

Here is my test to show this scenario.

1. Please change your code as below.
static int i = 0;
private void GetStreamObject(IAsyncResult ar)
{

Debug.WriteLine(++i);
byte[] buffer;
int bytes = 0;

//Extract state object from AsyncResult and get a local pointer to the
NetworkStream.
ObjectInfo oi = (ObjectInfo)ar.AsyncState;
NetworkStream ns = oi.networkStream;

lock(ns)
{
//Read the send buffer.
bytes = ns.EndRead(ar);
//Debug.WriteLine(bytes);

//Create a local buffer the size of the received data.
buffer = new byte[bytes];
//Copy the received data into the appropriate sized buffer.
Buffer.BlockCopy(oi.buffer, 0, buffer, 0, bytes);
StreamWriter sw = new StreamWriter(@"C:\buffer.txt",true);
sw.WriteLine("*");
foreach(byte b in buffer)
sw.Write(b);
sw.WriteLine("#");
sw.Close();
//Analyze what we have.
AnalyzeBytes(buffer);

//Create a new buffer for the next read.
//Probably don't really need this.
oi.buffer = new byte[BUFFER_SIZE];

//Start another async read.
AsyncCallback GetStreamCallback = new AsyncCallback(GetStreamObject);
tcpClient.GetStream().BeginRead(oi.buffer, 0, oi.buffer.Length,
GetStreamCallback, oi);
}
}

I Start the your program and use 5 in the for loop to send 5 objects in
one
button click.

The result is that.
1. The Debug.WriteLine(++i); show that the method GetStreamObject is only
called 4 times including the time we click the connect button.
2. I write the buffer to a file as attachment.

We will find that *# include the data we get from one receive.

We will find that the data in one receive *# maybe include two and half
objects' data but not whole object.
Also you may try to find the pattern "01000255" in the result below, you
will find that it occur five times exclude the first connect message.
Which means that actually we have received all the data with you reproduce
sample.

So I think the key point in your scenario is to design another method to
parse the whole buffer stream during the connected time, but not break the
stream into the items in the Queue. Because the item(byte array) may not
be
a whole complete object).

Anyway your idea that use a header to show following objects size is a
good
idea and that is what we will use network programming.

e.g. We must define the size of the Header which show what is the size of
the object.
Header---Date----Header----Data....

We should define the Header Size e.g. 16 bytes.

Please note the above data may not be in the four receive operations.
Maybe just one time receive, we will got all the data.

What we should do is.
Read 16bytes from the buffer we received, and know the size of the object
and then read the size to get object. And then accordind our design here,
we should read another 16bytes to read the data....





*
4010001000255255255255100000001220007477101115115971031016710897115115443286
1011141151051111106149464846505254494651495650564432671171081161171141016111
0101117116114971084432801179810810599751011218411110710111061110117108108510
0017679710910110884971081074677101115115971031013000137710111511597103101781
1710998101114137710111511597103101831161141051101031177101115115971031018412
1112101014830679710910110884971081074677101115115971031014377101115115971031
0184121112101115200020000000630002067111110110101991161011003211611132115101
1141181011144652522552552553067971091011088497108107467710111511597103101437
7101115115971031018412111210111510007118971081171019595082000000011#
*
21100#
*
0100025525525525510000000122000747710111511597103101671089711511544328610111
4115105111110614946484650525449465149565056443267117108116117114101611101011
1711611497108443280117981081059975101121841111071011106111011710810851000176
7971091011088497108107467710111511597103101300013771011151159710310178117109
9810111413771011151159710310183116114105110103117710111511597103101841211121
0101483067971091011088497108107467710111511597103101437710111511597103101841
2111210111520002000100063000197525225525525530679710910110884971081074677101
1151159710310143771011151159710310184121112101115100071189710811710195950820
0010001121100010002552552552551000000012200074771011151159710310167108971151
1544328610111411510511111061494648465052544946514956505644326711710811611711
4101611101011171161149710844328011798108105997510112184111107101110611101171
0810851000176797109101108849710810746771011151159710310130001377101115115971
0310178117109981011141377101115115971031018311611410511010311771011151159710
3101841211121010148306797109101108849710810746771011151159710310143771011151
1597103101841211121011152000200010006300019752522552552553067971091011088497
1081074677101115115971031014377101115115971031018412111210111510007118971081
1710195950820001000112110001000255255255255100000001220007477101115115971031
0167108971151154432861011141151051111106149464846505254494651495650564432671
1710811611711410161110101117116114971084432801179810810599751011218411110710
1110611101171081085100017679710910110884971081074677101115115971031013000137
7101115115971031017811710998101114137710111511597103101831161141051101031177
1011151159710310184121112101014830679710910110884971081074677101115115971031
0143771011151159710310184121112101115200020001000630001975252255255255306797
1091011088497108107467710111511597103101437710111511597103101841211121011151
0007118971081171019595082000100011211000100025525525525510000000122000747710
1115115971031016710897115115443286101114115105111110614946484650525449465149
5650564432671171081161171141016111010111711611497108443280117981081059975101
1218411110710111061110117108108510001767971091011088497108107467710111511597
1031013000137710111511597103101781171099810111413771011151159710310183116114
1051101031177101115115971031018412111210101483067971091011088497108107467710
1#
*
1151159710310143771011151159710310184121112101115200020001000630001975252255
2552553067971091011088497108107467710111511597103101437710111511597103101841
2111210111510007118971081171019595082000100011211000100025525525525510000000
1220007477101115115971031016710897115115443286101114115105111110614946484650
5254494651495650564432671171081161171141016111010111711611497108443280117981
0810599751011218411110710111061110117108108510001767971091011088497108107467
7101115115971031013000137710111511597103101781171099810111413771011151159710
3101831161141051101031177101115115971031018412111210101483067971091011088497
1081074677101115115971031014377101115115971031018412111210111520002000100063
0001975252255255255306797109101108849710810746771011151159710310143771011151
15971031018412111210111510007118971081171019595082000100011#



Best regards,

Peter Huang

Microsoft Online Community Support
==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscriptions/managednewsgroups/default.aspx#notif
ications.
Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscriptions/support/default.aspx.
==================================================
(This posting is provided "AS IS", with no warranties, and confers no
rights.)

 

[Peter Huang [MSFT]]

Hi,

I think you may try to use a new thread per TCP client and use Read block
call.
e.g.
Header 16bytes
Object data

This is the pseudocode
while(true)
{
//Use Read in a loop, but not beginread, until we get 16bytes from the
networkstream.

//new a byte array of size of the value in the 16bytes header.

//Use Read in a loop, until we get data of the size of object.
//Now we can parse it now, because one whole object is ready.
//We can raise an event to notify one object is ready
//or temporality put the object(we have already deserialized it into a
Queue and the client UI should have mechanism to Monitor the queue.
}




Best regards,

Peter Huang

Microsoft Online Community Support
==================================================
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
==================================================
This posting is provided "AS IS" with no warranties, and confers no rights.