TCP/IP Client/Server question

[Yossarian]

I have a handheld running CE .NET 4.2 and I am using c# with framework
1.1 to develop a solution for syncing data that is on the handheld with
the local pc.

Our handheld cradles only support network connections, no usb or
serial, so i have to use networking to get the data transfered.

Here's the problem. I have the server accepting connections fine, but
i'm a little confused how to actually send the server data, and then
how the server should receive the data.

Here's what i want it to do, please let me know if this is the wrong
way to use tcp.
1) Client establishes a connection to the server

2) Client sends data.
2.1) Client creates a byte array the size of one class object, fills
it with an instance of that object, sends it to the server. It does
this multiple times for every object, so in a for loop it will cycle
through an array of objects and send them one at a time.
OR
2.2) it creates a byte array and fills it with all of the objects it
needs to send, and then does one send statement pushing all the data at
once.

3) Server receives data
3.1) Server recieves that data into a byte array the size of one
object, decodes the object and puts it in an array for later use.
Server then receives the next number of bytes for one object, decodes
it, stores it, and moves on, forever until all objects have been
receieved.
OR
3.2) Server recieves all of the data and puts it in a big string or
something, knowing that the client added a delimeter to figure out when
an object begins and ends, and then after all data has been received it
parses all that.

Here's what i am doing now. Client builds big byte array with all
objects. Each object is encoded in a pre-determined format and the
server knows how to decode it. Client has one send statement that
sends the entire array. Server pulls in 35 bytes, decodes those bytes
into a class object, then pulls in the next 35 bytes, decodes, etc
etc....

The problem: If i don't put some sort of Thread.Sleep(10) the server
decodes the bytes faster than the client sends them so i don't recieve
all the data. Here's some example code:

Client:
ItemQuote.ItemQuote[] quotes = new ItemQuote.ItemQuote[10000];
int itemNumber = 0;
TcpClient client = new TcpClient('127.0.0.1', 1234);
NetworkStream netStream = client.GetStream();

for (int i=0; i<20; i++)
{
quotes = new ItemQuote.ItemQuote(itemNumber, "5mm Super Widgets",
1000, 12999,
true, false);
itemNumber += 1;

}

ItemQuoteEncoderBin coder = new ItemQuoteEncoderBin();
byte[] codedQuote = coder.encode(quotes);

netStream.Write(codedQuote, 0, codedQuote.Length);

netStream.Close();
client.Close();

Notes: Fill quotes array with ItemQuote objects, each with a different
itemNumber value. Encode the array into a byte array where each object
is just layed out in a row. Write the data to the network stream and
close the stream and connection.

Server:
(Note: clntSock is assigned elsewhere in the class. It is a TcpClient
object)
NetworkStream stream = clntSock.GetStream();
byte[] packet = new byte[35]; // 35 is the size of one class object in
bytes
ItemQuote.ItemQuote quote;
ItemQuoteDecoder decoder = new ItemQuoteDecoderBin();

do
{
stream.Read(packet, 0, packet.Length);
quote = decoder.decode(packet);
*do something with quote here*
Thread.Sleep(10);

} while (stream.DataAvailable);

Notes: Pretty straightforward. Get the read/write stream, read 35
bytes at a time, do something with those bytes, then continue doing
this until stream.DataAvailable is false.

Thread.Sleep(10) is required in order to keep DataAvailable to be true
before all the data is sent, but it feels like a hack rather than a
solution.

Maybe a solution would be to have the client first send the number of
objects it is sending, and do a for loop having it read that many
objects in and then quitting?

What is an exceptable solution for this? Everything i think up feels
like a hack and not a clean standard way to do it. The scenario is:
Client has a large amount of data to send, multiple class objects, and
wants to send them all at once. Server needs to accept it all and then
do something with it.

I can't use remoting because it's not supported on the compact
framework.

Thanks!

 

[Onwuka Emeka]

One way could be to put a header of some sort telling the server how large
the data actually is i.e the first byte of the data you are sending to the
server from the client can be used to tell the server the size of the data
to read. i.e

//from the client
byte[] codedQuote = coder.encode(quotes);
byte[] dataToSend = new byte[codedQuote.Length + 1]
dataToSend[0] = (byte)codedQuote.Length;
codedQuote.CopyTo(dataToSend,1);

//i also prefer using Raw sockets instead of the TcpClient class
socket.Send(dataToSend);

//on the server also using raw sockets
byte[] header = new byte[1];
byte[] message = null;
int read = socket.Recieve(header)
if(read == 1){
//it read something
message = new byte[(int)header[0]];
socket.Recieve(message);
}
//play around with message

hope this helps

I have a handheld running CE .NET 4.2 and I am using c# with framework
1.1 to develop a solution for syncing data that is on the handheld with
the local pc.

Our handheld cradles only support network connections, no usb or
serial, so i have to use networking to get the data transfered.

Here's the problem. I have the server accepting connections fine, but
i'm a little confused how to actually send the server data, and then
how the server should receive the data.

Here's what i want it to do, please let me know if this is the wrong
way to use tcp.
1) Client establishes a connection to the server

2) Client sends data.
2.1) Client creates a byte array the size of one class object, fills
it with an instance of that object, sends it to the server. It does
this multiple times for every object, so in a for loop it will cycle
through an array of objects and send them one at a time.
OR
2.2) it creates a byte array and fills it with all of the objects it
needs to send, and then does one send statement pushing all the data at
once.

3) Server receives data
3.1) Server recieves that data into a byte array the size of one
object, decodes the object and puts it in an array for later use.
Server then receives the next number of bytes for one object, decodes
it, stores it, and moves on, forever until all objects have been
receieved.
OR
3.2) Server recieves all of the data and puts it in a big string or
something, knowing that the client added a delimeter to figure out when
an object begins and ends, and then after all data has been received it
parses all that.

Here's what i am doing now. Client builds big byte array with all
objects. Each object is encoded in a pre-determined format and the
server knows how to decode it. Client has one send statement that
sends the entire array. Server pulls in 35 bytes, decodes those bytes
into a class object, then pulls in the next 35 bytes, decodes, etc
etc....

The problem: If i don't put some sort of Thread.Sleep(10) the server
decodes the bytes faster than the client sends them so i don't recieve
all the data. Here's some example code:

Client:
ItemQuote.ItemQuote[] quotes = new ItemQuote.ItemQuote[10000];
int itemNumber = 0;
TcpClient client = new TcpClient('127.0.0.1', 1234);
NetworkStream netStream = client.GetStream();

for (int i=0; i<20; i++)
{
quotes = new ItemQuote.ItemQuote(itemNumber, "5mm Super Widgets",
1000, 12999,
true, false);
itemNumber += 1;

}

ItemQuoteEncoderBin coder = new ItemQuoteEncoderBin();
byte[] codedQuote = coder.encode(quotes);

netStream.Write(codedQuote, 0, codedQuote.Length);

netStream.Close();
client.Close();

Notes: Fill quotes array with ItemQuote objects, each with a different
itemNumber value. Encode the array into a byte array where each object
is just layed out in a row. Write the data to the network stream and
close the stream and connection.

Server:
(Note: clntSock is assigned elsewhere in the class. It is a TcpClient
object)
NetworkStream stream = clntSock.GetStream();
byte[] packet = new byte[35]; // 35 is the size of one class object in
bytes
ItemQuote.ItemQuote quote;
ItemQuoteDecoder decoder = new ItemQuoteDecoderBin();

do
{
stream.Read(packet, 0, packet.Length);
quote = decoder.decode(packet);
*do something with quote here*
Thread.Sleep(10);

} while (stream.DataAvailable);

Notes: Pretty straightforward. Get the read/write stream, read 35
bytes at a time, do something with those bytes, then continue doing
this until stream.DataAvailable is false.

Thread.Sleep(10) is required in order to keep DataAvailable to be true
before all the data is sent, but it feels like a hack rather than a
solution.

Maybe a solution would be to have the client first send the number of
objects it is sending, and do a for loop having it read that many
objects in and then quitting?

What is an exceptable solution for this? Everything i think up feels
like a hack and not a clean standard way to do it. The scenario is:
Client has a large amount of data to send, multiple class objects, and
wants to send them all at once. Server needs to accept it all and then
do something with it.

I can't use remoting because it's not supported on the compact
framework.

Thanks!

 

[Barry Kelly]

Here's the problem. I have the server accepting connections fine, but
i'm a little confused how to actually send the server data, and then
how the server should receive the data.

The problem: If i don't put some sort of Thread.Sleep(10) the server
decodes the bytes faster than the client sends them so i don't recieve
all the data.

There are pretty much exactly four solutions to this problem:

1) Have the client close the connection after all data is sent, so the
server can simply read until the stream is closed.

2) Have the client send the length of content before the content
itself. This is the approach used by HTTP, for example, in the
Content-Length header.

3) Have the client send some kind of "end of data chunk" marker, and
have the server recognize it. This is the approach used by HTTP for
recognizing the end of headers and the start of content data: a blank
line after the headers indicates the end of headers.

4) Use a packet-based approach where data lumps are of a fixed size,
so both client and server know what to do, and pad out the data if it
is the wrong size.
-- Barry

 

[Barry Kelly]

do
{
stream.Read(packet, 0, packet.Length);
quote = decoder.decode(packet);
*do something with quote here*
Thread.Sleep(10);

} while (stream.DataAvailable);

By the way, you don't want to read data like this. You need to read
somewhat like:

MyPacketBuilder packetBuilder = new MyPacketBuilder();
while (packetBuilder.ReadFromStream(stream))
{
foreach (Packet packet in packetBuilder.Packets)
ProcessPacket(packet);
packetBuilder.Packets.Clear();
}

Where:
MyPacketBuilder - something which can read partial packets from a
stream and assemble them into packets.
MyPacketBuilder.ReadFromStream() - reads as many bytes are available
from the stream and tries to assemble as many packets from the bytes
read as possible, and returns false if the stream is closed (a call to
the Stream returned 0).
MyPacketBuilder.Packets - a collection of packets assembled from the
data read from the stream.

In other words, you need to perform buffer management, and
incrementally build your packets from the little byte array slices you
read from the TCP stream.

The loop above assumes a thread is dedicated to the TCP connection.
You can rework it to work with asynchronous methods like BeginRead()
etc., but I'd leave that until (1) you get it working and (2) you are
unhappy with the scalability, because working with asynchronous
methods can be quite tricky.
-- Barry

 

[Nick Hounsome]

There are pretty much exactly four solutions to this problem:

1) Have the client close the connection after all data is sent, so the
server can simply read until the stream is closed.

2) Have the client send the length of content before the content
itself. This is the approach used by HTTP, for example, in the
Content-Length header.

3) Have the client send some kind of "end of data chunk" marker, and
have the server recognize it. This is the approach used by HTTP for
recognizing the end of headers and the start of content data: a blank
line after the headers indicates the end of headers.

4) Use a packet-based approach where data lumps are of a fixed size,
so both client and server know what to do, and pad out the data if it
is the wrong size.
-- Barry

All of these still require you to accumulate data as you describe in your
other post.

Even with 4 using small "packets" it is entirely legal and possible
(although hugely unlikely) for an intermediate network node to chop it up.

There is no way around this because TCP is just a stream.

 

[Barry Kelly]

Even with 4 using small "packets" it is entirely legal and possible
(although hugely unlikely) for an intermediate network node to chop it up.

There is no way around this because TCP is just a stream.

I'm aware of that. I noticed the OP's read loop late, and that's why I
posted the second reply.

-- Barry