TCPClient Losing Data

[DrZoop]

Long story short: I am misusing the TCPClient class. I am sending
400-500 byte messages, and data is being lost.

How can the TCPClient class be misused to lose data?

Background notes:
-I know the TCP protocol doesn't lose data. Don't tell me that. I am
educated.
-I am buffering all data being received before checking for messages
and reading them, so don't tell me this either. I have quite a spiffy
message system tried and proven. It's just these large chunks of data
screwing me over.
-I am using BeginRead/EndRead to receive my data.
-I am using a huge receive and send buffers, this simply can't be the
problem unless I'm misusing them.
-Changing the receive buffer does affect how long before it breaks.
However, how many bytes it takes before it breaks is inconsistent,
even with the same bytes being sent each time.
-Through tedious byte by byte comparison, it appears that "older"
bytes are being overwritten with newer, more recently received bytes.
By comparing the sent and received bytes, there is a gap of missing
bytes in the center. It's as if the buffer is too small, though, 3
megabytes for 500 bytes should be plenty of space... plus, the EndRead
function should be called anyway!
-I have read that the ReceiveTimeOut property should be set to
something else than zero to ensure no loss of data. Why is this? What
if it ran out of time and no bytes were received? Wouldn't this return
a length of zero bytes and appear to be a disconnection?
-Where it breaks is inconsistent. I can do the same login procedure
time and time again, and it will break at different places.

-Kinda unrelated: What is the State argument for in the BeginRead
function!? MSDN says you should "at the minimum pass the NetworkStream
class." One example passes the TCPClient class. Another example passes
seemingly random things (the form, a button on a toolstrip). The
example I based mine off of passed nothing/null.

Worst case scenario: can someone show me an example (I learn best by
examples honestly) that features a little program that can send and
receive using the TCPClient class many complete 400-500 byte messages?

Thanks; I'm sorry for the ugliness of this post. I lost my full,
detailed, nicely written post.

 

[Guest]

Long story short: I am misusing the TCPClient class. I am sending

400-500 byte messages, and data is being lost.

-I am using BeginRead/EndRead to receive my data.

Some time ago, I gave up on BeginWhatever/EndWhatever owing to, well, I
don't know exactly what, but I had troubles that would not go away. Pressed
for time, I implemented what I needed with synchronous methods and I launched
a thread to achieve asynchronous operations. In some cases, I launched one
thread per use, and in other cases, I used one persistent thread with a queue
for batching. I never looked back, and I am a happy camper.

My suggestion is to try a test that isolates the problem to Begin/End.
Would it be much work to comment out these calls and replace them with their
synchronous counterparts? Not too much work I hope. Then run a stress test
to see if you lose data. Hopefully, you will continue to lose data, and
further fault isolation should be a snap because of the simpler single thread
setting. But my guess is that your data loss problems will go away. Now you
will have good news (you have a working model and a partly isolated fault in
the original design) and bad news (you don't have the answer to your original
question). What happens next is up to you.

I suspect you may find my response unsatisfying - sorry about that. We all
get set in our ways using techniques we like and trust and avoiding others.
I neither like nor trust the begin/end .net approach to asynchronous
operations. Too complicated, too quirky, too much going on under the
covers, and that is why I responded. Good luck on what sounds like a
challenging problem.

 

[DrZoop]

Some time ago, I gave up on BeginWhatever/EndWhatever owing to, well, I
don't know exactly what, but I had troubles that would not go away. Pressed
for time, I implemented what I needed with synchronous methods and I launched
a thread to achieve asynchronous operations. In some cases, I launched one
thread per use, and in other cases, I used one persistent thread with a queue
for batching. I never looked back, and I am a happy camper.

My suggestion is to try a test that isolates the problem to Begin/End.
Would it be much work to comment out these calls and replace them with their
synchronous counterparts? Not too much work I hope. Then run a stress test
to see if you lose data. Hopefully, you will continue to lose data, and
further fault isolation should be a snap because of the simpler single thread
setting. But my guess is that your data loss problems will go away. Now you
will have good news (you have a working model and a partly isolated fault in
the original design) and bad news (you don't have the answer to your original
question). What happens next is up to you.

I suspect you may find my response unsatisfying - sorry about that. We all
get set in our ways using techniques we like and trust and avoiding others.
I neither like nor trust the begin/end .net approach to asynchronous
operations. Too complicated, too quirky, too much going on under the
covers, and that is why I responded. Good luck on what sounds like a
challenging problem.

Sadly, that is actually the MOST USEFUL post I have gotten in a while
regarding coding problems. I wanted out-of-the-box thinking, and you
did it. [RANT] Everyone else tells me the same old "newbie"
programming tips: use message lengths or delimiters, increase the
buffer size, be sure you're accumulating all data received in a
buffer... all things I stated in my post and said I was doing. Those
posts combined with my existing coding problem is incredibly
frustrating. [/RANT]

I will try regular Read instead of ReadBegin/ReadEnd and see if it
helps. I thought about this myself, however, due to all my research,
they said never bother--stick with Begin/End. So, I did what I was
reading... don't bother. I will try it now, however, this will make
my event model screwy and will need to start creating my own threads
again (I actually had it work similarly once, but I didn't like the
threads floating around). However, that's a cost I'm willing to take
to make it work correctly.

Many thanks!!!!!!!!!

------

I will copy and paste my post on another forum (I have to post in
three places minimum or else it will take ages (or forever) for me to
get help on my problem). It contains the code, super commented, along
with some further details.

It sends and receives byte arrays. It's pretty bleedingly
straightforward. I have a fancy message class (my own binary stream
that actually counts booleans as a single bit, among other things
which can make messages smaller) that uses message length UShorts
attached to the front of each message with checksums to make sure they
were not tampered with. This class works fine. I have tested it on
its own. It works fine with small messages through TCP (and UDP). It
is the large messages that are screwing me over. I am doing something
wrong in my usage of the .NET TCPClient class that is causing me to
lose bytes.

Private Sub BeginRead()
If Connected() Then
SyncLock myClient.GetStream 'original example used
synclocks
If myByteBuffer.Length <> myClient.ReceiveBufferSize
Then 'resizes the application byte buffer.  Do not worry, this buffer
is only used for one read.  It is set earlier to set the
ReceiveBufferSize to an incredible unnecessary size.
ReDim myByteBuffer(myClient.ReceiveBufferSize + 1)
End If

Try
myClient.GetStream.BeginRead(myByteBuffer, 0,
myClient.ReceiveBufferSize, New AsyncCallback(AddressOf ReceivedData),
myClient.GetStream) 'earlier I didn't have "New AsyncCallBack."  My
original example had only "AddressOf ReceivedData" (ReceivedData is
the function with EndRead.  Either way, the code seems to work.  Data
is received.
Catch
Console.WriteLine("BeginRead broke.")
End Try
End SyncLock
End If
End Sub

Private Sub ReceivedData(ByVal ar As IAsyncResult)
If HasConnected Then
Dim Length As Integer
Dim NS As NetworkStream = CType(ar.AsyncState,
NetworkStream) 'A newer example used this opposed to what is commented
out below.
Length = NS.EndRead(ar)
'Try
'    SyncLock myClient.GetStream
'        Length = myClient.GetStream.EndRead(ar)
'    End SyncLock
'Catch
'will be caught by length <= 0 below
'End Try

If Length <= 0 Then
Close("Connection interrupted.")
Exit Sub
Else
SyncLock myReceiveBuffer 'is the synclock necessary?
myReceiveBuffer is the true buffer that buffers everything.
'update byte counts
myBytesReceived += Length 'just useful to know,
for debugging; you may ignore
myBytesReceivedLast = Length

If ThreadSafe Then 'set flag if thread safe 'you
may ignore this as well
myBytesReceivedChanged = True
Else
RaiseEvent BytesReceived(myBytesReceived,
Length)
End If

Dim Shortened(Length - 1) As Byte 'this shortens
myByteBuffer to the length it really received
Array.Copy(myByteBuffer, Shortened, Length) 'same
as previous
myReceiveBuffer.PushBytes(Shortened) 'puts the
trimmed data into my special message class buffer.  Do not question
this class, because I had the same problem before using this version
of it.  This new version has been tested with smaller messages and
works just dandy fine.

'only pop messages and raise events here if not
threadsafe.
If Not ThreadSafe Then
PopMessages() 'pops complete messages from the
myReceiveBuffer class.  I told you I was buffering data.  Whenever I
ask this question, everyone assumes I have never done TCP before.
End If
'End If
End SyncLock

BeginRead() 'begin reading again after we have read
message
End If
End If
End Sub

I apologize again for the exasperation of my posts, and I thank you
for any brainstorming you have on this problem.

 

[Jon Skeet [C# MVP]]

I will copy and paste my post on another forum (I have to post in
three places minimum or else it will take ages (or forever) for me to
get help on my problem). It contains the code, super commented, along
with some further details.

<snip>

It's got lots of comments, but:
1) It contains code we should apparently ignore. It's worth getting
rid of such code.
2) It's not complete - we can't run it and see the problem.

The best way to get help - one which almost always works, in my
experience, is to post a short but *complete* program which
demonstrates the problem. This should:

1) Compile as written (preferably with no need to create a Visual
Studio project etc)
2) Only contain the necessary code (console apps tend to be simpler
than GUI apps)
3) Show what goes wrong

At that point it's much easier to reproduce the issue and help you.

See http://pobox.com/~skeet/csharp/complete.html for more information.

Jon