Walking away from UDP

[Nak]

Hi there,

After recently coming across issues with implementing a UDP server for
interprocess communication; involving file transfer, I have decided to go
onto TCP.

Now I *have* created a TCP server in the past for VB.NET but had
troubles when it comes to disconnecting all clients and cleaning up
remaining references. The problems arrise when using the IAsyncResult for
receiving data, once started a method is needed to successfully stop it at
*any* time.

I have looked back on past threads that I was involved in at the time
but still struggle to find the solution. As I understand it, I have to have
a public shared ManualResetEvent object for the client, as well as
implementing a state object to pass to the BeginRead method. But that is
about as much as I know, once I have a client connected to my server it
struggles to close correctly, as references are still "flying" about.

This is a brief outline of my current implementation...

Server)

* Contains a collection of client objects

1) A "start" method is called which contains a parameter for the
local port to listen on
2) A background thread is started which loops continuously calling
the "connect" method of a new client object and passing it the TCP listeners
AcceptTcpClient result.
3) Once a client connects its event handlers are initialized and it
is stored into the collection object. An event is then raised to notify the
main application that a client has just connected.

Client)

1) A "connect" method is called which contains an already initialized
TcpClient object which is to be passed by the server.
2) The TcpClients GetStream.BeginRead method is called and an
IAsyncResult object is created, this is stored in "module" level.
3) The TcpClients BeginRead async callback is raised when data is
recieved, if the data is 0 in length then the client is considered to be
disconnecting, otherwise an event is raised notifying the server of the
recieved data from the client and the BeginRead method is started again.

I have created a state object which contains 1 public property of
boolean type called "abortRequested" but I am unsure how to implement this.
If I create a public method called "close" for the client what should it
contain? Sorry for this long post but I am very eagre to get *a* method
working as I seem to be going around in circles! Thanks loads in advance!!!

Nick.

 

[Peter Huang]

Hi Nick,

1) A "connect" method is called which contains an already initialized
TcpClient object which is to be passed by the server.

I am somewhat confused about the line above, why we need to pass the
TcpClient object from the server to the client?

In your scenario, it seems that we have the model as below.
Client Server
new TcpClient(A) -------------------------> TCPListener

|

|

----->TcpClient (B)


Then the A will communicate with B.
Here are two samples you may have a look.
Async TCP Socket Server
http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=d789d
245-4cf7-45af-9660-c375fc3f796c

Multi Threaded VB.NET TCP/IP Listener
http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=75e68
966-d567-47b3-a255-9a51f9cb0eb7

So far .NET framework did not support canceling an Asynchronous
Callback(i.e. the BeginRead and EndRead).
But we can set the read unit less, so that every time the BeginRead will
try to read the unit count of bytes from the networkstream.
e.g. (Just an example) we can set the value to 1, i.e. one byte per
BeginRead, then after EndRead call in the Asynchronous Callback(because
just one byte it will return quickly), then we can check a bool value to
see if we need to abort the read operation, if true, we can just close the
tcpclient and the Asynchronous Callback will exit, else we can keep read
the next byte.

If you still have any concern, please feel free to let me know.


Best regards,

Peter Huang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.

 

[Nak]

Hi Peter,

Excellent, I was hoping for your response as you seem to have allot of
knowledge on this subject.

So far .NET framework did not support canceling an Asynchronous
Callback(i.e. the BeginRead and EndRead).
But we can set the read unit less, so that every time the BeginRead will
try to read the unit count of bytes from the networkstream.

Wow! Now that suprises me, may I ask if you know what the underlying
mechanism is for BeginRead? Does it simply start a thread which constantly
checks the stream to seem how much data is in it, and when it reaches a
certain threshold the callback is fired?

Can this mechanism be simulated? Could I make my own BeginRead that can
be cancelled without any problems? Or would this require using the
"sockets" classes rather than TCPClient?

e.g. (Just an example) we can set the value to 1, i.e. one byte per
BeginRead, then after EndRead call in the Asynchronous Callback(because
just one byte it will return quickly), then we can check a bool value to
see if we need to abort the read operation, if true, we can just close the
tcpclient and the Asynchronous Callback will exit, else we can keep read
the next byte.

So the only way to cancel via this method would be to set the callback
to 1 byte and send a message to *yourself* to force the callback to be
invoked? Maybe even send a particular message to shut down the server, it
seems a pretty weird way of doing it but if it is the only way then I shall
have to. Surely other people have had this problem? How else would you
perform an asynchronous read on a socket without using the Asynchronous
callbacks?

Thanks for your help

Nick.

 

[Peter Huang]

Hi Nick,

BeginRead is kind of asynchronout callback. Usually we can consider it
similar as that we create a new thread and call the Read method in the
thread. After we call the BeginRead, the system will run a thread(From the
threadpool managed by system) to call the callback function in the thread
and EndRead was called to attempt to retrieve data back(just as we call the
Read, it will be blocked if no data arrived , so that the mainthread that
called the BeginRead will keep running. So the callback will be called
after we call the BeginRead, not after the data is arrived. Although we can
create a new thread and call the read method, and when we want to abort the
read operation, we can just abort the thread, but the method is not
recommended, because this may cause some resources not cleaned properly. A
better method is to let the thread exit itself, i.e. I have suggested
before read a few bytes everytime and evaluate a bool value to know if we
need exit the thread. In the last post I suggest one byte per time, that is
just a example, because if we read 1 MB per time, it will give us less
opportunity to cancel the read operation, because the 1 MB may take longer
time to complete reading than 1 byte. So the proper value should be decided
by your own according to the network connectivity status, bandwidth,
network adapt and so on. You may try to use the
myNetworkStream.DataAvailable to evaluate if the data is available, because
if myNetworkStream.DataAvailable is false, the EndRead will be blocked
until there is data arrived.

Here are the msdn documents for the three methods, you may try to compare
Read with BeginRead and EndRead,(Also there are example code in the links)
NetworkStream.Read Method
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/
frlrfsystemnetsocketsnetworkstreamclassreadtopic.asp
NetworkStream.EndRead
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/
frlrfsystemnetsocketsnetworkstreamclassreadtopic.asp
NetworkStream.BeginRead Method
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/
frlrfsystemnetsocketsnetworkstreamclassreadtopic.asp

Best regards,

Peter Huang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.

 

[Nak]

Hi Peter,

... Although we can
create a new thread and call the read method, and when we want to abort the
read operation, we can just abort the thread, but the method is not
recommended, because this may cause some resources not cleaned properly...

This is pretty much what is happening to me when I use BeginRead anyway,
the application just refuses to close correctly unless the connection has
been completely closed and disposed before hand. But if *no* data is being
sent and the connection hasn't been closed from the remote end then I have
issues.

better method is to let the thread exit itself, i.e. I have suggested
before read a few bytes everytime and evaluate a bool value to know if we
need exit the thread. In the last post I suggest one byte per time, that is
just a example, because if we read 1 MB per time, it will give us less
opportunity to cancel the read operation, because the 1 MB may take longer
time to complete reading than 1 byte. So the proper value should be decided
by your own according to the network connectivity status, bandwidth,
network adapt and so on.

Yeah this does sound like a good suggestion with 1 problem, if *no* data
is being sent anyway then the BeginRead call-back never fires, and the only
ways to make it fire is to send data or close the connection from the remote
end, which isn't always possible. But I see what you mean, it makes sense
to operate in this manor so that the connection can be closed much quicker.

Here are the msdn documents for the three methods, you may try to compare
Read with BeginRead and EndRead,(Also there are example code in the links)
NetworkStream.Read Method
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/
frlrfsystemnetsocketsnetworkstreamclassreadtopic.asp
NetworkStream.BeginRead Method
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/
frlrfsystemnetsocketsnetworkstreamclassreadtopic.asp

Thanks loads again, I shall read them intently!

Nick.

 

[Peter Huang]

Hi Nick,

I look forward to your futher update, if you still have any concern on this
issue, please feel free to post here.

Best regards,

Peter Huang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.

 

[Nak]

Hi Peter,

I finally appear to have found a method that works successfully,

Private Sub startReading()
cTrdDownloadThread = New Threading.Thread(AddressOf doStartReading)
cTrdDownloadThread.IsBackground = True
Call cTrdDownloadThread.Start()
End Sub

Firstly a background thread is started to constantly check the network
stream

Private Sub doStartReading()
Dim pBytRxBuffer(READ_BUFFER_SIZE) As Byte
While (Not cBlnCancel)
Try
Dim pNStmStream As NetworkStream = cTCClient.GetStream
Dim pIntRxSize As Integer = pNStmStream.Read(pBytRxBuffer, 0,
READ_BUFFER_SIZE + 1)
If (pIntRxSize = 0) Then
cBlnCancel = True
RaiseEvent clientDisconnect(Me)
Else
If (pIntRxSize < (READ_BUFFER_SIZE + 1)) Then
Dim pBytRxBufferCrop(pIntRxSize - 1) As Byte
Call Array.Copy(pBytRxBuffer, pBytRxBufferCrop,
pIntRxSize)
RaiseEvent receivedData(Me, pBytRxBuffer)
Else
RaiseEvent receivedData(Me, pBytRxBuffer)
End If
End If
Catch ex As Exception
Call cTCClient.Close()
cBlnCancel = True
RaiseEvent clientDisconnect(Me)
End Try
End While
End Sub

The background thread constantly calls Read, thus blocking the thread
until data is recieved or the connection is closed. As it is a background
thread there is absolutly no problem with cancelling it, unlike an Async
Callback. The Try - Catch block is there as a precaution though the only
time I could think that it would be needed is during the NetworkStream read
method, if this were to raise an exception. I am not sure if the Close
method is needed in this block as I haven't actually seen any exceptions
raised, but none the less, the clients can now be disconnected when I
desire.

Thanks loads for your help Peter, its been a real insight.

Nick.

 

[Nak]

Hi there again!

I shall retract that previous statement, I have found yet more problems!

Take this for example,

When the Read method is called the thread is blocked, apparently the
only way to get around this is by checking DataAvailable. Now the problem
with DataAvailable is that it does *not* tell you if the connection has been
closed or not! The only way to know this is to call Read and have 0 bytes
returned to you, now isn't this bizaar?

So according the the Microsoft documentation the following should be
performed,

1. Check CanRead to prevent an IOException.
2. Check DataAvailable to prevent calling Read unnecessarily and blocking
the thread.
3. Call Read to return as much data as possible from the stream.

Problem # 1

CanRead remains True even if the remote end has disconnected. This
means you must call Read to check for a disconnection.

Problem # 2

DataAvailable will remain False during the event of a disconnection as 0
bytes are available! And a disconnection is detected by 0 bytes being
returned from the Read method.

Now this appears to be slightly strange to myself, I have a solution
that is slightly better than using BeginRead as I can cancel the thread at
any time and allow my program to close gracefully. But unfortunately this
results in an unreliable method of communication.

My question is this, is this the TCPClient at fault? If so, would using
"Sockets" help me or would I be in exactly the same situation? I never had
this problem with WinSock in VB6, I always found that to be quite reliable,
should I revert and make a .NET wrapper for it?

Hmmm, and much more hmmming besides...

Nick.

 

[Nak]

Hi yet again...

Deja vous?

Anyway, I *think* I have it this time, to successfully close the socket
when required and even have HyperTerminal pick up the disconnection, without
crashing the application and without leaving references open all over the
place. So this is what I have,

--------------------------------------
Private Sub startReading()
cTrdDownloadThread = New Threading.Thread(AddressOf doStartReading)
cTrdDownloadThread.IsBackground = True
Call cTrdDownloadThread.Start()
End Sub

Right, the above method is used for starting an asyncronous reading
process. Simple as that.
--------------------------------------
--------------------------------------
Private Sub doStartReading()
Dim pBytRxBuffer(READ_BUFFER_SIZE) As Byte
Do
Dim pIntRxSize As Integer
Try
pIntRxSize = cTCClient.GetStream.Read(pBytRxBuffer, 0,
READ_BUFFER_SIZE + 1)
Catch ex As Exception
pIntRxSize = 0
End Try

If (pIntRxSize = 0) Then
cBlnCancel = True
RaiseEvent clientDisconnect(Me)
Else
If (pIntRxSize < (READ_BUFFER_SIZE + 1)) Then
Dim pBytRxBufferCrop(pIntRxSize - 1) As Byte
Call Array.Copy(pBytRxBuffer, pBytRxBufferCrop, pIntRxSize)
RaiseEvent receivedData(Me, pBytRxBuffer)
Call Array.Clear(pBytRxBuffer, 0, READ_BUFFER_SIZE)
Else
RaiseEvent receivedData(Me, pBytRxBuffer)
End If
End If
Loop Until (cBlnCancel)
Call cleanUp()
End Sub

The Read method is encased in a try block to detect the stream being
shutdown locally and prevent nasty exceptions occuring. The number of bytes
recieved is set to 0, this kind of simulates a remote disconnection. Next
the number of bytes recieved is evaluated, if 0 the cancel flag is raised
and the clientDisconnect event is fired. If the client actually recieved
data then this is copied into an array and the array is passed back via the
receivedData event.

At the bottom cleanUp is called to remove references to the TcpClient
and Thread object.
--------------------------------------
--------------------------------------
Private Sub cleanUp()
cTrdDownloadThread = Nothing
cTCClient = Nothing
End Sub

The Cleanup method.
--------------------------------------
--------------------------------------
Public Sub cancel()
Call cTCClient.GetStream.Close()
End Sub

This is how the connection is closed locally, it forces an exception to
be raised @ the stream Read line in doStartReading. As simple as that.
--------------------------------------

Now I shall go and test yet more and probably come back saying this is a
heap of crap too! Who knows....

Nick.

 

[Nak]

All still works very very well!!!! :-D

Nick.

 

[Peter Huang]

Hi Nick,

If you still have any concern on this issue, please feel free to post here
and I will follow up.

Best regards,

Peter Huang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.