TCP data loss

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

I have encountered with a most disturbing TCP problem:

I have cases (too many of them) that result in data loss.
Some inforamation on my test configuration:
I have to PC's which are the same (Dell Dimension 2400).
PC1 is the tcp server, PC2 is the tcp client, they two configured to 65535
bytes tco buffer/window, and working in synchronous manner while for each
client the server start a dedicated thread.
The client send the data size in the first block and this way the server
knows when to stop processing the data from this client by accumulating the
retuned valued from the Socket.Receive(..) till it gets to the size received
in the first block.

When the server is handling a single client, no data is lost. But when I
switch the PC's so PC2 is the server and PC1 is the client, data is lost
causing the server to keep on waiting on the Socket.Receive(..) while the
client is done.

When the server (on PC1) is handling several clients (multiple threads
generated in PC2) again data is lost and some of the clients data is missing
in the server side again causing the server to keep on waiting on the
Socket.Receive(..) while all clients are done sending all their data.

I'm really don't know how fix that or what am I doing wrong.

Any advice will be more then welcome.
 
Sounds like you have some miscalculations or mistakes of that sort in
your application. I think it would help if you posted some more
important parts of your code or equivalent logic on how you are handling
multiple clients and things of that sort.
 
Ok, here is the code. I hope I didn't make any mistakes coping and cleaning
the code when posting it:

//////////////////////////////////////////////////////////////////////////
/// The TCP Client
//////////////////////////////////////////////////////////////////////////
int blockSize = 65535;
byte [] rowData = new byte[1048576];// 1 MByte
int blockCount = rowData.Length/blockSize + ((rowData.Length % blockSize ==
0) ? 0 : 1);
IPEndPoint ep = new IPEndPoint(IPAddress.Parse("Server IP"), 31001);
clientSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream,
ProtocolType.Tcp);
clientSocket.SetSocketOption(SocketOptionLevel.Socket,
SocketOptionName.SendBuffer, blockSize);
clientSocket.Connect(ep);
byte [] info;
byte [] initBuf = new byte[8];
// Setting the block size into first 4 bytes.
info = BitConverter.GetBytes(blockSize);
int i = 0;
for( ; i<info.Length; ++i )
{
initBuf = info;
}
// Setting the overall data size into bytes 4 to 7.
info = BitConverter.GetBytes(rowData.Length);
for( i=0; i<info.Length; ++i )
{
initBuf[4+i] = info;
}
clientSocket.Send( initBuf );

// Sending the data in a 65535 block size.
int length, sentSize;
for( i=0; i < blockCount ; ++i )
{
if( i < (blockCount - 1) )
{
length = blockSize;
}
else
{
length = rowData.Length - (i * blockSize);
}
sentSize = clientSocket.Send(rowData, i*blockSize, length,
System.Net.Sockets.SocketFlags.None);
}
clientSocket.Close();

//////////////////////////////////////////////////////////////////////////
/// The TCP Server
//////////////////////////////////////////////////////////////////////////
IPEndPoint ipNport = new IPEndPoint(IPAddress.Parse("Server IP"), 31001)
TcpListener m_server = new TcpListener(ipNport);
ArrayList m_socketListenersList = new ArrayList();
// Start the Server and start the thread to listen to client requests.
m_server.Start();
m_serverThread = new Thread( new ThreadStart(ServerThreadStart) );
m_serverThread.Start();

private void ServerThreadStart()
{
// Client Socket variable;
Socket clientSocket = null;
TCPSocketListener socketListener = null;
while( true )
{
// Wait for any client requests and if there is any
// request from any client accept it (Wait indefinitely).
clientSocket = m_server.AcceptSocket();
socketListener = new TCPSocketListener(clientSocket);
// Add the socket listener to an array list in a thread safe fashion.
lock( m_socketListenersList )
{
m_socketListenersList.Add(clientSocket);
}
// Start a communicating with the client in a different thread.
socketListener.StartSocketListener();
}
}

public class TCPSocketListener
{
private int m_ID;
private int m_DataSize = 0;
private int m_TotalByteRcvd = 0;
private bool m_IsfirstBlock = true;
private Socket m_clientSocket = null;

public TCPSocketListener(Socket clientSocket)
{
m_clientSocket = clientSocket;
}
public void StartSocketListener()
{
m_clientListenerThread = new Thread(new
ThreadStart(SocketListenerThreadStart));
m_clientListenerThread.Start();
}
private void SocketListenerThreadStart()
{
int size = 0;
byte [] byteBuffer = new byte[131072];// 128 Kilobyte.

while( true )
{
size = m_clientSocket.Receive(byteBuffer);
if( size > 0 )
{
if( m_IsfirstBlock )
{
m_TotalByteRcvd = 0;
blocksSize = BitConverter.ToInt32(rowdata, 4);
m_DataSize = BitConverter.ToInt32(rowdata, 8);
m_Buffer = new byte[dataSize];
m_clientSocket.SetSocketOption(SocketOptionLevel.Socket,
SocketOptionName.ReceiveBuffer, blocksSize);
}
else
{
m_TotalByteRcvd += size;
if( m_TotalByteRcvd >= m_DataSize )
{
break;
}
}
}
}
m_clientSocket.Close();
}
}
/////////////////////////////////////////////////////////////

I hope it's clear enough.

Any Idea will highly appreciated.
 
I assume you have tried debugging both client and server in problem
situations. Did you notice anything different? I find it very strange
that when you replace the computer roles (client/server), the program
stops working.

For me just by looking at the code is really hard to say what could be
wrong. I took a look and nothing seemed out of order there anyways.
 
First off, it would be helpful if you could post a complete sample (one that
anyone can compile and run without modifications). But anyway, comments
inline:

Sharon said:
Ok, here is the code. I hope I didn't make any mistakes coping and
cleaning
the code when posting it:

//////////////////////////////////////////////////////////////////////////
/// The TCP Client
//////////////////////////////////////////////////////////////////////////
int blockSize = 65535;
byte [] rowData = new byte[1048576];// 1 MByte
int blockCount = rowData.Length/blockSize + ((rowData.Length % blockSize
==
0) ? 0 : 1);
IPEndPoint ep = new IPEndPoint(IPAddress.Parse("Server IP"), 31001);
clientSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream,
ProtocolType.Tcp);
clientSocket.SetSocketOption(SocketOptionLevel.Socket,
SocketOptionName.SendBuffer, blockSize);
clientSocket.Connect(ep);
byte [] info;
byte [] initBuf = new byte[8];
// Setting the block size into first 4 bytes.
info = BitConverter.GetBytes(blockSize);
int i = 0;
for( ; i<info.Length; ++i )
{
initBuf = info;
}
// Setting the overall data size into bytes 4 to 7.
info = BitConverter.GetBytes(rowData.Length);
for( i=0; i<info.Length; ++i )
{
initBuf[4+i] = info;
}
clientSocket.Send( initBuf );

// Sending the data in a 65535 block size.
int length, sentSize;
for( i=0; i < blockCount ; ++i )
{
if( i < (blockCount - 1) )
{
length = blockSize;
}
else
{
length = rowData.Length - (i * blockSize);
}
sentSize = clientSocket.Send(rowData, i*blockSize, length,
System.Net.Sockets.SocketFlags.None);
}
clientSocket.Close();

//////////////////////////////////////////////////////////////////////////
/// The TCP Server
//////////////////////////////////////////////////////////////////////////
IPEndPoint ipNport = new IPEndPoint(IPAddress.Parse("Server IP"), 31001)
TcpListener m_server = new TcpListener(ipNport);
ArrayList m_socketListenersList = new ArrayList();
// Start the Server and start the thread to listen to client requests.
m_server.Start();
m_serverThread = new Thread( new ThreadStart(ServerThreadStart) );
m_serverThread.Start();

private void ServerThreadStart()
{
// Client Socket variable;
Socket clientSocket = null;
TCPSocketListener socketListener = null;
while( true )
{
// Wait for any client requests and if there is any
// request from any client accept it (Wait indefinitely).
clientSocket = m_server.AcceptSocket();
socketListener = new TCPSocketListener(clientSocket);
// Add the socket listener to an array list in a thread safe fashion.
lock( m_socketListenersList )
{
m_socketListenersList.Add(clientSocket);
}
// Start a communicating with the client in a different thread.
socketListener.StartSocketListener();
}
}

public class TCPSocketListener
{
private int m_ID;
private int m_DataSize = 0;
private int m_TotalByteRcvd = 0;
private bool m_IsfirstBlock = true;
private Socket m_clientSocket = null;

public TCPSocketListener(Socket clientSocket)
{
m_clientSocket = clientSocket;
}
public void StartSocketListener()
{
m_clientListenerThread = new Thread(new
ThreadStart(SocketListenerThreadStart));
m_clientListenerThread.Start();
}
private void SocketListenerThreadStart()
{
int size = 0;
byte [] byteBuffer = new byte[131072];// 128 Kilobyte.

while( true )
{
size = m_clientSocket.Receive(byteBuffer);
if( size > 0 )
{
Click to expand...


At this point, it appears you assume you have received exactly 8 bytes but
that is not necessarily the case. 'size' could be anything from 1 to number
of bytes in the TCP receive buffer. Most notably, if the number of bytes is
greater than 8, then you're missing out the extra bytes here as the next
iteration won't pick them up.
if( m_IsfirstBlock )
{
m_TotalByteRcvd = 0;
blocksSize = BitConverter.ToInt32(rowdata, 4);
m_DataSize = BitConverter.ToInt32(rowdata, 8);
m_Buffer = new byte[dataSize];
m_clientSocket.SetSocketOption(SocketOptionLevel.Socket,
SocketOptionName.ReceiveBuffer, blocksSize);
}
else
{
m_TotalByteRcvd += size;
if( m_TotalByteRcvd >= m_DataSize )
{
break;
}
}
}
}
m_clientSocket.Close();
}
}
/////////////////////////////////////////////////////////////
Click to expand...

So the lesson is that never assume anything about the amount of data you
receive with TCP. If the sender calls Send two times trying to send e.g.,
16-bytes each time, the receiver might receive all of it in one 16-byte
chunk, or in two 8-byte chunks, or... It is the receiver's responsibility to
handle all possible cases correctly.

HTH,
Sami
 
Hi Sami,

Thanks a lot for your answer, your observation is the problem.

The client is sending 8 byte buffer in the first block, therefore I assumed
at the server side that this is the first buffer size I should receive, BUT
there are cases when this first block is 73008 bytes and not 8 bytes, causing
me to lose 73000 bytes.

So now, after I fixed it, it works fine even if I run 20 fast clients and a
server PC that is also running Outlook and Internet Radio.

So now it all good.

Again, thanks a lot.

Sharon
 
Back
Top