NetworkStream.Read - Thread freezes

[ohmmega]

hello out there,

I use a thread data from a linuxserver. i ask for data, i read the
data (fix in size) - so far so good. this works great for the first 61
datapackages, then my thread abruptly freezes, because of the read
method. i can't locate the real reason. if I reconnect the tcpClient
after each package it works fine, but after several minutes the linux
programm quits (not my fault, not my code, not to change). has anybody
any clues about reason why this happens?

i just want to make a robust code. i thought about another thread
which works as watchdog - dataloss is no problem in this case. is
there a approved design pattern (snipplets) for this task?

thanks
rené

<code>
public byte[] getPackage(int PackageSize)
{
int SumBytes = 0;

byte[] PackageData= new byte[PackageSize];

NetworkStream myStream = getStream(...); //nonrelevant

while ( true )
{
if (myStream.DataAvailable)
{
SumBytes += myStream.Read(PackageData, SumBytes, PackageSize-
SumBytes);

if (SumBytes >= PackageSize)
{
break;
}
}
}

return PackageData;
}
</code>

 

[Jon Skeet [C# MVP]]

hello out there,

I use a thread data from a linuxserver. i ask for data, i read the
data (fix in size) - so far so good. this works great for the first 61
datapackages, then my thread abruptly freezes, because of the read
method. i can't locate the real reason. if I reconnect the tcpClient
after each package it works fine, but after several minutes the linux
programm quits (not my fault, not my code, not to change). has anybody
any clues about reason why this happens?

Have you used a packet sniffer (eg wireshark) to determine whether any
more data has actually been sent?

Jon

 

[ohmmega]

Have you used a packet sniffer (eg wireshark) to determine whether any
more data has actually been sent?

Jon

not yet, because i've a server log, which tells me that the server
acknowledges my request (port A) and so i start listening on port B.
the problem seems to be, that the server don't transmit sometimes.

furthermore i thought that the use of networkStream.dataAvailable
should prevent from such deadlocks (the thread don't loop forever -
i've implemented a timeout, which isn't listet in the snipplet above.
it's definitly hangs up the read method).

 

[UL-Tomten]

furthermore i thought that the use of networkStream.dataAvailable
should prevent from such deadlocks [...]

Technically, it's a blocking call, not a deadlock, but you're right;
NetworkStream.Read() should return immediately if DataAvailable (which
is why it doesn't necessarily read as many bytes as you wanted). Are
you sure the parameters to Read() are all valid, and that nobody else
is using the stream?

 

[ohmmega]

furthermore i thought that the use of networkStream.dataAvailable
should prevent from such deadlocks [...]

Technically, it's a blocking call, not a deadlock, but you're right;
NetworkStream.Read() should return immediately if DataAvailable (which
is why it doesn't necessarily read as many bytes as you wanted). Are
you sure the parameters to Read() are all valid, and that nobody else
is using the stream?

concerning the parameters: i've just tried out the code from the msdn.
they are using a StringBuilder instead of an byte array - ok.
since i use a byte array, i can see no error in my code (anybody
else?).

refering to the streamaccess: i'm definitly the only one, who's using
this stream on that port - but: it's running over the companynetwork
with serveral switches in beetween. i think the problem occoured only
after accessing over network - direct connection worked that far. but
i will try it again to make this sure.

can you tell me why my code blocks, even after checking for available
data?

 

[UL-Tomten]

concerning the parameters: i've just tried out the code from the msdn.
they are using a StringBuilder instead of an byte array - ok.

Well, myReadBuffer in the MSDN example is a byte[1024]; the
StringBuffer is not relevant to the NetworkStream.Read() call.

Things to try:

1. Try the code reading from a local port on your own machine.
2. To make sure no other part of your code is somehow tampering with
your Socket/NetworkStream, make a new test (perhaps using the Snippet
Compiler) using only what's needed to actually test the Read() against
the server.
3. Try outside the debugger.
4. Run the code on a different machine.

 

[ohmmega]

concerning the parameters: i've just tried out the code from the msdn.
they are using a StringBuilder instead of an byte array - ok.

Well, myReadBuffer in the MSDN example is a byte[1024]; the
StringBuffer is not relevant to the NetworkStream.Read() call.

Things to try:

1. Try the code reading from a local port on your own machine.
2. To make sure no other part of your code is somehow tampering with
your Socket/NetworkStream, make a new test (perhaps using the Snippet
Compiler) using only what's needed to actually test the Read() against
the server.
3. Try outside the debugger.
4. Run the code on a different machine.

i've just tested the direct communication and to my frustration the
system hangs up.
concerning your comments:
1. you mean writing my own server in c#, arn't you?
2. this overrates myself: what do you mean with snippet compiler? i've
only access to visual studio 2003. if there is such a compiler or
compile option let me know.
3. release? also game over.
4. same problem.

i also tried wireshark, but it swampped me a little, so i couldn't get
any usefull clues.

any other usefull hints?
thanks so long.

 

[UL-Tomten]

i've just tested the direct communication [...]

What's "direct communication"?

1. you mean writing my own server in c#, arn't you?

Yeah, anything that behaves like a simple server, so you can test the
Read(). What the server does is not important. Or just Read() from an
existing server, since you don't care about the results: connect to
the IIS, Send() a HTTP request (or something that will make IIS
respond with an error message) and Read() some bytes. If that blocks
too, then the problem is not the server.

2. this overrates myself: what do you mean with snippet compiler?

Snippet Compiler compiles snippets. It's a mini-environment for
testing code. http://www.sliver.com/dotnet/SnippetCompiler/

 

[ohmmega]

i've just tested the direct communication [...]

What's "direct communication"?

1. you mean writing my own server in c#, arn't you?

Yeah, anything that behaves like a simple server, so you can test the
Read(). What the server does is not important. Or just Read() from an
existing server, since you don't care about the results: connect to
the IIS, Send() a HTTP request (or something that will make IIS
respond with an error message) and Read() some bytes. If that blocks
too, then the problem is not the server.

2. this overrates myself: what do you mean with snippet compiler?

Snippet Compiler compiles snippets. It's a mini-environment for
testing code.http://www.sliver.com/dotnet/SnippetCompiler/

with direct connection i mean: pc - cable - device (no switches, no
routers)

maybe i write a server tomorrow, today i'm just ***** ò¿ó

snippet: ah ja.

my boss says that the server sometimes makes problems if there is no
one on there to catch requested packages. that's strange, because i'm
always waiting for the whole package. damn.

 

[Peter Duniho]

[...]
can you tell me why my code blocks, even after checking for available
data?

I don't know the specifics of how the DataAvailable property in .NET is
implemented. However, assuming it uses the most obvious mechanism --
receiving with the MSG_PEEK flag -- it is not guaranteed to avoid a
blocking receive.

All that calling recv(..., MSG_PEEK) does is give you an instantaneous
observation of the state of the socket. That state can in fact change
the moment you return from checking the state, and if for whatever
reason the buffered data has to be discarded before you actually try to
receive the data, the attempt to receive will block.

Now, whether that is affecting what's going on here or not is hard to
say. Generally, as long as the algorithm does not actually rely on the
receive not blocking, the worst that will happen is that you'll just
block until the data can be resent, and then it will go ahead and be
received.

But regardless, it's very important to understand the limitations of
MSG_PEEK, and the likelihood that these limitations apply also to the
DataAvailable property.

Pete

 

[Jon Skeet [C# MVP]]

But regardless, it's very important to understand the limitations of
MSG_PEEK, and the likelihood that these limitations apply also to the
DataAvailable property.

It would be nice if it were all documented appropriately. In
particular, documentation for DataAvailable states:

<quote>
If DataAvailable is true, a call to Read returns immediately.
</quote>

That certainly *appears* not to be happening based on what's been
stated, although I'd like to see it happening on my own box before
coming to any real conclusions

 

[UL-Tomten]

I don't know the specifics of how the DataAvailable property in .NET is
implemented. However, assuming it uses the most obvious mechanism --
receiving with the MSG_PEEK flag -- it is not guaranteed to avoid a
blocking receive.

Snooping at the Socket class, it looks like it doesn't MSG_PEEK, but
its evil twin ioctlsocket(FIONREAD). There is even a KB about the
pitfalls when doing this: http://support.microsoft.com/kb/192599

That state can in fact change the moment you return from checking the
state, and if for whatever reason the buffered data has to be discarded
before you actually try to receive the data, the attempt to receive will
block.

I still think it's strange that this behavior is so consistent in
ohmmega's application. Perhaps there is indeed something fishy about
the network traffic.

When reading the .NET Socket documentation, I constantly get the
feeling that the documentation is trying to hide the gory details of
network streaming for us. But since the .NET Socket API is such a thin
wrapper around Winsock2, that seems counter-productive, because we
still have to dig up the Winsock2 documentation every time something
doesn't work as expected. NetworkStream doesn't even behave like other
streams in the framework. Maybe it's just extremely hard to construct
a truly managed (in the framework sense of the word) wrapper around
BSD sockets?

 

[Jon Skeet [C# MVP]]

NetworkStream doesn't even behave like other streams in the framework.

Could you elaborate on this? It's always nice to know "gotchas" from
other people...

 

[UL-Tomten]

Could you elaborate on this? It's always nice to know "gotchas" from
other people...

Well, I wasn't referring to anything beyond what's in the
documentation. The gotchas I have are the same as any socket
programmer, which are the ones the connection protocols impose on you,
and that the managed API can't protect you from. The differences
between reading from disk and reading from Jane's TCP server are
perhaps subtle, but it's clear from people's experiences that the idea
of blocking/non-blocking together with synchronous/asynchronous
haven't intuitively been addressed. When you're reading from a stream,
you have to treat the NetworkStream as a special case, because it
responds differently to interrupts (WSAEINTR), it doesn't block
sometimes, it can tell you (Or Can It?) when data is available... I
feel like the Socket API (or rather, the managed network
communications layer) hasn't made the same maturing from 1.1 to 2.0
the rest of the framework has (and 3.5 only contains classes that
allow performance enhancements, introducing another APM on top of
Begin/End). Don't you agree?

 

[Jon Skeet [C# MVP]]

Well, I wasn't referring to anything beyond what's in the
documentation. The gotchas I have are the same as any socket
programmer, which are the ones the connection protocols impose on you,
and that the managed API can't protect you from. The differences
between reading from disk and reading from Jane's TCP server are
perhaps subtle, but it's clear from people's experiences that the idea
of blocking/non-blocking together with synchronous/asynchronous
haven't intuitively been addressed. When you're reading from a stream,
you have to treat the NetworkStream as a special case, because it
responds differently to interrupts (WSAEINTR), it doesn't block
sometimes, it can tell you (Or Can It?) when data is available... I
feel like the Socket API (or rather, the managed network
communications layer) hasn't made the same maturing from 1.1 to 2.0
the rest of the framework has (and 3.5 only contains classes that
allow performance enhancements, introducing another APM on top of
Begin/End). Don't you agree?

It certainly has the extra DataAvailable member, but other streams may
or may not buffer data so that calls sometimes will block and sometimes
won't.

The main problem I see is with people assuming that a call to
Stream.Read will return as much data as they asked for - and relying on
that is a bad idea whatever kind of stream you're using.