GetStream().Read problem

[Mike]

Hi,

I have an iteration to retrieve a number of messages from a server. Within this iteration, I am using the following code:

do
{
readBytes = base.GetStream().Read(received, 0, received.Length);
string textToAdd = Encoding.ASCII.GetString(received, 0, readBytes);
myCompleteMessage = String.Concat(myCompleteMessage, textToAdd);
}
while(base.GetStream().DataAvailable == true);

The problem that I have seen is that not the entire content of the message is retrieved. For example, I first read the header and then the body, and when I read the second message, the header still contains some information that belonged to the previous message's body.

Any idea?

Thanks
Mike

 

[Jon Skeet [C# MVP]]

I have an iteration to retrieve a number of messages from a server.
Within this iteration, I am using the following code:

do
{
readBytes = base.GetStream().Read(received, 0, received.Length);
string textToAdd = Encoding.ASCII.GetString(received, 0, readBytes);
myCompleteMessage = String.Concat(myCompleteMessage, textToAdd);
}
while(base.GetStream().DataAvailable == true);

The problem that I have seen is that not the entire content of the
message is retrieved. For example, I first read the header and then
the body, and when I read the second message, the header still
contains some information that belonged to the previous message's
body.

The problem is your use of DataAvailable, which only says whether or
not there's any data available right now. You should use something else
in the protocol to say how long messages are, or specify the message
end, etc.

You should also use a StringBuilder rather than calling String.Concat
repeatedly, as currently you're creating loads of extra strings for no
reason.

 

[Ignacio Machin \( .NET/ C# MVP \)]

Hi Mike,


The thing is is that you are reading raw data, there is no concept of "message" as a logic data organization. either you improve your protocol to include a way to indicate the end of message ; for example SMTP use a line with a single dot "." to indicate the end of the email being sent;

or if you don;t have control over the protocol you can read the raw data and then later do the processing of it.


Also as Jon suggested you should not use String but StringBuilder.

Cheers,

--
Ignacio Machin,
ignacio.machin AT dot.state.fl.us
Florida Department Of Transportation


Hi,

I have an iteration to retrieve a number of messages from a server. Within this iteration, I am using the following code:

do
{
readBytes = base.GetStream().Read(received, 0, received.Length);
string textToAdd = Encoding.ASCII.GetString(received, 0, readBytes);
myCompleteMessage = String.Concat(myCompleteMessage, textToAdd);
}
while(base.GetStream().DataAvailable == true);

The problem that I have seen is that not the entire content of the message is retrieved. For example, I first read the header and then the body, and when I read the second message, the header still contains some information that belonged to the previous message's body.

Any idea?

Thanks
Mike

 

[Mike]

Hi Ignacio,

Thanks for your answer. I am actually coding against a NNTP server, but the current code I have is slow retrieving messages, so I am trying to find another solution. I have tried this other code:

1: while (true)
2: {
3: byte[] received = new byte[base.ReceiveBufferSize];
4: stream.Read(received, 0, received.Length);
5: b.Append(Encoding.ASCII.GetString(received));
6: }

but when the iteration goes through line 4 the second time and the server only sent a single line, the application hangs. I could not find a way to stop the iteration when there is no data or all data has been read.

What am I doing wrong here?

Thanks
Mike




"Ignacio Machin ( .NET/ C# MVP )" <ignacio.machin AT dot.state.fl.us> wrote in message Hi Mike,


The thing is is that you are reading raw data, there is no concept of "message" as a logic data organization. either you improve your protocol to include a way to indicate the end of message ; for example SMTP use a line with a single dot "." to indicate the end of the email being sent; or if you don;t have control over the protocol you can read the raw data and then later do the processing of it.


Also as Jon suggested you should not use String but StringBuilder.

Cheers,

--
Ignacio Machin,
ignacio.machin AT dot.state.fl.us
Florida Department Of Transportation


Hi,

I have an iteration to retrieve a number of messages from a server. Within this iteration, I am using the following code:

do
{
readBytes = base.GetStream().Read(received, 0, received.Length);
string textToAdd = Encoding.ASCII.GetString(received, 0, readBytes);
myCompleteMessage = String.Concat(myCompleteMessage, textToAdd);
}
while(base.GetStream().DataAvailable == true);

The problem that I have seen is that not the entire content of the message is retrieved. For example, I first read the header and then the body, and when I read the second message, the header still contains some information that belonged to the previous message's body.

Any idea?

Thanks
Mike

 

[Jon Skeet [C# MVP]]

Thanks for your answer. I am actually coding against a NNTP server,
but the current code I have is slow retrieving messages, so I am
trying to find another solution. I have tried this other code:

1: while (true)
2: {
3: byte[] received = new byte[base.ReceiveBufferSize];
4: stream.Read(received, 0, received.Length);
5: b.Append(Encoding.ASCII.GetString(received));
6: }

but when the iteration goes through line 4 the second time and the
server only sent a single line, the application hangs. I could not
find a way to stop the iteration when there is no data or all data
has been read.

What am I doing wrong here?

The client can never know when the server's going to send some more
data unless the connection has been closed. The above will actually
continue for ever, and normally you'd check whether Read returned 0
(for "connection closed"). That leads me to another problem you've got
in the code above - you're always *assuming* that Read will fill the
whole of your buffer. Your code *ought* to be more like:

byte[] buffer = new byte[base.ReceiveBufferSize];
while (true)
{
int read = stream.Read(buffer, 0, buffer.Length);
if (read==0)
{
break;
}
b.Append (Encoding.ASCII.GetString(received, 0, read));
}

That still won't solve your problem though, because it will hang while
the NNTP server waits for some more input from you (at least, it'll
hang until the server decides to time your connection out).

Depending on what command you're using, the response will be terminated
either at the end of a line or (IIRC) with a "." on a line on its own -
consult the relevant RFC for details. When you've found that
terminator, *that's* the point at which to take whatever your next
action is. Note that (possibly not for NNTP, but for other protocols)
you may end up reading into the next message or bit of response data,
in which case you would need to maintain the rest of the buffer after
the terminator, and use that when trying to find the *next* terminator.
(Indeed, you may have read several responses in one go.)

 

[Ignacio Machin \( .NET/ C# MVP \)]

Hi Mike,

I think that first of all you have to read the NNTP RFC
I did a quick google search I got this http://www.faqs.org/rfcs/rfc977.html

There you will find explained how to "talk" with an NNTP server.

I think that it will be easier for you if you just use a component for this, try : http://groups.google.com/groups?q=NNTP+group:microsoft.public.dotnet.*&ie=UTF-8&hl=en

you will get a lot of post regarding C# and NNTP

btw, you should not create the buffer inside the while, in this way on each cycle you are creating a new array and wasting the previous one. Kinda of similar of what happened with the String in your previous code.


Cheers,

--
Ignacio Machin,
ignacio.machin AT dot.state.fl.us
Florida Department Of Transportation

Hi Ignacio,

Thanks for your answer. I am actually coding against a NNTP server, but the current code I have is slow retrieving messages, so I am trying to find another solution. I have tried this other code:

1: while (true)
2: {
3: byte[] received = new byte[base.ReceiveBufferSize];
4: stream.Read(received, 0, received.Length);
5: b.Append(Encoding.ASCII.GetString(received));
6: }

but when the iteration goes through line 4 the second time and the server only sent a single line, the application hangs. I could not find a way to stop the iteration when there is no data or all data has been read.

What am I doing wrong here?

Thanks
Mike




"Ignacio Machin ( .NET/ C# MVP )" <ignacio.machin AT dot.state.fl.us> wrote in message Hi Mike,


The thing is is that you are reading raw data, there is no concept of "message" as a logic data organization. either you improve your protocol to include a way to indicate the end of message ; for example SMTP use a line with a single dot "." to indicate the end of the email being sent; or if you don;t have control over the protocol you can read the raw data and then later do the processing of it.


Also as Jon suggested you should not use String but StringBuilder.

Cheers,

--
Ignacio Machin,
ignacio.machin AT dot.state.fl.us
Florida Department Of Transportation


Hi,

I have an iteration to retrieve a number of messages from a server. Within this iteration, I am using the following code:

do
{
readBytes = base.GetStream().Read(received, 0, received.Length);
string textToAdd = Encoding.ASCII.GetString(received, 0, readBytes);
myCompleteMessage = String.Concat(myCompleteMessage, textToAdd);
}
while(base.GetStream().DataAvailable == true);

The problem that I have seen is that not the entire content of the message is retrieved. For example, I first read the header and then the body, and when I read the second message, the header still contains some information that belonged to the previous message's body.

Any idea?

Thanks
Mike

 

[Mike]

Thanks for your suggestions. I think I finally made it work

Mike

Thanks for your answer. I am actually coding against a NNTP server,
but the current code I have is slow retrieving messages, so I am
trying to find another solution. I have tried this other code:

1: while (true)
2: {
3: byte[] received = new byte[base.ReceiveBufferSize];
4: stream.Read(received, 0, received.Length);
5: b.Append(Encoding.ASCII.GetString(received));
6: }

but when the iteration goes through line 4 the second time and the
server only sent a single line, the application hangs. I could not
find a way to stop the iteration when there is no data or all data
has been read.

What am I doing wrong here?

The client can never know when the server's going to send some more
data unless the connection has been closed. The above will actually
continue for ever, and normally you'd check whether Read returned 0
(for "connection closed"). That leads me to another problem you've got
in the code above - you're always *assuming* that Read will fill the
whole of your buffer. Your code *ought* to be more like:

byte[] buffer = new byte[base.ReceiveBufferSize];
while (true)
{
int read = stream.Read(buffer, 0, buffer.Length);
if (read==0)
{
break;
}
b.Append (Encoding.ASCII.GetString(received, 0, read));
}

That still won't solve your problem though, because it will hang while
the NNTP server waits for some more input from you (at least, it'll
hang until the server decides to time your connection out).

Depending on what command you're using, the response will be terminated
either at the end of a line or (IIRC) with a "." on a line on its own -
consult the relevant RFC for details. When you've found that
terminator, *that's* the point at which to take whatever your next
action is. Note that (possibly not for NNTP, but for other protocols)
you may end up reading into the next message or bit of response data,
in which case you would need to maintain the rest of the buffer after
the terminator, and use that when trying to find the *next* terminator.
(Indeed, you may have read several responses in one go.)

 

[Mike]

Hi Ignacio,

Thanks for your suggestions. It now works okay

Mike


"Ignacio Machin ( .NET/ C# MVP )" <ignacio.machin AT dot.state.fl.us> wrote in message Hi Mike,

I think that first of all you have to read the NNTP RFC
I did a quick google search I got this http://www.faqs.org/rfcs/rfc977.html

There you will find explained how to "talk" with an NNTP server.

I think that it will be easier for you if you just use a component for this, try : http://groups.google.com/groups?q=NNTP+group:microsoft.public.dotnet.*&ie=UTF-8&hl=en

you will get a lot of post regarding C# and NNTP

btw, you should not create the buffer inside the while, in this way on each cycle you are creating a new array and wasting the previous one. Kinda of similar of what happened with the String in your previous code.


Cheers,

--
Ignacio Machin,
ignacio.machin AT dot.state.fl.us
Florida Department Of Transportation

Hi Ignacio,

Thanks for your answer. I am actually coding against a NNTP server, but the current code I have is slow retrieving messages, so I am trying to find another solution. I have tried this other code:

1: while (true)
2: {
3: byte[] received = new byte[base.ReceiveBufferSize];
4: stream.Read(received, 0, received.Length);
5: b.Append(Encoding.ASCII.GetString(received));
6: }

but when the iteration goes through line 4 the second time and the server only sent a single line, the application hangs. I could not find a way to stop the iteration when there is no data or all data has been read.

What am I doing wrong here?

Thanks
Mike




"Ignacio Machin ( .NET/ C# MVP )" <ignacio.machin AT dot.state.fl.us> wrote in message Hi Mike,


The thing is is that you are reading raw data, there is no concept of "message" as a logic data organization. either you improve your protocol to include a way to indicate the end of message ; for example SMTP use a line with a single dot "." to indicate the end of the email being sent; or if you don;t have control over the protocol you can read the raw data and then later do the processing of it.


Also as Jon suggested you should not use String but StringBuilder.

Cheers,

--
Ignacio Machin,
ignacio.machin AT dot.state.fl.us
Florida Department Of Transportation


Hi,

I have an iteration to retrieve a number of messages from a server. Within this iteration, I am using the following code:

do
{
readBytes = base.GetStream().Read(received, 0, received.Length);
string textToAdd = Encoding.ASCII.GetString(received, 0, readBytes);
myCompleteMessage = String.Concat(myCompleteMessage, textToAdd);
}
while(base.GetStream().DataAvailable == true);

The problem that I have seen is that not the entire content of the message is retrieved. For example, I first read the header and then the body, and when I read the second message, the header still contains some information that belonged to the previous message's body.

Any idea?

Thanks
Mike