How can I overcome the buffer limitation using StreamReader.ReadTo

[Guest]

Hi, I'm using VS 2005, ,.net 2 for C# windows application. I'm using Process
to run a C application and redirecting its standard output so I can read it
with StreamReader.ReadToEnd. It's only reading the 1st 5 lines or so when
there is about 30 lines of data.
I also tried using the Process.BeginOutputReadlin and a
DataReceivedEvntHandler to gather output but I also only get about 5 lines of
the data.
There seems to be a buffer size issue. Can someone tell me how can I
resolve this problem. Thank you.

 

[Jon Skeet [C# MVP]]

Hi, I'm using VS 2005, ,.net 2 for C# windows application. I'm using Process
to run a C application and redirecting its standard output so I can read it
with StreamReader.ReadToEnd. It's only reading the 1st 5 lines or so when
there is about 30 lines of data.
I also tried using the Process.BeginOutputReadlin and a
DataReceivedEvntHandler to gather output but I also only get about 5 lines of
the data.
There seems to be a buffer size issue. Can someone tell me how can I
resolve this problem. Thank you.

It should work fine. Do you know if your C application is actually
finishing? You should read from standard error as well as standard out
- otherwise if it's trying to write to standard error, it may be
blocking.

 

[Guest]

Here is my code on first try to read the standard output. In the debug mode
I can see some lines of the data are read in but the process never has a
exited status. I even took away the time out value and it just hangs and
sits there and I can see the stream buffer hits a limit and hangs.
listFiles.WaitForExit();
Is there a way to read all my data in without a buffer limitation? Thank you.

public static int GetNisFile(System.Diagnostics.ProcessStartInfo psi,
ref DataTable dtAccounts,
ref ToolStripStatusLabel slMain, ref StatusStrip ssMain, string
userOrGroup)
{

string accountFile="", output="";
int numOfAccounts = 0;
System.IO.StreamReader fileOutput = null, myOutput = null;
System.Diagnostics.Process listFiles;
try
{
listFiles = System.Diagnostics.Process.Start(psi);
if (listFiles != null)
{
myOutput = listFiles.StandardError;
fileOutput = listFiles.StandardOutput;
listFiles.WaitForExit(30000);
if (listFiles.HasExited)
{
// Read and display lines from the file until the end of
// the file is reached.
accountFile = fileOutput.ReadToEnd();
output = myOutput.ReadToEnd();
if (accountFile != null && accountFile != "")
numOfAccounts =
CPAUnix.ParseAndInsertAccounts(accountFile,
ref dtAccounts, ref slMain, ref ssMain, userOrGroup);
}
else
MessageBox.Show("Time out retrieving NIS" + userOrGroup +
"accounts.",
"PowerADvantage");
}
else
MessageBox.Show("Error starting process to read file.",
"PowerADvantage");

return numOfAccounts;

}
catch (Win32Exception wex)
{
MessageBox.Show(wex.Message, "PowerADvantage");
return numOfAccounts;
}
}

 

[Guest]

Here is my code using BeginOutputReadLine. This code doesn't have the time
out issue but it only returns 5 lines of data.

static System.Text.StringBuilder sb = new System.Text.StringBuilder();
static void proc_OutputDataReceived(object sender, DataReceivedEventArgs e)
{
sb.Append(e.Data);
}

public static int GetNisFile(System.Diagnostics.ProcessStartInfo psi,
ref DataTable dtAccounts,
ref ToolStripStatusLabel slMain, ref StatusStrip ssMain, string
userOrGroup)
{

string accountFile = "", output = "", line = "", theResult="";
int numOfAccounts = 0;
System.IO.StreamReader fileOutput = null, myOutput = null;
System.Diagnostics.Process listFiles = new Process();
listFiles.StartInfo = psi;
try
{
String inputText;
int numInputLines = 0;
listFiles.OutputDataReceived += new
DataReceivedEventHandler(proc_OutputDataReceived);
//listFiles = System.Diagnostics.Process.Start(psi);
// Use a stream writer to synchronously write the sort input.
listFiles.Start();
//StreamReader nisReader = listFiles.StandardOutput;
listFiles.BeginOutputReadLine();
listFiles.WaitForExit();
theResult = sb.ToString();
listFiles.CancelOutputRead();
listFiles.Kill();
return numOfAccounts;

}
catch (Win32Exception wex)
{
MessageBox.Show(wex.Message, "PowerADvantage");
return numOfAccounts;
}
}

 

[Jon Skeet [C# MVP]]

Here is my code on first try to read the standard output. In the debug mode
I can see some lines of the data are read in but the process never has a
exited status. I even took away the time out value and it just hangs and
sits there and I can see the stream buffer hits a limit and hangs.
listFiles.WaitForExit();
Is there a way to read all my data in without a buffer limitation? Thank you.

You need to read the data in a different thread, basically. I don't
have the time to write the code right now, but if you have one thread
reading the stdout and one reading stderr, you shouldn't see this
problem.

 

[Guest]

Thanks Jon, I did read the article on the parent, child process deadlock
issue. I modified my code to use sync. read for the stderr and async. read
for the stadout. But, I'm still getting just a few lines of my stdout. I
ran the C application again and it gets over 100 lines of data. Thank you.
static System.Text.StringBuilder sb = new System.Text.StringBuilder();
static void proc_OutputDataReceived(object sender, DataReceivedEventArgs e)
{
sb.Append(e.Data);
}

public static int GetNisFile(System.Diagnostics.ProcessStartInfo psi, string
parm,
ref DataTable dtAccounts,
ref ToolStripStatusLabel slMain, ref StatusStrip ssMain, string
userOrGroup)
{

string accountFile = "", output = "", line = "", theResult="";
int numOfAccounts = 0;
String inputText;
int numInputLines = 0;
System.IO.StreamReader fileOutput = null, myOutput = null;
System.Diagnostics.Process listFiles = new Process();
//listFiles.StartInfo = psi;
try
{
listFiles.OutputDataReceived += new
DataReceivedEventHandler(proc_OutputDataReceived);
listFiles.StartInfo = new ProcessStartInfo(Application.StartupPath +
"\\ypcat.exe", parm);
listFiles.StartInfo.UseShellExecute = false;
listFiles.StartInfo.RedirectStandardOutput = true;
listFiles.StartInfo.RedirectStandardError = true;
//listFiles = System.Diagnostics.Process.Start(psi);
// Use a stream writer to synchronously write the sort input.
if (listFiles.Start())
{
//StreamReader nisReader = listFiles.StandardOutput;
listFiles.BeginOutputReadLine();
string error = listFiles.StandardError.ReadToEnd();
listFiles.WaitForExit();
if(listFiles.HasExited)
theResult = sb.ToString();
listFiles.CancelOutputRead();
listFiles.Close();
}
return numOfAccounts;



}
catch (Win32Exception wex)
{
MessageBox.Show(wex.Message, "PowerADvantage");
return numOfAccounts;
}
}

 

[Jon Skeet [C# MVP]]

Thanks Jon, I did read the article on the parent, child process deadlock
issue. I modified my code to use sync. read for the stderr and async. read
for the stadout. But, I'm still getting just a few lines of my stdout.

Without full code to run ourselves, it's pretty tricky to diagnose
this.

I suggest you first swap out the original process for a "dummy"
process written in C# which just writes 100 lines of output to both
stdout and stderror - then come up with a short but complete program
in C# which shows the problem occurring. That way we can all look at
the same issue.

Jon

 

[Guest]

Ok, I will do that. Sounds like a good test. Also, I forgot to mention that
I'm running on Windows 2000 server. Would this be the problem?

 

[Jon Skeet [C# MVP]]

Ok, I will do that. Sounds like a good test. Also, I forgot to mention that
I'm running on Windows 2000 server. Would this be the problem?

Shouldn't be, no.

 

[Guest]

I created 2 applications as you said and it works! So, I use the new C#
application and instead of calling the dummy program I called my C
application and display the output to a textbox and that works too! Now, I'm
really confused about why it doesn't work in my program.

 

[Guest]

Jon, thank you for all your help. It's now working. My eventhandler was
stripping the \n (in String.Append) and through my code it was retuning 0
records when it didn't find \n. But the process redirect is wokring
properly.

 

[Jon Skeet [C# MVP]]

Jon, thank you for all your help. It's now working. My eventhandler was
stripping the \n (in String.Append) and through my code it was retuning 0
records when it didn't find \n. But the process redirect is wokring
properly.

Excellent - glad to hear it