System.Diagnostics.Process: Asynchronous reading console output causes problems

[Thorsten Tarrach]

Hi,

I'm trying to run a console application and interpret its output. But I also
want to force the programme to exit after 1 second if it hasn't done so.
My first try was to use WaitForExit and read the console. Unfortunately the
console buffer is limited and the programme waits for the buffer be be
consumed before producing more.

So I turned to asynchronous reading using BeginOutputReadLine. But this
seams to result sometimes in not all lines being read. This behaviour is not
deterministic at all. On the same input the programme will fail about half
the time.

The solution would be to either increase the console buffer to make
synchronous reading possible after the process exited or fix my code

I have one further question: When I call Start on the process which is a
command line application it causes to pop up a black command line window
that takes focus. Setting WindowStyle to ProcessWindowStyle.Hidden does not
seem to help.

Here is my code. It is F# I'm afraid but it the problem in somewhere in my
usage of the .net Framework's Process class:

let result = ref ""

let dataRecv (senderbj) (e : DataReceivedEventArgs) =
lock result (fun () -> result := !result ^ e.Data ^ "\r\n")

let dataRecvHandler = new DataReceivedEventHandler(dataRecv)

let timeOut = 1000

let discharge (str:string) =
let file = new
StreamWriter(Path.Combine(Application.StartupPath,"formula.out"),false,System.Text.Encoding.ASCII)
file.Write(str)
file.Close()
let pi = new
ProcessStartInfo(Path.Combine(Application.StartupPath,"eprover"),"--tstp-format
--cpu-limit=2 -xAuto -tAuto " ^
Path.Combine(Application.StartupPath,"formula.out"))
pi.RedirectStandardOutput <- true
pi.RedirectStandardError <- true
pi.UseShellExecute <- false
pi.WindowStyle <- ProcessWindowStyle.Hidden
let p = Process.Start(pi)
lock result (fun () -> result := "")
p.OutputDataReceived.AddHandler dataRecvHandler
p.BeginOutputReadLine()
if not(p.WaitForExit(timeOut)) then
p.Kill()
p.CancelOutputRead()
p.OutputDataReceived.RemoveHandler dataRecvHandler
p.Close()
else
p.CancelOutputRead()
p.OutputDataReceived.RemoveHandler dataRecvHandler
lock result (fun () -> result := p.StandardError.ReadToEnd() ^
!result)
p.Close()

 

[Thorsten Tarrach]

Hi Pete,

thanks a lot. The CreateNoWindow property was what I was looking for. I've
seen it dozens of times, but always thought the default is true.

For CancelOutputRead I was hoping it would read all lines to the end. I
tried uncommenting it without effect. The truncation is not due to the
killing of the process. It happens on the code path where the process exists
normally.

Thorsten

[...]

I have one further question: When I call Start on the process which is
a command line application it causes to pop up a black command line
window that takes focus. Setting WindowStyle to
ProcessWindowStyle.Hidden does not seem to help.

I know this is possible, but can't recall for sure the solution. Have
you tried setting CreateNoWindow in the ProcessStartInfo class to "true"?

FYI... just looked at the docs, and there's a user comment saying that as
long as you set UseShellExecute to "false", the CreateNoWindow property
works.

Pete

 

[Hongye Sun [MSFT]]

Thanks for Pete's insightful analysis.

Hi Thorsten,

Thanks for your post. My name is Hongye Sun [MSFT] and it is my pleasure to
work with you on this issue.

I want to complement Pete with the following comments:

One possible method in synchronize mode:
The synchronous mode buffer cannot be changed. It is hardcoded as 0x1000.
In synchronous mode, when application running, the child process will first
write to the buffer until it exceeds the buffer size and then it will wait
for the parent process read the buffer. It can continue running after the
buffer is readed. If the parent process waits for child process to run one
second, both child and parent processs will stop running because the child
process is waiting for buffer and the parent process is waiting for child
process. The solution to break this waiting is let parent process to read
the buffer. In order to explain myself well, here is a sample code (I do
not know F# also, so I write it in C#):
-------------------------------------------------
ProcessStartInfo psi = new ProcessStartInfo(@"<Console App
Path>");
psi.CreateNoWindow = true;
psi.RedirectStandardError = true;
psi.RedirectStandardOutput = true;
psi.UseShellExecute = false;

Process p = Process.Start(psi);

Stopwatch s = new Stopwatch();
s.Start();

while (s.ElapsedMilliseconds < 1000)
{
result += p.StandardOutput.ReadLine() + "\r\n";
}
s.Stop();

p.Close();
Console.Write(result);
Console.Read();
-------------------------------------------------
The sample read the buffer line by line untill the time is greater than one
second. This works fine in our lab's environment.

For the asynchronous mode, I rewrite the code as below. It works perfect to
me. Please try the following code first. If it does not work, please show
us source code of the target console application (eprover.exe). You can
also send it to my mail box (e-mail address removed), remove 'online.'
-------------------------------------------------
DataReceivedEventHandler dataRec = new
DataReceivedEventHandler(delegate(object sender, DataReceivedEventArgs e)
{
lock (result)
{
result += (e.Data + "\r\n");
}
});

ProcessStartInfo psi = new ProcessStartInfo(@"<Console App
Path>");
psi.CreateNoWindow = true;
psi.RedirectStandardError = true;
psi.RedirectStandardOutput = true;
psi.UseShellExecute = false;

Process p = Process.Start(psi);
p.OutputDataReceived += dataRec;

lock (result) { result = ""; }
p.BeginOutputReadLine();

if (!p.WaitForExit(1000))
{
p.Kill();
p.CancelOutputRead();
p.OutputDataReceived -= dataRec;
p.Close();
}
else
{
p.CancelOutputRead();
p.OutputDataReceived -= dataRec;
lock (result) { result = p.StandardError.ReadToEnd() +
result; }
}
Console.Write(result);
Console.Read();
-------------------------------------------------

Please feel free to contact me if you have any question. I am glad to help.

Have a nice day.

Regards,
Hongye Sun ([email protected], remove 'online.')
Microsoft Online Community Support

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
(e-mail address removed).

==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/en-us/subscriptions/aa948868.aspx#notifications.

Note: MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within?2 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions. Issues of this
nature are best handled working with a dedicated Microsoft Support Engineer
by contacting Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/en-us/subscriptions/aa948874.aspx
==================================================
This posting is provided "AS IS" with no warranties, and confers no rights.

 

[Thorsten Tarrach]

Hi Sun,

thanks, that synchronous version seems to be working so far.

One more question:
I also would like to feed the input to the app via stdin. Unfortunately this
has to be terminated by Ctrl+Z or Ctrl+D. How can I send this to the
console? I already tried
p.StandardInput.Write("\u001A")
without success.

Many thanks,
Thorsten

Thanks for Pete's insightful analysis.

Hi Thorsten,

Thanks for your post. My name is Hongye Sun [MSFT] and it is my pleasure
to
work with you on this issue.

I want to complement Pete with the following comments:

One possible method in synchronize mode:
The synchronous mode buffer cannot be changed. It is hardcoded as 0x1000.
In synchronous mode, when application running, the child process will
first
write to the buffer until it exceeds the buffer size and then it will wait
for the parent process read the buffer. It can continue running after the
buffer is readed. If the parent process waits for child process to run one
second, both child and parent processs will stop running because the child
process is waiting for buffer and the parent process is waiting for child
process. The solution to break this waiting is let parent process to read
the buffer. In order to explain myself well, here is a sample code (I do
not know F# also, so I write it in C#):
-------------------------------------------------
ProcessStartInfo psi = new ProcessStartInfo(@"<Console App
Path>");
psi.CreateNoWindow = true;
psi.RedirectStandardError = true;
psi.RedirectStandardOutput = true;
psi.UseShellExecute = false;

Process p = Process.Start(psi);

Stopwatch s = new Stopwatch();
s.Start();

while (s.ElapsedMilliseconds < 1000)
{
result += p.StandardOutput.ReadLine() + "\r\n";
}
s.Stop();

p.Close();
Console.Write(result);
Console.Read();
-------------------------------------------------
The sample read the buffer line by line untill the time is greater than
one
second. This works fine in our lab's environment.

For the asynchronous mode, I rewrite the code as below. It works perfect
to
me. Please try the following code first. If it does not work, please show
us source code of the target console application (eprover.exe). You can
also send it to my mail box (e-mail address removed), remove 'online.'
-------------------------------------------------
DataReceivedEventHandler dataRec = new
DataReceivedEventHandler(delegate(object sender, DataReceivedEventArgs e)
{
lock (result)
{
result += (e.Data + "\r\n");
}
});

ProcessStartInfo psi = new ProcessStartInfo(@"<Console App
Path>");
psi.CreateNoWindow = true;
psi.RedirectStandardError = true;
psi.RedirectStandardOutput = true;
psi.UseShellExecute = false;

Process p = Process.Start(psi);
p.OutputDataReceived += dataRec;

lock (result) { result = ""; }
p.BeginOutputReadLine();

if (!p.WaitForExit(1000))
{
p.Kill();
p.CancelOutputRead();
p.OutputDataReceived -= dataRec;
p.Close();
}
else
{
p.CancelOutputRead();
p.OutputDataReceived -= dataRec;
lock (result) { result = p.StandardError.ReadToEnd() +
result; }
}
Console.Write(result);
Console.Read();
-------------------------------------------------

Please feel free to contact me if you have any question. I am glad to
help.

Have a nice day.

Regards,
Hongye Sun ([email protected], remove 'online.')
Microsoft Online Community Support

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
(e-mail address removed).

==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/en-us/subscriptions/aa948868.aspx#notifications.

Note: MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within?2 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions. Issues of this
nature are best handled working with a dedicated Microsoft Support
Engineer
by contacting Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/en-us/subscriptions/aa948874.aspx
==================================================
This posting is provided "AS IS" with no warranties, and confers no
rights.

 

[Thorsten Tarrach]

If it helps you in any way the eprover.exe can be found here:
http://www.cag.csail.mit.edu/~kostas/E/

Just enter anything in the console and press Ctrl+Z to terminate input. You
should then get an error on strerr.

 

[Hongye Sun [MSFT]]

Hi Thorsten,

From my experience, It may not be possible to simulate Ctrl-D and Ctrl-Z in
this way. I am consulting product group for confirmation.

Do you mind letting us know your business requirement behind the issue? It
may have other workarounds to archieve the same result. I saw that the
eprover.exe is running on cygwin. Does that mean the exe file is a third
party file and you cannot change its implementation?

One possible workaround that I can imagine is to make the console
application focusing on the desktop and simulate the keyboard
programmatically, but it seems violate your requirment to hide the console.

Another API is GenerateConsoleCtrlEvent Function
http://msdn.microsoft.com/en-us/library/ms683155(VS.85).aspx.
However, it can only simulate Ctrl-C and Ctrl-Break.

I will let you know once I get the confirmation from product group.

Regards,
Hongye Sun ([email protected], remove 'online.')
Microsoft Online Community Support

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
(e-mail address removed).

This posting is provided "AS IS" with no warranties, and confers no rights.

 

[Thorsten Tarrach]

Hi Sun,

This programme is a prototype implementation for a paper our group is
working on. The idea is to communicate with the eprover, a first order logic
prover, and have it evaluate certain statements to true or false. On some
input the prover may just loop forever.

This eprover is a linux application that runs under windows in cygwin, or
with the cygwin dll present. While the source is available I neither have
the time nor the expertise to work on it. The author does not officially
support Windows and is not helpful in this regard.

The eprover has a switch which allows it to specify a time limit, but that
switch does not work under Windows and is ignored. As a result I need to
kill the eprover from the .net framework, but also get the command line
output to interpret it.

Currently I write the statement to a file which I then feed to the prover.
Would be nicer to feed it through stdin.

Thanks, Thorsten

 

[Hongye Sun [MSFT]]

Thanks very much, Pete.

Close the stream should send EOF to the console. This is the same signal of
what Ctrl-D and Ctrl-Z send. I should think about it earlier. Thanks again
for correct my mistake.

Regards,
Hongye Sun ([email protected], remove 'online.')
Microsoft Online Community Support

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
(e-mail address removed).

 

[Thorsten Tarrach]

Thanks, Pete. That did indeed work as expected.

Simply close the stream:

p.StandardInput.Close();

That should do it.

Pete