don't wait for return of Process.Start() from web service

[Randomity]

Hi All

I have a web service that makes a Process.Start() call to start a long
running task that runs on the remote machine and I don't require any
further input from that machine. So I want my web method to return
immediately after the Process.Start() call. I have this configuration
on several machines now, and it works fine, except for one. It will
start the Process and wait for the end of its execution. Since this
can take up to a couple of hours, the service call times out and I get
an error, even if everything is running as planned.

Any ideas in this regard would be highly welcome!

Here's pretty much the code I have in my web service:

[WebMethod]
public void ImportData(string customer)
{

EventLog.WriteEntry(ConfigurationManager.AppSettings["LogSource"],
"Running Import for " + customer, EventLogEntryType.Information); //
Write messages to the event log of the system

try
{
using (Process importer = new Process())
{
importer.StartInfo.FileName =
ConfigurationManager.AppSettings["ImportEXEPath"];
importer.StartInfo.Arguments =
string.Format("folder:\"{0}\" customer:\"{1}\"",
ConfigurationManager.AppSettings["SettingsFolder"], customer); // Add
command line arguments for the process

if (!
string.IsNullOrEmpty(ConfigurationManager.AppSettings["args_rci"]))
importer.StartInfo.Arguments +=
string.Format(" code:{0}",
ConfigurationManager.AppSettings["args_rci"]); // Add some additional
arguments, if provided in the web.config

importer.StartInfo.UseShellExecute = false;
importer.StartInfo.RedirectStandardOutput = true;


EventLog.WriteEntry(ConfigurationManager.AppSettings["LogSource"],
DateTime.Now.ToString("dd.MM.yyyy HH:mm:ss.sss") + " Starting Process
" + importer.StartInfo.FileName);
importer.Start();

// The web method should return immediately now,
instead it takes forever to return and then times out


EventLog.WriteEntry(ConfigurationManager.AppSettings["LogSource"],
importer.StandardOutput.ReadToEnd(), EventLogEntryType.Warning); // I
added this line to see what the process was doing, since it didn't
return for a long time. It was the output coming from my command line
application, which means it's a synchronous call (instead kind of fire-
and-forget)


EventLog.WriteEntry(ConfigurationManager.AppSettings["LogSource"],
"Process started successfully", EventLogEntryType.SuccessAudit);
}
}
catch (Exception ex)
{

EventLog.WriteEntry(ConfigurationManager.AppSettings["LogSource"],
ex.Message, EventLogEntryType.Error);

EventLog.WriteEntry(ConfigurationManager.AppSettings["LogSource"],
ex.StackTrace, EventLogEntryType.Error);
}
}

The method executes without any errors/exceptions, I can view all the
log entries, including the "Process started successfully" in the Event
Viewer.

Thanks for the help

Dominik

 

[Randomity]

Hi All

I have a web service that makes a Process.Start() call to start a long
running task that runs on the remote machine and I don't require any
further input from that machine. So I want my web method to return
immediately after the Process.Start() call.

Then you have to return immediately after the Process.Start() call.

[...]
  [WebMethod]
         public void ImportData(string customer)
         {

EventLog.WriteEntry(ConfigurationManager.AppSettings["LogSource"],
"Running Import for " + customer, EventLogEntryType.Information); //
Write messages to the event log of the system

             try
             {
                 using (Process importer = new Process())
                 {
[...]

EventLog.WriteEntry(ConfigurationManager.AppSettings["LogSource"],
DateTime.Now.ToString("dd.MM.yyyy HH:mm:ss.sss") + " Starting Process
" + importer.StartInfo.FileName);
                     importer.Start();

                     // The web method should return immediately now,
instead it takes forever to return and then times out

Then you have to return immediately now, instead of calling
TextWriter.ReadToEnd().

EventLog.WriteEntry(ConfigurationManager.AppSettings["LogSource"],
importer.StandardOutput.ReadToEnd(), EventLogEntryType.Warning); // I
added this line to see what the process was doing, since it didn't
return for a long time. It was the output coming from my command line
application, which means it's a synchronous call (instead kind of fire-
and-forget)

EventLog.WriteEntry(ConfigurationManager.AppSettings["LogSource"],
"Process started successfully", EventLogEntryType.SuccessAudit);
                 }
             }
             catch (Exception ex)
             {

EventLog.WriteEntry(ConfigurationManager.AppSettings["LogSource"],
ex.Message, EventLogEntryType.Error);

EventLog.WriteEntry(ConfigurationManager.AppSettings["LogSource"],
ex.StackTrace, EventLogEntryType.Error);
             }
         }

The method executes without any errors/exceptions, I can view all the
log entries, including the "Process started successfully" in the Event
Viewer.

You'll need to post a better code example if you want help.  There's not
enough information here for anyone to be able to reproduce the problem.

Also, the code you posted does not correlate well with your description
of the problem.  In particular:

   • Calling ReadToEnd() on the stdout TextWriter is necessarily going
to synchronize your code with the process lifetime, not returning until
the process has itself exited.

So, as the code stands now, it appears to be written to do exactly what
you don't want it to do.

   • All that seeing "Process started successfully" in the event log
tells you is that the calls to Process.Start() and
TextWriter.ReadToEnd() did eventually return, with no interruption of
that thread on the server side.  It doesn't rule out a client-side timeout.

What you really need to look at are the timestamps for the events in the
event log.  That will tell you when those messages actually got written,
giving you a better idea of how long the process actually took and what
timing of the execution flow in that section of code is.

Without further clarification about the code example, I'd say that the
main thing you need to do is get rid of the call to ReadToEnd(), at
least in that thread (you can run it asynchronously…just make sure the
thread that is servicing the client is permitted to return immediately).

If that's not actually the problem, you need to clarify why you posted
code that has that blocking call in it, and why you believe that code
example is relevant.

Pete

Hi Pete

Thanks for the input! I actually already did what you proposed, to
look at the timestamps. I only added the ReadToEnd() call after
looking at the timestamps. In the event log it would show all messages
until " Starting Process {Filename}..." and the process.Start() would
be executed, but wouldn't return (this without the ReadToEnd() after
the process start!). I added the ReadToEnd() to get some information
on what the thread is doing, which of course gave me the standard
output. Even without the ReadToEnd() it will execute as with the
ReadToEnd(), I don't see any difference in the processes running in
the background, memory usage or response time of the web service.
Does that information help at all? You can pretty much ignore the
ReadToEnd(), because it was added after the described problem. But
I'll remove that line to see, if anything changes...

Thanks again

Dominik

 

[Randomity]

[...]Even without the ReadToEnd() it will execute as with the
ReadToEnd(), I don't see any difference in the processes running in
the background, memory usage or response time of the web service.
Does that information help at all? You can pretty much ignore the
ReadToEnd(), because it was added after the described problem. But
I'll remove that line to see, if anything changes...

The only thing that comes to mind absent a proper code example is that
if your process emits enough output, then if you _don't_ read the output
as the process runs, the output buffer will fill and the process will
get stuck.

But that should affect only the process itself.  Your method that starts
the process should still report the processing starting successfully and
return.  And of course, in that scenario, calling ReadToEnd() should
_fix_ the problem, since your thread would then be retrieving the output
as needed.

I'd be skeptical that a full output buffer would explain the problem.

That said, part of the issue is that the problem description itself is
quite vague, as is the code example.  There's not really much
information to go on here.  It's not clear from your post what the exact
sequence of code execution is, nor whether you have actually attempted
to run the code in the debugger to step through it and see what it does,
etc.

For that matter, one way to read your description is that on the server
side, everything goes according to expectations, but the process you are
running simply takes a long time and that's normal.  In that case, I
could well believe that on the client side, you exceed a timeout and get
an error.

If that's all that's going on, then you either need to make your process
complete sooner, or extend the timeout length.  Note that one way of
accomplishing the former is to break the process down into smaller
pieces and return the results incrementally.

Pete

Hi Pete

I don't know what more code you require for the problem? The web
service really doesn't do much, to put my code a bit simpler, all I do
is pretty much
{
Process p = new Process(); // Create a new process
p.StartInfo.FileName = "my_Application.exe"; // Assign the
executable
p.StartInfo.Arguments = "some args"; // set the command line
arguments
p.Start(); // start the process and then return
}

Everything else is to get a hand on what's happening. I've been
stepping through the code on my machine, and after the p.Start()-step,
it returns immediately. It doesn't, however, on the server. That's
what I'm not understanding. I have an exe that's running a long task
and I don't need a response for the process calling the web service.
So I want to start my Application through the web service and then
return immediately after that.

My code was like follows, before I put in the whole exception handling
and event logging in to get a hand on why it's taking so long to run

public void ImportData(string customer)
{
using (Process importer = new Process())
{
importer.StartInfo.FileName = "ma_application.exe";

importer.StartInfo.Arguments = "folder customer";

importer.Start();
}
}

I put the process in a using-clause to make sure it's disposed
properly. But even that I added later on to make sure I'm not wasting
any memory on the web server. Does that information help at all? Maybe
I'm misunderstanding the concept of Process.Start(), but shouldn't it
be unimportant what kind of process is started, as long as the start
was successful? (Unless of course I'd also call WairForExit() on
it...)

Dominik

 

[Randomity]

On 11/8/11 11:43 PM, Randomity wrote:

[...]Even without the ReadToEnd() it will execute as with the
ReadToEnd(), I don't see any difference in the processes running in
the background, memory usage or response time of the web service.
Does that information help at all? You can pretty much ignore the
ReadToEnd(), because it was added after the described problem. But
I'll remove that line to see, if anything changes...

The only thing that comes to mind absent a proper code example is that
if your process emits enough output, then if you _don't_ read the output
as the process runs, the output buffer will fill and the process will
get stuck.

But that should affect only the process itself.  Your method that starts
the process should still report the processing starting successfully and
return.  And of course, in that scenario, calling ReadToEnd() should
_fix_ the problem, since your thread would then be retrieving the output
as needed.

I'd be skeptical that a full output buffer would explain the problem.

That said, part of the issue is that the problem description itself is
quite vague, as is the code example.  There's not really much
information to go on here.  It's not clear from your post what the exact
sequence of code execution is, nor whether you have actually attempted
to run the code in the debugger to step through it and see what it does,
etc.

For that matter, one way to read your description is that on the server
side, everything goes according to expectations, but the process you are
running simply takes a long time and that's normal.  In that case, I
could well believe that on the client side, you exceed a timeout and get
an error.

If that's all that's going on, then you either need to make your process
complete sooner, or extend the timeout length.  Note that one way of
accomplishing the former is to break the process down into smaller
pieces and return the results incrementally.

Pete

Hi Pete

I don't know what more code you require for the problem? The web
service really doesn't do much, to put my code a bit simpler, all I do
is pretty much
{
  Process p = new Process(); // Create a new process
  p.StartInfo.FileName = "my_Application.exe"; // Assign the
executable
  p.StartInfo.Arguments = "some args"; // set the command line
arguments
  p.Start(); // start the process and then return

}

Everything else is to get a hand on what's happening. I've been
stepping through the code on my machine, and after the p.Start()-step,
it returns immediately. It doesn't, however, on the server. That's
what I'm not understanding. I have an exe that's running a long task
and I don't need a response for the process calling the web service.
So I want to start my Application through the web service and then
return immediately after that.

My code was like follows, before I put in the whole exception handling
and event logging in to get a hand on why it's taking so long to run

public void ImportData(string customer)
{
   using (Process importer = new Process())
   {
      importer.StartInfo.FileName = "ma_application.exe";

      importer.StartInfo.Arguments = "folder customer";

      importer.Start();
   }

}

I put the process in a using-clause to make sure it's disposed
properly. But even that I added later on to make sure I'm not wasting
any memory on the web server. Does that information help at all? Maybe
I'm misunderstanding the concept of Process.Start(), but shouldn't it
be unimportant what kind of process is started, as long as the start
was successful? (Unless of course I'd also call WairForExit() on
it...)

Dominik

Ok, the web server seems to have had a problem. I've been testing all
morning to get it working, and suddenly it worked, for no apparent
reason. All other machines running this web service have been working
correctly from the beginning. I don't get it...

Thanks for the support and input Pete!

Dominik

 

[Randomity]

I don't know what more code you require for the problem?  [...]

The best questions include everything needed to reproduce the problem
without having to write any new code.  A problem like yours is
particularly difficult, because it involves so many different
components: a web server, client, and at least one external process.
But that doesn't remove the need to provide precise details for all
components involved; it just makes that more challenging to do.

We here in the newsgroup can (and often will) speculate based on
incomplete information, but such speculation is often wrong, irrelevant
and, in the worst case, distracting from the real problem, whatever that
happens to be.

Beyond that, even without 100% of the code and other components, a
complete description is best.  More details about the external process
could have been helpful, but even more important would be specifics
about what exactly is different in the case of the scenario that doesn't
work as compared to the ones that do.  If there really is no actual
difference, then being specific about that is just as important.

Anyway, all of the above is mainly for future reference.  I gather from
your follow-up message that you've determined the problem to not be a
programming error, but rather some problem with a particular computer
(configuration, hardware, whatever).  So perhaps no amount of detail
would have led to a solution in this case, except to provide some added
confidence that the code itself is correct (i.e. to help avoid spending
time looking in the wrong place for the problem).

In any case, I'm glad the issue has been identified and that you've got
your system working again!

Pete

Thanks Pete, I'll definitely remember your advice for future
questions! I must admit, it was my first post, and as such, I still
have to better understand what information I think is useful, and what
others think would be helpful. I just realized now by what you said
that there was still a difference between my understanding of what was
required and your request for more code
It really seems to be a system problem, not a code problem (99.9%
sure).

Thanks for your time and help, I really appreciate that!

Dominik