windows Service Hangs

[Jacob Crossley]

Hello all. We have about 10 Window's services that we wrote in c#. We use
them to process row's that we have queued up in various SQL tables.

The services seem to hang at least once in any given 24 hour period. Once we
reset the service, it goes about its processing business until the next time
it hangs. We are wondering if the hanging problem is due to a load issue
(too many windows services on a single machine), a code flaw, or an inherent
c# bug? We used to write NT services in VB6 using the ntsvc.ocx. Those
services never hung. I'm wondering if we are going to have to rewrite all of
our c# services in old school vb6. I've included the Service code
below...any help would be much appreciated.

using System;
using System.Collections;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.ServiceProcess;
using System.Configuration;

namespace IAR_Service
{
public class IAR : System.ServiceProcess.ServiceBase
{
private System.Timers.Timer tmrWinService;
/// <summary>
/// Required designer variable.
/// </summary>
private System.ComponentModel.Container components = null;

public IAR()
{
// This call is required by the Windows.Forms Component Designer.
InitializeComponent();

// TODO: Add any initialization after the InitComponent call
}

// The main entry point for the process
static void Main()
{
System.ServiceProcess.ServiceBase[] ServicesToRun;

// More than one user Service may run within the same process. To add
// another service to this process, change the following line to
// create a second service object. For example,
//
// ServicesToRun = New System.ServiceProcess.ServiceBase[] {new
Service1(), new MySecondUserService()};
//
ServicesToRun = new System.ServiceProcess.ServiceBase[] { new IAR() };

System.ServiceProcess.ServiceBase.Run(ServicesToRun);
}

/// <summary>
/// Required method for Designer support - do not modify
/// the contents of this method with the code editor.
/// </summary>
private void InitializeComponent()
{
this.tmrWinService = new System.Timers.Timer();

((System.ComponentModel.ISupportInitialize)(this.tmrWinService)).BeginInit()
;
//
// tmrWinService
//
//this.tmrWinService.Enabled = true;
this.tmrWinService.Interval = 20000;
this.tmrWinService.Elapsed += new
System.Timers.ElapsedEventHandler(this.tmrWinService_Elapsed);
//
// IAR
//
this.AutoLog = false;
this.CanPauseAndContinue = true;
this.ServiceName = "IAR";

((System.ComponentModel.ISupportInitialize)(this.tmrWinService)).EndInit();

}

/// <summary>
/// Clean up any resources being used.
/// </summary>
protected override void Dispose( bool disposing )
{
if( disposing )
{
if (components != null)
{
components.Dispose();
}
}
base.Dispose( disposing );
}

/// <summary>
/// Set things in motion so your service can do its work.
/// </summary>
protected override void OnStart(string[] args)
{
// TODO: Add code here to start your service.
tmrWinService.Enabled = true;
tmrWinService.AutoReset = true;
tmrWinService.Start();
//see if log exists. if not, create
if (!EventLog.SourceExists("IAR-Service")) EventLog.CreateEventSource("",
"IAR-Service", ".");
//log
EventLog log = new EventLog();
log.Log = "IAR-Service";
log.Source = "WinServiceIAR.OnStart";
log.WriteEntry("Starting IAR windows Service",
EventLogEntryType.Information);
log.Dispose();
}

/// <summary>
/// Stop this service.
/// </summary>
protected override void OnStop()
{
// TODO: Add code here to perform any tear-down necessary to stop your
service.
tmrWinService.Enabled = true;
tmrWinService.AutoReset = true;
tmrWinService.Stop();
//see if log exists. if not, create
if (!EventLog.SourceExists("IAR-Service")) EventLog.CreateEventSource("",
"IAR-Service", ".");
//log
EventLog log = new EventLog();
log.Log = "IAR-Service";
log.Source = "WinServiceIAR.OnStop";
log.WriteEntry("Stopping IAR windows Service",
EventLogEntryType.Information);
log.Dispose();
}

private void tmrWinService_Elapsed(Object sender,
System.Timers.ElapsedEventArgs e)
{
this.tmrWinService.Stop();
try
{
UserServices oUser = new UserServices();
ScriptingTelnet st = new ScriptingTelnet();
//CALL
oUser.processIAR();

}
catch (Exception err)
{
//see if log exists. if not, create
if (!EventLog.SourceExists("IAR-Service"))
EventLog.CreateEventSource("", "IAR-Service", ".");
//log
EventLog log = new EventLog();
log.Log = "IAR-Service";
log.Source = "WinServiceIAR.tmrWinService_Elapsed";
log.WriteEntry(err.Message.ToString(), EventLogEntryType.Error);
log.Dispose();
}
finally
{
this.tmrWinService.Start();
}
}
}
}

 

[Willy Denoyette [MVP]]

What's the purpose of this?

ScriptingTelnet st = new ScriptingTelnet();

Willy.

Hello all. We have about 10 Window's services that we wrote in c#. We use
them to process row's that we have queued up in various SQL tables.

The services seem to hang at least once in any given 24 hour period. Once
we
reset the service, it goes about its processing business until the next
time
it hangs. We are wondering if the hanging problem is due to a load issue
(too many windows services on a single machine), a code flaw, or an
inherent
c# bug? We used to write NT services in VB6 using the ntsvc.ocx. Those
services never hung. I'm wondering if we are going to have to rewrite all
of
our c# services in old school vb6. I've included the Service code
below...any help would be much appreciated.

using System;
using System.Collections;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.ServiceProcess;
using System.Configuration;

namespace IAR_Service
{
public class IAR : System.ServiceProcess.ServiceBase
{
private System.Timers.Timer tmrWinService;
/// <summary>
/// Required designer variable.
/// </summary>
private System.ComponentModel.Container components = null;

public IAR()
{
// This call is required by the Windows.Forms Component Designer.
InitializeComponent();

// TODO: Add any initialization after the InitComponent call
}

// The main entry point for the process
static void Main()
{
System.ServiceProcess.ServiceBase[] ServicesToRun;

// More than one user Service may run within the same process. To add
// another service to this process, change the following line to
// create a second service object. For example,
//
// ServicesToRun = New System.ServiceProcess.ServiceBase[] {new
Service1(), new MySecondUserService()};
//
ServicesToRun = new System.ServiceProcess.ServiceBase[] { new IAR() };

System.ServiceProcess.ServiceBase.Run(ServicesToRun);
}

/// <summary>
/// Required method for Designer support - do not modify
/// the contents of this method with the code editor.
/// </summary>
private void InitializeComponent()
{
this.tmrWinService = new System.Timers.Timer();

((System.ComponentModel.ISupportInitialize)(this.tmrWinService)).BeginInit()
;
//
// tmrWinService
//
//this.tmrWinService.Enabled = true;
this.tmrWinService.Interval = 20000;
this.tmrWinService.Elapsed += new
System.Timers.ElapsedEventHandler(this.tmrWinService_Elapsed);
//
// IAR
//
this.AutoLog = false;
this.CanPauseAndContinue = true;
this.ServiceName = "IAR";

((System.ComponentModel.ISupportInitialize)(this.tmrWinService)).EndInit();

}

/// <summary>
/// Clean up any resources being used.
/// </summary>
protected override void Dispose( bool disposing )
{
if( disposing )
{
if (components != null)
{
components.Dispose();
}
}
base.Dispose( disposing );
}

/// <summary>
/// Set things in motion so your service can do its work.
/// </summary>
protected override void OnStart(string[] args)
{
// TODO: Add code here to start your service.
tmrWinService.Enabled = true;
tmrWinService.AutoReset = true;
tmrWinService.Start();
//see if log exists. if not, create
if (!EventLog.SourceExists("IAR-Service"))
EventLog.CreateEventSource("",
"IAR-Service", ".");
//log
EventLog log = new EventLog();
log.Log = "IAR-Service";
log.Source = "WinServiceIAR.OnStart";
log.WriteEntry("Starting IAR windows Service",
EventLogEntryType.Information);
log.Dispose();
}

/// <summary>
/// Stop this service.
/// </summary>
protected override void OnStop()
{
// TODO: Add code here to perform any tear-down necessary to stop your
service.
tmrWinService.Enabled = true;
tmrWinService.AutoReset = true;
tmrWinService.Stop();
//see if log exists. if not, create
if (!EventLog.SourceExists("IAR-Service"))
EventLog.CreateEventSource("",
"IAR-Service", ".");
//log
EventLog log = new EventLog();
log.Log = "IAR-Service";
log.Source = "WinServiceIAR.OnStop";
log.WriteEntry("Stopping IAR windows Service",
EventLogEntryType.Information);
log.Dispose();
}

private void tmrWinService_Elapsed(Object sender,
System.Timers.ElapsedEventArgs e)
{
this.tmrWinService.Stop();
try
{
UserServices oUser = new UserServices();
ScriptingTelnet st = new ScriptingTelnet();
//CALL
oUser.processIAR();

}
catch (Exception err)
{
//see if log exists. if not, create
if (!EventLog.SourceExists("IAR-Service"))
EventLog.CreateEventSource("", "IAR-Service", ".");
//log
EventLog log = new EventLog();
log.Log = "IAR-Service";
log.Source = "WinServiceIAR.tmrWinService_Elapsed";
log.WriteEntry(err.Message.ToString(), EventLogEntryType.Error);
log.Dispose();
}
finally
{
this.tmrWinService.Start();
}
}
}
}

 

[Jacob Crossley]

That just instantiates another class in the sln....Its a custom class that
adds and removes people from an acl on a router to restrict/permit internet
access at our ISP...I could include that code if you'd like, but all of our
services hang, regardless of what they do in the worker classes...I'd
propose that the service would hang even if it called a class that wrote
'hello world' to a text file.

What's the purpose of this?

ScriptingTelnet st = new ScriptingTelnet();

Willy.

Hello all. We have about 10 Window's services that we wrote in c#. We use
them to process row's that we have queued up in various SQL tables.

The services seem to hang at least once in any given 24 hour period. Once
we
reset the service, it goes about its processing business until the next
time
it hangs. We are wondering if the hanging problem is due to a load issue
(too many windows services on a single machine), a code flaw, or an
inherent
c# bug? We used to write NT services in VB6 using the ntsvc.ocx. Those
services never hung. I'm wondering if we are going to have to rewrite all
of
our c# services in old school vb6. I've included the Service code
below...any help would be much appreciated.

using System;
using System.Collections;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.ServiceProcess;
using System.Configuration;

namespace IAR_Service
{
public class IAR : System.ServiceProcess.ServiceBase
{
private System.Timers.Timer tmrWinService;
/// <summary>
/// Required designer variable.
/// </summary>
private System.ComponentModel.Container components = null;

public IAR()
{
// This call is required by the Windows.Forms Component Designer.
InitializeComponent();

// TODO: Add any initialization after the InitComponent call
}

// The main entry point for the process
static void Main()
{
System.ServiceProcess.ServiceBase[] ServicesToRun;

// More than one user Service may run within the same process. To add
// another service to this process, change the following line to
// create a second service object. For example,
//
// ServicesToRun = New System.ServiceProcess.ServiceBase[] {new
Service1(), new MySecondUserService()};
//
ServicesToRun = new System.ServiceProcess.ServiceBase[] { new IAR() };

System.ServiceProcess.ServiceBase.Run(ServicesToRun);
}

/// <summary>
/// Required method for Designer support - do not modify
/// the contents of this method with the code editor.
/// </summary>
private void InitializeComponent()
{
this.tmrWinService = new System.Timers.Timer();

((System.ComponentModel.ISupportInitialize)(this.tmrWinService)).BeginInit()
;
//
// tmrWinService
//
//this.tmrWinService.Enabled = true;
this.tmrWinService.Interval = 20000;
this.tmrWinService.Elapsed += new
System.Timers.ElapsedEventHandler(this.tmrWinService_Elapsed);
//
// IAR
//
this.AutoLog = false;
this.CanPauseAndContinue = true;
this.ServiceName = "IAR";

((System.ComponentModel.ISupportInitialize)(this.tmrWinService)).EndInit();

}

/// <summary>
/// Clean up any resources being used.
/// </summary>
protected override void Dispose( bool disposing )
{
if( disposing )
{
if (components != null)
{
components.Dispose();
}
}
base.Dispose( disposing );
}

/// <summary>
/// Set things in motion so your service can do its work.
/// </summary>
protected override void OnStart(string[] args)
{
// TODO: Add code here to start your service.
tmrWinService.Enabled = true;
tmrWinService.AutoReset = true;
tmrWinService.Start();
//see if log exists. if not, create
if (!EventLog.SourceExists("IAR-Service"))
EventLog.CreateEventSource("",
"IAR-Service", ".");
//log
EventLog log = new EventLog();
log.Log = "IAR-Service";
log.Source = "WinServiceIAR.OnStart";
log.WriteEntry("Starting IAR windows Service",
EventLogEntryType.Information);
log.Dispose();
}

/// <summary>
/// Stop this service.
/// </summary>
protected override void OnStop()
{
// TODO: Add code here to perform any tear-down necessary to stop your
service.
tmrWinService.Enabled = true;
tmrWinService.AutoReset = true;
tmrWinService.Stop();
//see if log exists. if not, create
if (!EventLog.SourceExists("IAR-Service"))
EventLog.CreateEventSource("",
"IAR-Service", ".");
//log
EventLog log = new EventLog();
log.Log = "IAR-Service";
log.Source = "WinServiceIAR.OnStop";
log.WriteEntry("Stopping IAR windows Service",
EventLogEntryType.Information);
log.Dispose();
}

private void tmrWinService_Elapsed(Object sender,
System.Timers.ElapsedEventArgs e)
{
this.tmrWinService.Stop();
try
{
UserServices oUser = new UserServices();
ScriptingTelnet st = new ScriptingTelnet();
//CALL
oUser.processIAR();

}
catch (Exception err)
{
//see if log exists. if not, create
if (!EventLog.SourceExists("IAR-Service"))
EventLog.CreateEventSource("", "IAR-Service", ".");
//log
EventLog log = new EventLog();
log.Log = "IAR-Service";
log.Source = "WinServiceIAR.tmrWinService_Elapsed";
log.WriteEntry(err.Message.ToString(), EventLogEntryType.Error);
log.Dispose();
}
finally
{
this.tmrWinService.Start();
}
}
}
}

 

[Willy Denoyette [MVP]]

Willy.

That just instantiates another class in the sln....Its a custom class that
adds and removes people from an acl on a router to restrict/permit
internet
access at our ISP...I could include that code if you'd like, but all of
our
services hang, regardless of what they do in the worker classes...I'd
propose that the service would hang even if it called a class that wrote
'hello world' to a text file.

No need to check this with a simple "hello world" service ,I have a
couple of services ,running for weeks now, performing some tasks at regular
intervals using System.Threading.Timer.

Is it possible that this custom class takes longer than the interval? And,
when it's the possibility exists, are you sure that, your custom class is
threadsafe, and there is no possibility for race conditions leading to
deadlocks?
I suggest you watch some resources like # of handles, # of run-able threads
when the service runs and when it hangs.
Another thing you can do is attach a debugger to your hung service and try
to find out where it hangs.

Willy.

 

[Jacob Crossley]

I don't think that the time it takes the custom class to work is an issue at
all. The timer is stopped at the beginning of the timer elapsed event and
not re-started until the finally block (after the worker class has had time
to do its work). Do you agree?

 

[Willy Denoyette [MVP]]

Jacob,

If you are sure there is no chance for reentrancy, I would suggest that you
get rid of the timer stop and start in your eventhandler.
That way you will allways use the same kernel timer object, which is not
true when you stop and start the timer.

If there is a chance for reentrency I would take another approach as
illustrated here....

....
int timerEventRunning = 0; // 0=false, 1=true

private void tmrWinService_Elapsed(Object sender,
System.Timers.ElapsedEventArgs e)
{
// guard against reentrency in a threadsafe fashion
if(Interlocked.CompareExchange(ref timerEventRunning, 1, 0) == 0)
{
// Set timerEventRunning to 1 and ...
// handle your timer event here ...
.....
// when done ... reset TimerEventRunning
Interlocked.Decrement(ref timerEventRunning); // set
timerEventRunning to 0
}
// Timer event re-entered ... ignore
}

Willy.

 

[Jacob Crossley]

I will try your Interlocked.CompareExchange and see if that works. I don't
understand what you are talking about in the other choice when you say,
'...get rid of the timer stop and start in your eventhandler. That way you
will always use the same kernel timer object...." What do you mean by kernel
timer object...is that something behind the scenes or a library I explicitly
create and call?

 

[Jacob Crossley]

I really like your suggestion, however, I have one more question. Below the
line of code:

if(Interlocked.CompareExchange(ref timerEventRunning, 1, 0) == 0)
{

you have the comment:

// Set timerEventRunning to 1 and ...
// handle your timer event here ...

will I set timeEvenRunning to 1 by calling
Interlocked.Increment(ref timerEventRunning); ?

 

[Willy Denoyette [MVP]]

I really like your suggestion, however, I have one more question. Below the
line of code:


you have the comment:


will I set timeEvenRunning to 1 by calling
Interlocked.Increment(ref timerEventRunning); ?

Jacob,

Sorry for the confusion, here is a beter formatted snippet...

....
{
// guard against reentrancy in a threadsafe fashion
// Set timerEventRunning to 1, but only when it's 0 (eventhandler not
running)
if(Interlocked.CompareExchange(ref timerEventRunning, 1, 0) == 0)
{

// handle your timer event here ...
.....

// set timerEventRunning to 0
Interlocked.Decrement(ref timerEventRunning);
}
// else, eventhandler reentered - ignore
.....

Now back to your question in your previous post.

First you should now that System.Timers.Timer is using a "waitable timer
kernel object" behind the scenes.
When you call Stop (or set Disabled = false), the kernel object is released
(calling CoseHandle), when you call Start (or set Enabled=true), not only
you create a new kernel timer object, but there is also some cross thread
signaling going on to pass the object to the helper thread that is part of
the implementation of the Timer code. Passing the object involves taking a
lock, setting some variables and setting an event the helper thread is
waiting on. The calling thread then waits for an event from the helper that
indicates that it received and processed the new timer info. Only then will
Start (Enabled=true) return. The lock taken is lock(typeof(WaitableTimer)
IMO not the best lock to grab.

Therefore I suggest you just let the timer fire without interfering with it
and just guard against reentrancy like shown in above snippet.
The advantage is that chances for a deadlock in the timer enabling code is
removed, hopefully solving your problem.

Willy.

 

[Jacob Crossley]

Thanks for all your help Willy! Hopefully we'll have this tested and in
production by early next week.