Singleton Problem

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

Hello,
We have an application which communicates using remoting. There is a server
which is a Windows Service. The server exposes an object which is a
singleton. The client is a Web Application which makes calls to the service.
We are using tcp channel which is using binaryformatter by default. The
problem is that after a certain number of remoting calls the calls dont get
through to the server. The client application makes the call and hangs up.
The call never reaches the service. The initial calls get through. Earlier we
used http channel. Then the problem was worse. The client could not make more
than 20 calls. Now it hangs up after about 100 or so calls if made
sucessively. We had come to conclusion that it is some threadpool issue as we
observed that after some time the thread count goes high. The observation in
case of both http and tcp was that if the thread count crosses a certain
limit, the server does not accept more calls. If the calls don't come
successively, ie with in a short amount of time, the service does accept
calls. I believe this is not a synchronization problem as we have use sync
objects to protect in remoting calls. Also another point in this favor is
that the same code hangs up earlier with http channel.
The configuration for remote server is
<system.runtime.remoting>
<application>
<service>
<wellknown mode="Singleton" displayName="*****"
type="***.***,Service.exe" objectUri="***.rem" />
</service>
<channels>
<channel ref="tcp" port="1978"></channel>
</channels>
</application>
</system.runtime.remoting>

// This is in the shared assembly
public __gc __interface IInterface
{
virtual System::String* Request(System::String* a_pRequest) = 0;
};

// This is in an assembly with the service
public __gc class CInterface : public IInterface, public
System::MarshalByRefObject
{
public:
// Singleton Pattern applied
virtual System::String* Request(System::String* a_pRequest) {
privateLock(); Process(); privateUnLock();
System::GC::Collect(); // After ading this we get to process more calls
return "****" };
virtual System::Object* InitializeLifetimeService() {return 0;};
void privateRegisterForRemoting()
{
try
{

System::Runtime::Remoting::RemotingConfiguration::Configure(System::AppDomain::CurrentDomain->SetupInformation->ConfigurationFile);
System::Runtime::Remoting::RemotingServices::Marshal(this, "***.rem");
}
catch(System::Exception* a_pException)
{
WRITETOSYSTEMLOG(a_pException);
}
};
What could be the problem and how can this be addressed? We strongly think
this is some threadpool issue. Or there may be some problem the way we use
remoting.
We can change it to singlecall if needed, but first we want to check the issue
 
Sudesh,

The Request method calls privateLock, Process, and privateUnLock in
that order. One thing that jumps out at me is that you don't have the
Process method call wrapped in a try/catch/finally block. If an
unhandled exception is thrown in the Process method then the
privateUnLock method won't be called and all subsequent remoting calls
will be blocked indefinitely.

Brian
 
Hi Brian,
Process is just a dummy function. The actual processing function has try
catch blocks.
My main problem is that after a certain number of remoting calls the calls
dont get through to the server. I had tied searching on the net for the
problem and found from a couple of sites that people where facing a similar
problem. We got improvements in performance when we changed the channel to
tcp, but still it doesnot suffice. The explanation given for this was that
the clr threadpool gets exhausted and cant fork more threads for more
requests. I had tried to check with my code if there are any leaks but then i
get the threadcount decreasing after the request has been completed, so there
may be no such leak. But somewhere the requests have piledup at lower level
and the pool gets exhausted
My last approach is to try singlecall.
Cheers,
Sudesh
 
Have you attached a debbuger to the service when the call hangs? If so how
many threads are running on the service?
 
Hello Jared,
I had attached the debugger to the service. Infact I had tried to trace it
through the client to my servce. The request goes ahead from the client and
does not reach the service. I had tried to solve the problem in the following
steps.
1. Called GC::Collect from the methods (methods which are being remotely
called). This improved the performance a lot.
2. Changed from http channel to tcp channel. Even this improved the
performance, but still the requests are limited.

When looking for number of threads, the thread count at which it stops
functioning in case of tcp is 91 and in case of http it is 54. Out of these
around 33 threads are created by the service. The other threads are spawned
at every request. If there is some time lag in between requests, the thread
count goes down. but the service stops remoting requests after the threshold
mentioned above is reached. I had found a link which recommendd changing the
threadpool size, but i want to try this as the last solution.

Cheers,
sudesh


Jared Parsons said:
Have you attached a debbuger to the service when the call hangs? If so how
many threads are running on the service?

--
Jared Parsons [MSFT]
(e-mail address removed)
http://blogs.msdn.com/jaredpar
"This posting is provided "AS IS" with no warranties, and confers no rights"
Sudesh Sawant said:
Hi Brian,
Process is just a dummy function. The actual processing function has try
catch blocks.
My main problem is that after a certain number of remoting calls the calls
dont get through to the server. I had tied searching on the net for the
problem and found from a couple of sites that people where facing a
similar
problem. We got improvements in performance when we changed the channel to
tcp, but still it doesnot suffice. The explanation given for this was that
the clr threadpool gets exhausted and cant fork more threads for more
requests. I had tried to check with my code if there are any leaks but
then i
get the threadcount decreasing after the request has been completed, so
there
may be no such leak. But somewhere the requests have piledup at lower
level
and the pool gets exhausted
My last approach is to try singlecall.
Cheers,
Sudesh
Click to expand...
Click to expand...
 
Hello Jared,
Still the problem is not resolved.
Cheers,
Sudesh

Sudesh Sawant said:
Ok, Will try this.
Cheers,
Sudesh Sawant

Jared Parsons said:
You could try isolating the problem by hacking up your service to give back
a default answer and not spawn any of it's own threads and see if you still
hit the problem

--
Jared Parsons [MSFT]
(e-mail address removed)
http://blogs.msdn.com/jaredpar
"This posting is provided "AS IS" with no warranties, and confers no rights"
Sudesh Sawant said:
Hello Jared,
I had attached the debugger to the service. Infact I had tried to trace it
through the client to my servce. The request goes ahead from the client
and
does not reach the service. I had tried to solve the problem in the
following
steps.
1. Called GC::Collect from the methods (methods which are being remotely
called). This improved the performance a lot.
2. Changed from http channel to tcp channel. Even this improved the
performance, but still the requests are limited.

When looking for number of threads, the thread count at which it stops
functioning in case of tcp is 91 and in case of http it is 54. Out of
these
around 33 threads are created by the service. The other threads are
spawned
at every request. If there is some time lag in between requests, the
thread
count goes down. but the service stops remoting requests after the
threshold
mentioned above is reached. I had found a link which recommendd changing
the
threadpool size, but i want to try this as the last solution.

Cheers,
sudesh


:

Have you attached a debbuger to the service when the call hangs? If so
how
many threads are running on the service?

--
Jared Parsons [MSFT]
(e-mail address removed)
http://blogs.msdn.com/jaredpar
"This posting is provided "AS IS" with no warranties, and confers no
rights"
Hi Brian,
Process is just a dummy function. The actual processing function has
try
catch blocks.
My main problem is that after a certain number of remoting calls the
calls
dont get through to the server. I had tied searching on the net for the
problem and found from a couple of sites that people where facing a
similar
problem. We got improvements in performance when we changed the channel
to
tcp, but still it doesnot suffice. The explanation given for this was
that
the clr threadpool gets exhausted and cant fork more threads for more
requests. I had tried to check with my code if there are any leaks but
then i
get the threadcount decreasing after the request has been completed, so
there
may be no such leak. But somewhere the requests have piledup at lower
level
and the pool gets exhausted
My last approach is to try singlecall.
Cheers,
Sudesh
:

Sudesh,

The Request method calls privateLock, Process, and privateUnLock in
that order. One thing that jumps out at me is that you don't have the
Process method call wrapped in a try/catch/finally block. If an
unhandled exception is thrown in the Process method then the
privateUnLock method won't be called and all subsequent remoting calls
will be blocked indefinitely.

Brian

Sudesh Sawant wrote:
Hello,
We have an application which communicates using remoting. There is a
server
which is a Windows Service. The server exposes an object which is a
singleton. The client is a Web Application which makes calls to the
service.
We are using tcp channel which is using binaryformatter by default.
The
problem is that after a certain number of remoting calls the calls
dont
get
through to the server. The client application makes the call and
hangs
up.
The call never reaches the service. The initial calls get through.
Earlier we
used http channel. Then the problem was worse. The client could not
make more
than 20 calls. Now it hangs up after about 100 or so calls if made
sucessively. We had come to conclusion that it is some threadpool
issue
as we
observed that after some time the thread count goes high. The
observation in
case of both http and tcp was that if the thread count crosses a
certain
limit, the server does not accept more calls. If the calls don't
come
successively, ie with in a short amount of time, the service does
accept
calls. I believe this is not a synchronization problem as we have
use
sync
objects to protect in remoting calls. Also another point in this
favor
is
that the same code hangs up earlier with http channel.
The configuration for remote server is
<system.runtime.remoting>
<application>
<service>
<wellknown mode="Singleton" displayName="*****"
type="***.***,Service.exe" objectUri="***.rem" />
</service>
<channels>
<channel ref="tcp" port="1978"></channel>
</channels>
</application>
</system.runtime.remoting>

// This is in the shared assembly
public __gc __interface IInterface
{
virtual System::String* Request(System::String* a_pRequest) = 0;
};

// This is in an assembly with the service
public __gc class CInterface : public IInterface, public
System::MarshalByRefObject
{
public:
// Singleton Pattern applied
virtual System::String* Request(System::String* a_pRequest) {
privateLock(); Process(); privateUnLock();
System::GC::Collect(); // After ading this we get to process more
calls
return "****" };
virtual System::Object* InitializeLifetimeService() {return 0;};
void privateRegisterForRemoting()
{
try
{

System::Runtime::Remoting::RemotingConfiguration::Configure(System::AppDomain::CurrentDomain->SetupInformation->ConfigurationFile);
System::Runtime::Remoting::RemotingServices::Marshal(this,
"***.rem");
}
catch(System::Exception* a_pException)
{
WRITETOSYSTEMLOG(a_pException);
}
};
What could be the problem and how can this be addressed? We strongly
think
this is some threadpool issue. Or there may be some problem the way
we
use
remoting.
We can change it to singlecall if needed, but first we want to check
the issue
Click to expand...
Click to expand...
Click to expand...
 
So even the dummy call of service hangs after 91 times? if so - maybe the
problem lays on client side. how many clients do you use to test this? are
they multithreaded? how many proxies do you create on client side?

Peter
 
Back
Top