The timeout period elapsed prior to obtaining a connection from th

[John Staggs]

Hello,

Please read this all before giving an answer I'm doing some
troubleshooting on a web application that my company wrote. It's written in
asp.net 1.1. The error that the Event viewer gives is:

Timeout expired. The timeout period elapsed prior to obtaining a connection
from the pool. This may have occurred because all pooled connections were in
use and max pool size was reached.

Now here's the weird part. We have a 11 servers running running this web
application at RackSpace, it's made up of 4x frontend servers running MS
Server 2003 Standard R2 x64bit SP2 that's being load balanced by a hardware
LB. The web application is running on .Net 1.1.4322.2407. Then there is 7x
backend boxes running MS Server 2003 Enterprise R2 x64bit SP2. We host
around 2,0000 customer DB's in this environment. All these servers are all
Dual Proc Dual core, 8GB of ram etc.

Now we have a customer that hosts their own environment. It's in a windows
domain environment and they have a single frontend box, and a single backend
box. Their frontend server is MS Server 2003 Standard x64 SP2, and the web
application is running on .Net 1.1.4322.2407 as well. Their backend server
is running on Windows Server 2003 Enterprise Edition SP2 32bit, with only 2GB
of RAM and only a single customer database. Their database is only around
6.3GB

Now the actual weird part is, they get that error and it basically takes
down the web application, and they're using the exact same code as us. We're
hosting tons more db's and a lot more users obviously. I've gone through and
looked at about everything I can think of and I still can't put my finger on
it. Our Dev's have gone through the code but they can't find any type of
connection leak. We're using the default Max Pool Size at both locations as
well.

So I'm wondering if anyone has any pointers on where else we should be
looking.

 

[Stephany Young]

Well, after all that typing you ommitted the critical piece of information.

A connection to what?

 

[John Staggs]

Hey Stephany,

I'm not sure I understand your question, but we're using ADO to hit the
backend, and here's the exact error we get.

Timeout expired. The timeout period elapsed prior to obtaining a
connection from the pool. This may have occurred because all pooled
connections were in use and max pool size was reached.


Other things I noticed today was we do around 3GB - 3.5GB in raw IIS logs
daily, while this hosted customer does 300mb. Their backend box while being
32bit still only has 2GB...so I was wondering if that could play any part.
We have 8GB of memory and we're running 64bit OS and MSSQL on our boxes.

 

[Stephany Young]

OK. So the data access layer is ADO, but connecting to what?

Oracle?

MySql?

SQL Server?

What are thier current settings for connection pool size and connection
timeout.

Is this happening on every connection to the 'database' in the application
or just some?

 

[Dan Kelley]

Its near impossible to say with any certainty, although the most obvious
reason for this exception would be instances where connections to the DB are
not being closed and so released back into the pool.

Although the same code is being run, there may be a scenario where an
exception is being thrown on the customers system that isn't thrown on your
companies system.

Can you say with confidence that all connections are closed when finished
with, even in the event of an exception? i.e. are you using connections
either in a using block or a try/finally block?

Dan

 

[John Staggs]

Hey Stephany,

Sorry It was a long day yesterday, we're connecting to MSSQL Server 2k5.
On our production and the customers environment we're using the default
settings for the max connection pool size, which is 100 by default.

As far as the ConnectionTimeout we're using default as well, which is 15
secs. - I'm not sure if your meaning it's a lifespan timeout, which it's not.

When the problem starts happening every person that is logged into the web
application starts generating those errors (the timeout error). No new
connections are allowed to be made, and any existing ones creep to a stop.
Once IIS 6.0 is reset it runs fine again.

 

[John Staggs]

Hey Dan,

We cannot say with certainty that all connections are being closed for every
request, thats part of the problem. The application is so large that it's
impossible to say with certainty. But we are investigating areas where that
may be a problem.

We don't know of any peice of the web application that the remote customer
has and uses that nobody else users on our production either.

 

[John Staggs]

Does anyone know if it's normal when you run exec sp_who2 and you have a
process that last did something (LastBatch) nearly 24 hours ago? I have a
good handful from yesterday that are sleeping, awaiting commands.

I'm sitting around 88 total .net sqlclient data provider connections on
their server. It seems like it's adding more instead of using the existing
ones that are sleeping....and I figured GC would have came cleaned them up by
now.