Very anoying problem, please help (asp.net, IIS, regex)

[Tomislav Fistric]

Hi all.

I have very uncommon problem. I am running
IIS 6.0 on 2xXEON dual core 3.06 (8 virtual CPUs) with
4 gb of Ecc ram and i get system.outofmemory exception
on asp.net website. Thing is that i have big load on
site (site is a crawler that visits 500k pages daily).
I have optimized the code to take 5-10% of CPU usage
max but there is a problem with regex and memory.
Sometimes regex eats 600mb of memory and then iis
web worker crashes and starts spitting that error
(despite it has at least 1.4 gb more memory free???).
Is there any way to limit regex class memory usage?
I have made workaround in a way to use 5 web workers
instead of one (web garden) it works so far but
consumes too much of cpu (10-15%), while it is still
low i am expecting to raise daily visit by 4 times
so it will be alot then. Any suggestions?

Error i get in event log:

EventType clr20r3, P1 w3wp.exe, P2 6.0.3790.1830, P3 42435be1, P4
mscorlib, P5 2.0.0.0, P6 4333ab80, P7 116e, P8 29, P9
system.outofmemoryexception, P10 NIL.

BTW i have ms windows server ee 2003 and /3gb switch on.

 

[natemrice]

Hi all.

I have very uncommon problem. I am running
IIS 6.0 on 2xXEON dual core 3.06 (8 virtual CPUs) with
4 gb of Ecc ram and i get system.outofmemory exception
on asp.net website. Thing is that i have big load on
site (site is a crawler that visits 500k pages daily).
I have optimized the code to take 5-10% of CPU usage
max but there is a problem with regex and memory.
Sometimes regex eats 600mb of memory and then iis
web worker crashes and starts spitting that error
(despite it has at least 1.4 gb more memory free???).
Is there any way to limit regex class memory usage?
I have made workaround in a way to use 5 web workers
instead of one (web garden) it works so far but
consumes too much of cpu (10-15%), while it is still
low i am expecting to raise daily visit by 4 times
so it will be alot then. Any suggestions?

Error i get in event log:

EventType clr20r3, P1 w3wp.exe, P2 6.0.3790.1830, P3 42435be1, P4
mscorlib, P5 2.0.0.0, P6 4333ab80, P7 116e, P8 29, P9
system.outofmemoryexception, P10 NIL.

BTW i have ms windows server ee 2003 and /3gb switch on.

Have you tried switching back to a 2/2 kernel/user configuration? It's
hard to say for sure but using only 600mb would lead me to believe
that there is a configuration problem with .NET. machine.config may
have an improperly set value. You can also configure the worker
process to recycle more often but that would only mask the problem. If
you think you have a leak in your application, taking a memory dump
around the time your application usually hangs can reveal more
detail. Web gardens are a good option but you're still limited to only
2GB of user mode memory.

Ultimate fix: 64bit processing.

-Nate
http://www.naterice.com

 

[Tomislav Fistric]

Have you tried switching back to a 2/2 kernel/user configuration? It's
hard to say for sure but using only 600mb would lead me to believe
that there is a configuration problem with .NET. machine.config may
have an improperly set value. You can also configure the worker
process to recycle more often but that would only mask the problem. If
you think you have a leak in your application, taking a memory dump
around the time your application usually hangs can reveal more
detail. Web gardens are a good option but you're still limited to only
2GB of user mode memory.

Ultimate fix: 64bit processing.

-Nate
http://www.naterice.com

I think i have found my problem. It seems that IIS can't handle alot
of caching callbacks so it crashes on certain amount.