Performance Problems with Probing

[Jediah L.]

1. In our application which uses NTLM; .Net always requests a file as
Anonymous before requesting it using the NTLM authenticated credentials - in
an application with over a million hits a day - this is quite a bit of extra
posting going on. Anyone know how to stop this?

2. Even when supplying a probing hint in the web.config file; our .Net Win
Form embeded user controls probe 6 times before finding the file (the first
4 are variations of the file with .dll, and .exe names in /bin and /root
folders, the last 2 are ANON then NTLM requests for the correct file).

So for each of the 3000+ users - for each file requested - there is
approximatly 6 web requests handled on the server - this causes slowless in
the client application - and causes additional work for the servers.

3. Just recently all of our Smart Downloads have stopped working. I have
put a sniffer on the machines - and checked the IIS logs - the CLR has
stopped doing conditional gets (No If-Modified-Since headers) and IIS (of
course) is always responding by sending a 200 (after the initial 6 probes)
and resending the files each time - instead of responding w/ the 304's and
allowing the .Net download cache to do its work.

I've already went through a long list of trying things - however - i'm
running out of ideas - i'd appreciate any thoughts!!!

Thanks!

 

[Jerry Pisk]

1. You can't. NTLM (and pretty much every authentication scheme except
basic) requires at least two requests.

2. and 3. - Don't know...

Jerry

 

[Jediah L.]

In re #1:

Don't really understand why though - its a poor setup - and should be
configurable.

If you know NTLM was used for the first request - why not automatically use
it for the second request - and if its used for the first half a million
requests - why not use it for the next half a million.

Or why couldn't you tell the IE configuration [This site is always NTLM].

Seems like a waste in performance and bandwidth.

 

[Jerry Pisk]

IE does send two requests as well. And if you kept the connection open you
could reuse it without having to send two requests all the time, so - you're
right, you have a poor setup in your application. You should have a
connection pool that you'll use for your requests, instead of opening and
closing a connection each time.

Jerry

In re #1:

Don't really understand why though - its a poor setup - and should be
configurable.

If you know NTLM was used for the first request - why not automatically
use
it for the second request - and if its used for the first half a million
requests - why not use it for the next half a million.

Or why couldn't you tell the IE configuration [This site is always NTLM].

Seems like a waste in performance and bandwidth.

1. You can't. NTLM (and pretty much every authentication scheme except
basic) requires at least two requests.

2. and 3. - Don't know...

Jerry

 

[Jediah L.]

So you would suggest a better setup is to open a single HTTP connection and
keep it open for the 10 hour working day?

I wonder why IE doesn't do the same thing if it is such a great way to do
it? (In our application, we both use IE - with embeded Win Form User
Controls - and we use .Net Win Form .Exe's both exhibit the same problem).

But seriously - I appreciate your response - and didn't really expect you or
anyone else to have an answer to #1 - its a flaw imo in the way microsoft
has implemented their authentication scheme.

Any thoughts on #2 and #3 would also be greatly appreciated!

Thanks!

IE does send two requests as well. And if you kept the connection open you
could reuse it without having to send two requests all the time, so - you're
right, you have a poor setup in your application. You should have a
connection pool that you'll use for your requests, instead of opening and
closing a connection each time.

Jerry

In re #1:

Don't really understand why though - its a poor setup - and should be
configurable.

If you know NTLM was used for the first request - why not automatically
use
it for the second request - and if its used for the first half a million
requests - why not use it for the next half a million.

Or why couldn't you tell the IE configuration [This site is always NTLM].

Seems like a waste in performance and bandwidth.

1. You can't. NTLM (and pretty much every authentication scheme except
basic) requires at least two requests.

2. and 3. - Don't know...

Jerry

1. In our application which uses NTLM; .Net always requests a file as
Anonymous before requesting it using the NTLM authenticated credentials -
in
an application with over a million hits a day - this is quite a bit of
extra
posting going on. Anyone know how to stop this?

2. Even when supplying a probing hint in the web.config file; our

..Net
Win

Form embeded user controls probe 6 times before finding the file (the
first
4 are variations of the file with .dll, and .exe names in /bin and
/root
folders, the last 2 are ANON then NTLM requests for the correct file).

So for each of the 3000+ users - for each file requested - there is
approximatly 6 web requests handled on the server - this causes
slowless
in
the client application - and causes additional work for the servers.

3. Just recently all of our Smart Downloads have stopped working. I have
put a sniffer on the machines - and checked the IIS logs - the CLR has
stopped doing conditional gets (No If-Modified-Since headers) and IIS (of
course) is always responding by sending a 200 (after the initial 6 probes)
and resending the files each time - instead of responding w/ the

304's
and

allowing the .Net download cache to do its work.

I've already went through a long list of trying things - however - i'm
running out of ideas - i'd appreciate any thoughts!!!

Thanks!

 

[Jerry Pisk]

That's exactly what I suggest. It's not a flaw, it's just the way all secure
authentication schemes work (so don't blame Microsoft, it's not just NTLM).

Sorry I can't help you with the rest.

Jerry

So you would suggest a better setup is to open a single HTTP connection
and
keep it open for the 10 hour working day?

I wonder why IE doesn't do the same thing if it is such a great way to do
it? (In our application, we both use IE - with embeded Win Form User
Controls - and we use .Net Win Form .Exe's both exhibit the same problem).

But seriously - I appreciate your response - and didn't really expect you
or
anyone else to have an answer to #1 - its a flaw imo in the way microsoft
has implemented their authentication scheme.

Any thoughts on #2 and #3 would also be greatly appreciated!

Thanks!

IE does send two requests as well. And if you kept the connection open
you
could reuse it without having to send two requests all the time, so - you're
right, you have a poor setup in your application. You should have a
connection pool that you'll use for your requests, instead of opening and
closing a connection each time.

Jerry

In re #1:

Don't really understand why though - its a poor setup - and should be
configurable.

If you know NTLM was used for the first request - why not automatically
use
it for the second request - and if its used for the first half a
million
requests - why not use it for the next half a million.

Or why couldn't you tell the IE configuration [This site is always NTLM].

Seems like a waste in performance and bandwidth.


1. You can't. NTLM (and pretty much every authentication scheme except
basic) requires at least two requests.

2. and 3. - Don't know...

Jerry

1. In our application which uses NTLM; .Net always requests a file as
Anonymous before requesting it using the NTLM authenticated
credentials -
in
an application with over a million hits a day - this is quite a bit of
extra
posting going on. Anyone know how to stop this?

2. Even when supplying a probing hint in the web.config file; our .Net
Win
Form embeded user controls probe 6 times before finding the file
(the
first
4 are variations of the file with .dll, and .exe names in /bin and
/root
folders, the last 2 are ANON then NTLM requests for the correct file).

So for each of the 3000+ users - for each file requested - there is
approximatly 6 web requests handled on the server - this causes
slowless
in
the client application - and causes additional work for the servers.

3. Just recently all of our Smart Downloads have stopped working.
I
have
put a sniffer on the machines - and checked the IIS logs - the CLR has
stopped doing conditional gets (No If-Modified-Since headers) and
IIS
(of
course) is always responding by sending a 200 (after the initial 6
probes)
and resending the files each time - instead of responding w/ the 304's
and
allowing the .Net download cache to do its work.

I've already went through a long list of trying things - however - i'm
running out of ideas - i'd appreciate any thoughts!!!

Thanks!

 

[Steven Cheng[MSFT]]

Hi Jediah,

Regarding on the issue, I am
finding proper resource to assist you and we will update as soon as
posible. Thanks.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security(This posting is provided "AS IS",
with no warranties, and confers no rights.)

 

[Jeffrey Tan[MSFT]]

Hi Jediah,

Sorry for letting you wait for so long time.
=====================================
1. Based on my understanding, your #1 requirement is disabling the
anonymous authentication to improve your Asp.net application performance.

First, this requirement can only be done at server side, because you can
not force all the client customer browser(IE) to do anything. But again,
this http authentication request can only be controlled at client IE side,
because IE always send anonymous request, then with windows account
information. This indicates that this requirement can not be controlled.(As
I originally said, you can not control million client's browser setting)
Second, the key performance issue of Asp.net Application is server side
code parsing, Html code rendering and Html code client parsing, but not the
HTTP request, so you may pay more attention to improve the application
performance. For more information about improving Asp.net performance,
please refer to:
"Improving ASP.NET Performance"
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag/html/
scalenetchapt06.asp
Third, NTLM authentication is a complex authentication, which involves
several HTTP request and authentication information exchange. And the first
anonymous http request time can be ignored comparing with the NTLM
authentication.

2. I think it is the same as the first question

3. First, you may check IIS setting's "HTTP headers" tabpage, if you set
"Enable Content Expiration" with "Expiration Immediately".
Second, using the siffer, you may check, if the IIS reponse has "no-cache"
http header.
Third, I think the problem may be on your specific IE side. You may do some
test to confirm this:
Does your problem machine request other server pages?
Does other machine's IE also has this problem?

==================================================
Thank you for your patience and cooperation. If you have any questions or
concerns, please feel free to post it in the group. I am standing by to be
of assistance.

Best regards,
Jeffrey Tan
Microsoft Online Partner Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

 

[Jediah L.]

Thanks Jeff.

In our case - we have approximatly 3,000 users - but they are all intranet
users - so we could force IE settings if it would help.

What we are finding to be the more specific problem is this.

When We have a method (executing on the client) in javascript inside of an
HTML page - that requests to download 10 XML files - it will attempt the
first as Anonymous - and the other 9 as authenticated (I may have stated
otherwise in a previous email - if so I apologize).

In our .Net smart client - the connections are stateless - so we open and
close a connection for each request - so when we request 10 XML files - we
end up trying ANON and then NTLM AUTH for each of the ten files - thus
making 20+ HTTP requests for the 10 requested files.

It would be nice if we could tell our Web Request object - don't bother
trying ANON - use the Credentials we supplied (We do supply the credentials
too).



The Question #2 I believe had more to do with the additional probing that
..Net does when using embeded Windows Form User Controls in IE. We have
found that even though we specify a probing path (/assemblies in our
example) - it always goes through a series of probes first no matter what -
so for a file named Search.dll it looks for this:

/Search.dll
/Search/Search.dll
/bin/Search.dll
/bin/Search/Search.dll
/assemblies/Search.dll - receives 401 as it was Anon
/assemblies/Search.dll - 200 / or 304

How can we force it to stop probing needlessly - besides moving all of our
assemblies into the root folder?


The Question #3 we have been expericing this issue very sporadically - one
minute it will cache - one minute it will not.

I've found when it is not caching - the client is not setting the
If-Modified-Since headers - although I can't figure out why. In our case -
the files haven't been modified in almost a week; and previous 200 responses
do return the Last-Modified header.

The Expiry headers are not set at all - and we continue to get sporadic
responses of caching and no caching.


Any other information / suggestions you can provide would be great.

Thanks!

Hi Jediah,

Sorry for letting you wait for so long time.
=====================================
1. Based on my understanding, your #1 requirement is disabling the
anonymous authentication to improve your Asp.net application performance.

First, this requirement can only be done at server side, because you can
not force all the client customer browser(IE) to do anything. But again,
this http authentication request can only be controlled at client IE side,
because IE always send anonymous request, then with windows account
information. This indicates that this requirement can not be
controlled.(As
I originally said, you can not control million client's browser setting)
Second, the key performance issue of Asp.net Application is server side
code parsing, Html code rendering and Html code client parsing, but not
the
HTTP request, so you may pay more attention to improve the application
performance. For more information about improving Asp.net performance,
please refer to:
"Improving ASP.NET Performance"
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag/html/
scalenetchapt06.asp
Third, NTLM authentication is a complex authentication, which involves
several HTTP request and authentication information exchange. And the
first
anonymous http request time can be ignored comparing with the NTLM
authentication.

2. I think it is the same as the first question

3. First, you may check IIS setting's "HTTP headers" tabpage, if you set
"Enable Content Expiration" with "Expiration Immediately".
Second, using the siffer, you may check, if the IIS reponse has "no-cache"
http header.
Third, I think the problem may be on your specific IE side. You may do
some
test to confirm this:
Does your problem machine request other server pages?
Does other machine's IE also has this problem?

==================================================
Thank you for your patience and cooperation. If you have any questions or
concerns, please feel free to post it in the group. I am standing by to be
of assistance.

Best regards,
Jeffrey Tan
Microsoft Online Partner Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

 

[Jeffrey Tan[MSFT]]

Hi Jediah,

Thanks for your feedback.

#1. After confirming, it seems that there is no option for IE to send
crendential at the first request. That is, IE always uses the Anonymous
first.
Also, just as I original stated, NTLM authentication will involve several
HTTP request to be authenticated, while Anonymous requests only once, so it
will have much concern on the performance. And the other periods such as
page execute, database operation, html render out, are much longer than the
IIS authetication. So the bottleneck is not at this point. Although we are
making 20+ HTTP requests for the 10 requested files. But the time for this
can be ignored comparing with other periods. The number of HTTP requests
does not stand for long time for processing the request.
Hope I have explained it clear

#2. Sorry, for this issue, can you tell me the issue context? That is, when
the client search for "Search.dll" in such order? What is "Search.dll" in
your issue context?
Thanks very much.

#3. Also, for this issue, does this sporadically issue occurs on only one
machine or all the client machines have this problem? If all the client
machines have this problem, the problem should be the server side(may be
IIS or application). Else, the problem should only be your certain machine.

======================================
Thank you for your patience and cooperation. If you have any questions or
concerns, please feel free to post it in the group. I am standing by to be
of assistance.

Best regards,
Jeffrey Tan
Microsoft Online Partner Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

 

[Jediah L.]

#1 The problem here is really the fact that we open and close connections
(stateless application) for each Web Request from .Net. It's really over
150 files from over 3000 users that are being multipled x2 on the AUTH/NTLM
requests.

This is what the application does:

<Client> Hey I want an XML File
<CommunicationClass>[open connection][ANON] IIS I want an XML File
<IIS>[401]I'm sorry you're not authenticated
<CommunicationClass>Oh yeah, I have credentials - do you want them
<IIS>[200/304]Ok thanks, here is the file!
<CommunicationClass>[close connection]
<Client> Hey I want an XML File
<CommunicationClass>[open connection][ANON] IIS I want an XML File
<IIS>[401]I'm sorry you're not authenticated
<CommunicationClass>Oh yeah, I have credentials - do you want them
<IIS>[200/304]Ok thanks, here is the file!
<CommunicationClass>[close connection]

ETC...

Theoretically - we could open the HTTP channel and leave it open for good -
so as to always be authenticated - but that seems to take away the whole
idea of a stateless design... and I'm sure it would time out eventually...

What would be nice is if we could tell .Net to do this:

<Client> Hey I want an XML File
<CommunicationClass>[open connection][NTLM] IIS I want an XML File
<IIS>[200/304]Ok thanks, here is the file!
<CommunicationClass>[close connection]

It would cut down on alot of HTTP chatter. Does this make sense on what I'm
asking about?

#2

We have an HTML file which has an embeded DLL called Main.dll
Search.dll is a DLL referenced in Main.dll that is being probed for by the
CLR (because Main.dll is loaded up with the object tag).
We have specified a private probing hint path in our web.config
(/assemblies) and looking at logs from fusion we see the hint path is
recognized
however, the CLR always goes through a series of probes <no matter what> to
attempt to find our Search.dll before it uses our hint path

So, instead of the CLR just doing this

<CLR>Oh, you have a hint path, let me probe in that path for your DLL

It does this

<CLR> is the file here: /Search.dll
<CLR> is the file here: /Search/Search.dll
<CLR> is the file here: /bin/Search.dll
<CLR> is the file here: /bin/Search/Search.dll
<CLR> I still can't find it - maybe I should use the private hint path
<CLR> is the file here: /assemblies/Search.dll - receives 401 as it was Anon
<CLR> is the file here: /assemblies/Search.dll - 200 / or 304

#3 We've had this problem on multiple machines. We've also tried clearning
out our download cache and IE cache. When caching is not working - the
client is not sending conditional gets (IF-MODIFIED-SINCE).
We haven't figured out why the clients arent requesting caching -
(sporadically). We're still looking through this problem - we've already
tried quite a few different things - but any thoughts would be appreciated.

The scenario is similar. We have an EXE that runs off the web site. The
EXE uses Main.dll and Search.dll. We can run the EXE consecutively and
sometimes Main.dll/Search.dll is cached, and sometimes it is not.

I have packet traces showing requesting these DLL's three times in a row and
each time we receive a 200 (and i've double checked and IIS is not setting
the NO-CACHE header).



Thanks again!

Hi Jediah,

Thanks for your feedback.

#1. After confirming, it seems that there is no option for IE to send
crendential at the first request. That is, IE always uses the Anonymous
first.
Also, just as I original stated, NTLM authentication will involve several
HTTP request to be authenticated, while Anonymous requests only once, so
it
will have much concern on the performance. And the other periods such as
page execute, database operation, html render out, are much longer than
the
IIS authetication. So the bottleneck is not at this point. Although we are
making 20+ HTTP requests for the 10 requested files. But the time for this
can be ignored comparing with other periods. The number of HTTP requests
does not stand for long time for processing the request.
Hope I have explained it clear

#2. Sorry, for this issue, can you tell me the issue context? That is,
when
the client search for "Search.dll" in such order? What is "Search.dll" in
your issue context?
Thanks very much.

#3. Also, for this issue, does this sporadically issue occurs on only one
machine or all the client machines have this problem? If all the client
machines have this problem, the problem should be the server side(may be
IIS or application). Else, the problem should only be your certain
machine.

======================================
Thank you for your patience and cooperation. If you have any questions or
concerns, please feel free to post it in the group. I am standing by to be
of assistance.

Best regards,
Jeffrey Tan
Microsoft Online Partner Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

 

[Jeffrey Tan[MSFT]]

Hi Jediah,

Thanks for your feedback.

#1, I see your concern. But this is just the HTTP reqeust order, what we
really should concern is the totally wasted time. In this senario, the
failed anonymous HTTP request time is very few COMPARING to the entire
request, authentication, page parsing and response etc time. So what we
should improve is in the Asp.net code processing period, this is the
bottleneck.

Also, what you required also makes some sense, if you still have concern on
this issue, you may provide your suggestion to Microsoft at:
http://register.microsoft.com/mswish/suggestion.asp
or mailto: (e-mail address removed)

#2. I will spend a little more time on this issue.

#3. I suggest you explicitly specify a long expire time in IIS(such as 1
year), then if the problem still existed.
Open IIS's site property dialog, swich to HTTP Headers tabpage, click
"Enable Content Expiration", set 100 day(s) in the below textbox.
You need clear all the client IE's cache to enable this IIS setting. To
clear IE cache, open "Internet Options" dialog, click "Deletes files".

Thank you for your patience and cooperation. If you have any questions or
concerns, please feel free to post it in the group. I am standing by to be
of assistance.

Best regards,
Jeffrey Tan
Microsoft Online Partner Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

 

[Jediah L.]

By the way, thought I'd pass this along, if anyone is ever looking &
intrested.

There was a way to do #1 - surprisingly - using the
UnsafeAuthenticatedConnectionSharing property of the Web Request object.

I have some outstanding questions about the stability and any performance
implications - however - it does exactly what I was looking for.

 

[Jediah L.]

Thanks Jeff -

I just posted (higher up on the thread) that we actually found a resolution
to #1, in the form of the UnsafeAuthenticatedConnectionSharing property of
the web request object.

I do still have some concerns regarding the stability, reliability and the
performance implications of what is going on behind the scenes using this
property - i have requested additional information from our ADC.

If you could come up with any suggestions for #2 that would be great. What
we currently have in mind is moving ALL assemblies (currently about 150 of
them) to the root folder - that way the first probe will be the correct
probe. But for maintenance purposes - this is obviously not the best..

#3 still seems that the client is not requesting caching - but it appears to
be a bigger issue than that. While using a debug version of wininet.dll -
at the very end of the log file - it appears that the client is deleting
everything out of the cache (haven't figured out why yet).

When/If we find a resolution to #3 - i'll also post that. I appreciate your
willingness to help out!

Thanks!

 

[Jeffrey Tan[MSFT]]

Hi Jediah,

Thanks for your feedback.

Yes, I have received your reply of using
UnsafeAuthenticatedConnectionSharing property. Yes, this property may
bypass the Anonymous authentication issue, but this way means that you need
to write a HTTP client yourself to substitute IE, you also need to add the
HTML-parsing etc IE function, which will cost a lot of time. Also, many
functions of IE have been optimized, while you new-invented these functions
may not have such high performance, and these performance cost will be
greater than the Anonymous authentication cost. At last, you have to
substitute all the 3000 users' IE browser with the new one.

For the not-cache issue, I suggest you create a new smart client
application on your server, will this one have the same not-cache issue?
This information will help us to isolate the issue with your application.

Thank you for your patience and cooperation. If you have any questions or
concerns, please feel free to post it in the group. I am standing by to be
of assistance.

Best regards,
Jeffrey Tan
Microsoft Online Partner Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

 

[Jeffrey Tan[MSFT]]

Hi Jediah,

Sorry for letting you wait for so long time.

For the #2, this is the default behavior of .Net Framework. Actually,
<probing> element is not a way to suppress CLR's default searching order.
It is just an additionally way of adding a probing option for the CLR. That
is, if the CLR's default searching fails, CLR will use the private path
specified in the <probing> element to look for.

This information is documented in the below 2 articles:
"How the Runtime Locates Assemblies"
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguide/htm
l/cpconhowruntimelocatesassemblies.asp
and
"Step 4: Locating the Assembly through Codebases or Probing"
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguide/htm
l/cpconstep4locatingassemblythroughcodebasesorprobing.asp

Because you did not specify the culture information in app.config, CLR will
search like this:
<CLR> is the file here: /Search.dll
<CLR> is the file here: /Search/Search.dll
<CLR> is the file here: /bin/Search.dll
<CLR> is the file here: /bin/Search/Search.dll
<CLR> I still can't find it - maybe I should use the private hint path
<CLR> is the file here: /assemblies/Search.dll - receives 401 as it was Anon
<CLR> is the file here: /assemblies/Search.dll - 200 / or 304

Also, is your no-cache issue resolved through creating a new smart-client
application?

Thank you for your patience and cooperation. If you have any questions or
concerns, please feel free to post it in the group. I am standing by to be
of assistance.

Best regards,
Jeffrey Tan
Microsoft Online Partner Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

 

[Jeffrey Tan[MSFT]]

Hi Jediah,

Does my reply make sense to you? Do you still have concern on this issue?

Please feel free to feedback. Thanks

Best regards,
Jeffrey Tan
Microsoft Online Partner Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.