It's based on the browserCaps, which you can override in the web.config.
Otherwise it uses the built-in one found in the machine.config file
(C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\CONFIG)
If you do a search for "browserCaps" and look for "crawler" in that section,
you'll see what sets it off.
Obviously it can easily be fooled, anyone can simply change the header
information. I think for the most part however, you won't get false
positives (Crawler == true when it isn't), but you will get false negatives
(crawler == false when it is).
Follow up - how can I use, or access, "Request.Browser.Crawler" in a classical ASP page?
I need to detect if a browser is a crawler so that I do not update the page count. Of
course, because of the nature of crawler beasties, I can not do a redirect. And the
application is too extensive for me to do a rewrite at this time.
The only way I know of is to either (a) parse the header itself, or (b) use
a 3rd party component like BrowserHawk
(https://www.cyscape.com/products/bhawk/)
Want to reply to this thread or ask your own question?
You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.
