Distinguishing Crawlers

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

Hello,

Does anyone happen to have a Browsercaps update that properly sets the
Crawler attribute? I am seeing that google and others are being recognized as
browsers and not crawlers.

Thank you.
 
Ian said:
Hello,

Does anyone happen to have a Browsercaps update that properly sets the
Crawler attribute? I am seeing that google and others are being recognized as
browsers and not crawlers.

Thank you.
Click to expand...

recognized by what?
The browscap is an outdated tech.... may want to look elsewhere
 
The machine.config browsercaps section sets the HttpBrowserCapabilities
object properties correct?

I need to know when a search engine is crawling my site so I know not to
prompt it to select a country for pricing. If there is something other than
using the Browser.Crawler property I would love to hear about it.

Thanks!
 
You might want to try adding this meta tag
to the pages you don't want crawlers/robots to index :

<META NAME="ROBOTS" CONTENT="NOINDEX">




Juan T. Llibre
ASP.NET MVP
http://asp.net.do/foros/
Foros de ASP.NET en Español
Ven, y hablemos de ASP.NET...
======================
 
Crawlers have more legs. If they have MANY more legs, they are called
"millipedes."

--
;-),

Kevin Spencer
Microsoft MVP
..Net Developer
Ambiguity has a certain quality to it.
 
Back
Top