Set user agent header with webrequest

[Jon Spivey]

Hi,

I'm using webrequest to scrape some urls, works fine, however one url seems
to be throwing an error when it doesn't recognise the user agent. So I need
to set the user agent to something like MSIE, I've tried these
WebRequest.Headers.Set("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0;
Windows NT 5.1")
WebRequest.Headers.Add("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0;
Windows NT 5.1")
WebRequest.Headers.Item("User-Agent") = "Mozilla/4.0 (compatible; MSIE 6.0;
Windows NT 5.1; )")
and they all give the same error "System.ArgumentException: This header must
be modified with the appropriate property"

What's the correct way to set the User Agent header?

Cheers,
Jon

 

[jongalloway]

Trick question? HttpWebRequest.UserAgent

http://msdn.microsoft.com/library/d...ystemnethttpwebrequestclassuseragenttopic.asp
- Jon
http://weblogs.asp.net/jgalloway