The underlying connection was closed: An unexpected error occurred

[Guest]

Receiving error "The underlying connection was closed: An unexpected error
occurred on a receive." when requesting a webpage from VB.NET. Here is the
simple code:

Dim strStartPage As String 'URL of the requested page
Dim reqMain As HttpWebRequest
Dim resMain As HttpWebResponse

strStartPage = "http://www.smartpages.com"
reqMain = System.Net.HttpWebRequest.Create(strStartPage)
resMain = reqMain.GetResponse()

This code works if I request "http://www.google.com" or "http://www.msn.com"
so why does it not work for "http://www.smartpages.com"? Error is received
on the GetResponse call.

I have SP1 installed for .NET Framework 1.1.

 

[Guest]

Ok...turns out I solved my own problem. Looks like smartpages.com is
querying the UserAgent field to find out the browser type and customizing the
returned html request based on the browser type where other sites like
google.com or msn.com are not. I solved the problem by adding the following
line before the GetResponse line:

reqMain.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;
SV1; .NET CLR 1.1.4322)"