Issue with httpwebrequest and redirect

[scott rosa]

I am having an issue with using the httpwebrequest to
perform a screenscrape. The site I am retrieving data
from has forms based authentication. When I pass the
correct credentials it authenticates properly, but then I
get an error saying too many redirections attempted. Here
is the code I am using:
--------------------------------------------------------
Private Sub Button1_Click(ByVal sender As
System.Object, ByVal e As System.EventArgs) Handles
Button1.Click
Dim x As String

x = readHtmlPage("http://awebsite.com")

End Sub

Function readHtmlPage(ByVal url As String) As String
Dim objResponse As WebResponse

Dim result As String



Dim myCred As New NetworkCredential("uname", "pw")
Dim myCache As New CredentialCache
myCache.Add(New Uri(url), "Basic", myCred)


Dim wr As HttpWebRequest = HttpWebRequest.Create
(url)
wr.Credentials = myCache
wr.AllowAutoRedirect = True
wr.MaximumAutomaticRedirections = 5



Dim myWebResponse As HttpWebResponse =
wr.GetResponse()

Dim sr As New StreamReader(myWebResponse.GetResponseStream
())
result = sr.ReadToEnd()

'clean up StreamReader
sr.Close()

Return result
End Function

----------------------------------------------

Can anyone assist?

Thanks,

Scott

 

[Rich Blum]

I am having an issue with using the httpwebrequest to
perform a screenscrape. The site I am retrieving data
from has forms based authentication. When I pass the
correct credentials it authenticates properly, but then I
get an error saying too many redirections attempted.

Scott -

Is there a reason you set the MaximumAutomaticRedirections
property to 5? The default is 50. Also, with forms based
authentication, the login credential is stored as a cookie. I don't
see where you capture cookies returned from the web site. Hope this
helps out some.

Rich Blum
Author of "C# Network Programming" (Sybex)
http://www.sybex.com/sybexbooks.nsf/Booklist/4176