HttpWebRequest

[Giles Evans]

I've been struggling with this for too long now, perhaps someone here could
help. My ultimate goal is to programically retreive the Flood Insurance
Rate Map (FIRM) information for the properties of our clients for inclusion
in our on-line listing database. I can't seem to get the Post request right
which just returns the original form rather than the page with the FIRM
information. Can anyone help? My code follows..

Imports System.Net
Imports System.IO
Partial Class _Default
Inherits System.Web.UI.Page

Protected Sub Page_Load(ByVal sender As Object, ByVal e As
System.EventArgs) Handles Me.Load
'Set uri to URL containing form
Dim uri As New
Uri("http://msc.fema.gov/webapp/wcs/stor...eView?storeId=10001&catalogId=10001&langId=-1")

'Define Parameters
Dim userType As String = "G"
Dim productType As String = "1"
Dim street As String = "77 Blueberry Lane" 'tried replacing all
spaces with + do I need to or will the encoding do this?
Dim city As String = "Westwood"
Dim state As String = "MA"
Dim zip As String = "02090"
Dim button As String = "Search by Street Address"
Dim cookies As New CookieContainer

'create string to post
Dim Data As String = "userType=" + userType _
+ "&productType=" + productType _
+ "&address=" + street _
+ "&city=" + city _
+ "&state=" + state _
+ "&zipcode=" + zip _
+ "&productSearchButton1=" +
button

If (uri.Scheme = uri.UriSchemeHttp) Then

'Populate Cookie Container
Dim webRequest As HttpWebRequest = HttpWebRequest.Create(uri)
webRequest.Method = WebRequestMethods.Http.Get
webRequest.CookieContainer = cookies
webRequest.Timeout = 30000
webRequest.GetResponse().Close()

'Open connection to Post
Dim request As HttpWebRequest = HttpWebRequest.Create(uri)

request.Accept = "image/gif, image/jpeg, image/pjpeg,
application/x-ms-application, application/vnd.ms-xpsdocument,
application/xaml+xml, application/x-ms-xbap, application/vnd.ms-excel,
application/vnd.ms-powerpoint, application/msword,
application/x-shockwave-flash, */*"
request.Referer =
"http://msc.fema.gov/webapp/wcs/stor...eView?storeId=10001&catalogId=10001&langId=-1"

request.UserAgent = "Mozilla/4.0 (compatible; MSIE 8.0; Windows
NT 6.0; WOW64; Trident/4.0; GTB6; SLCC1; .NET CLR 2.0.50727; Media Center PC
5.0; OfficeLiveConnector.1.4; OfficeLivePatch.0.0; .NET CLR 3.5.30729; .NET
CLR 3.0.30729; InfoPath.2)"
request.ContentType = "application/x-www-form-urlencoded"

request.ContentLength = Data.Length
request.KeepAlive = True
request.CookieContainer = cookies
request.AllowAutoRedirect = True
request.Method = WebRequestMethods.Http.Post

'Post Data
Dim writer As New StreamWriter(request.GetRequestStream())
writer.Write(Data)
writer.Close()

'Get Response
Dim response As HttpWebResponse = request.GetResponse()
Dim reader As New StreamReader(response.GetResponseStream())
Dim tmp As String = reader.ReadToEnd()
If Not response Is Nothing Then
response.Close()

'Show response in main page
Me.panel1.GroupingText = tmp
End If
End If
End Sub
End Class

 

[Giles Evans]

I've solved it...

It appears that CookieContainer doesn't handle cookies for domains as it
should. Example:

Say I receive a cookie for domain ".someDomain.com". This is a valid domain
(according to RFC 2109 ), which represents all subdomains for
"someDomain.com" along with "someDomain.com" itself. Unfortunately the
CookieContainer doesn't seem to properly handle such cookies that begin with
a dot, and when I add this cookie to a CookieContainer and send out an
HttpWebRequest, the cookie never goes out since the request's address URI
will never exactly be ".someDomain.com".