HttpWebRequest & HttpWebResponse Cookies

[Cheung, Jeffrey Jing-Yen]

I have a windows form application that generates a request, downloads an image, and waits the user
to enter in login info. Unfortunately, this image is dynamic and based on session data. I have
read documents on the CookieCollection property of HttpWebRequest. Currently, I have the
functionality in my code to be able to accept cookies, and return them upon a new HttpWebRequest;
however, upon further inspection of the returning HttpWebResponse.Cookies property, I notice that
the cookies that are supposed to be there regarding my session data are not returned (one such
cookie named, JSESSIONID). What other methods are there available to me to retrieve cookies that
would contain session data?

Here is a code snippet (please excuse the wordwrap)

Dim strURL As String = "https://www.thisurlisatest.com/"
Dim objHTTPRequest As HttpWebRequest = CType(WebRequest.Create(strURL), HttpWebRequest)
objHTTPRequest.CookieContainer = New CookieContainer()
Dim objHTTPResponse As HttpWebResponse = CType(objHTTPRequest.GetResponse(), HttpWebResponse)
Dim objReceiveStream As Stream = objHTTPResponse.GetResponseStream()
' I successfully retrieve the cookies here
m_Cookies = objHTTPResponse.Cookies
Dim objCookie As Cookie
For Each objCookie In m_Cookies
Console.WriteLine(objCookie.Name + "=" + objCookie.Value)
Next

' This only outputs this: language_pref=en_US
' There should be more cookies that contain session id data

Thanks in advance for any light you can shed on this matter,

Jeff Cheung

 

[Cor]

Hi Cheung,

Do you have multiple webservers that you use the cookieCollection?

Otherwise you can have a look at the normal HttpRequest.cookie class

It is much easier to handle (if you dont forget to set the live time)

I hope this helps

Cor

 

[Cheung, Jeffrey Jing-Yen]

Cor,

I'm not passing the cookie collection to multiple webservers; however, this is a windows form
application. How do you propose that I get the System.Web.HttpRequest object in my form application?

I did find the solution to my problem; however, it brought upon a bigger issue for me unfortunately.
I ended up looking at the Set-Cookie item in the Headers property in the HttpWebResponse object,
and adding that value to the Cookie item in the Headers property in the HttpWebRequest object. This
works flawlessly as long as I do not pass a CookieCollection in the HttpWebRequest.

According to this link:

http://authors.aspalliance.com/damianm/article/4/5/httpWebRequest.aspx.vb.view.aspx

if I populate the CookieCollection, this will override the Cookie header in the request object.

So my problem now is that I'm able to successfully login and retain my session, but any subsequent
page request returns nothing in the ResponseStream from HttpWebResponse.GetResponseStream().
Looking at the HttpWebResponse.StatusCode yields 200, but the content-length is always 0 whenever I
make any requests. Any ideas?

Thanks in advance,

Jeff Cheung

 

[Cor]

Hi Cheung,

I never used that cookiecontainer because the other ones are in my opinion
more simple.

But my problem with cookies is always that I forget to set the livetime of a
cookie, it disapears immidiatly.

I did not see that in your code, maybe it is your problem also, so before I
start checking things that I never did before, can you check that yourself?

Cor

 

[Cheung, Jeffrey Jing-Yen]

Cor,

Thanks again for your response. According to the RFC, the Set-Cookie header in the response is
responsible for setting the "live time" of the cookie. I say this with the assumption that you are
talking about the expires attribute for the cookie. The server should be responsible for setting
the expiry date of the cookie; however, I did set it to expire a month explicitly when I read your
message just to test it out. Unfortunately, I still get 0 bytes returned... But I know my session
is still intact because the response doesn't return any new Set-Cookie headers. If my session had
reset, then there would be a new Set-Cookie header containing the Session ID.

Regards,

Jeff Cheung

 

[Cor]

Hi Jeff,

I become confused. I can not see how a windowform application can set a
cookie.

The cookie is for the connection from the webform and the user to check that
the right user answers (one of the reasons for a cookie).

Can you explain something more, because I think I am on the wrong track.

Cor

 

[Cheung, Jeffrey Jing-Yen]

Cor,

Sure thing... My windows form application has HttpWebRequest and HttpWebResponse objects. You are
correct, since it's not a webform, I have no need to use the Response.Cookies method. The
differences are:

ASP.NET application
- You are writing the cookies to the client's browser

My application
- I am writing the cookies to the HttpWebRequest object, since the HttpWebRequest is going to be
making the calls to the webserver.

This goes back to my original question. The documentation in MSDN directs me to use the
CookieCollection in order to maintain any cookies in my subsequent requests since the HttpWebRequest
is designed to only make one request to a URI. In essence, my application is more of a web browser
with the exception that I do not have to render any of the html that I parse. My goal is to
impersonate a user logging in to a predefined site, and automating administrative tasks. Let me
know if this clears things up.

Regards,

Jeff Cheung

 

[Cor]

Hi Jeff,

It soundsvery difficult, but when your program it is a kind of webbrowser,
it needs the cookies from the client.

As far as I know handles the classes you use the cookies from a client from
the side of the webserver.

But you can of course try to read the cookies from the client direct, I did
try that also but stopped, because I could not find an enumerating for the
cookie container and because that it can be on every computer (and specialy
every Microsoft OS) on another place I stopped with it.

But as far as I remember me the cookie is not that difficult file to read, I
thought just a kind of textfile.

Cor

Sure thing... My windows form application has HttpWebRequest and

HttpWebResponse objects. You are

correct, since it's not a webform, I have no need to use the Response.Cookies method. The
differences are:

ASP.NET application
- You are writing the cookies to the client's browser

My application
- I am writing the cookies to the HttpWebRequest object, since the HttpWebRequest is going to be
making the calls to the webserver.

This goes back to my original question. The documentation in MSDN directs me to use the
CookieCollection in order to maintain any cookies in my subsequent

requests since the HttpWebRequest

 

[Cheung, Jeffrey Jing-Yen]

Cor,

Thanks again for your response. I don't think that you fully understand my problem. I think that I
have the cookies functionality licked. I suspect the site that I am trying to connect to has
multiple redirects, so my code isn't accounting for those since the AllowAutoRedirect property is
set to true on my HttpWebRequest. I'll research this a little more because I just simply see more
cookies being written to my browser than in my application.

When you say that I can read the cookies from the client directly, you misunderstand my scenario.
It's not as simple as accessing the Request.Cookies collection. The "client" in this case is not
the user using the program. The "client" is the HttpWebRequest object. This class represents what
a normal browser would do - make web requests.

As far as the CookieContainer property, this is the "container" on the HttpWebRequest object. Think
of the HttpWebRequest object as a pseudo browser itself; it has its own method of storing cookies.
Since the CookieContainer property is just a collection of the object type "Coookie", enumerating
this property is very easy; however, my specific problem (you can try it out yourself in a Windows
Form application, you learn a lot about the HTTP protocol) is that the Cookies property on the
HttpWebResponse object doesn't return any cookies for the session data. They're just simply not
there. Because of this, I've had to resort to writing headers explicitly.

Regards,

Jeff Cheung

 

[Guest]

Ok, I hope this sounds basic

I'm able to set up a HttpWebRequst to go to a site, post my form data
successfully logon, and get the cookie passed

Now how do I navigate to the page I want to go to on this site
The address property of the WebRequest is read only
If I create a new WebRequest I lose my cookie container and properties, an
the website response says "you have been automatically logged off, pleas
log in"

Any Ideas
Thanks
Eri

 

[Eria Tlov]

Well, I got something to work.
I create an HttpWebRequest with a cookie container, copy the session ID out
to a string, and then write that string into the headers of subsequent
request. By subsequent request, I mean creation of new HttpWebRequest. I
still haven't figured out if you can navigate with one request. It seems to
navigate with autoredirects to a page away from the RequestURI, but not
manually.
thanks,
Eria

 

[Tony Archer]

Could you please post the code you are using to accomplish this? I'm trying
to do something very similar.

 

[Eria Tlov]

Dim myReq As HttpWebRequest = WebRequest.Create(myURL)

myReq.CookieContainer = New CookieContainer()



Dim myResponse As HttpWebResponse = myReq.GetResponse

myResponse.Cookies = myReq.CookieContainer.GetCookies(myReq.RequestUri)

myAspSession = myResponse.Cookies(0).ToString



'Then for subsequent requests

Dim myReq As HttpWebRequest = WebRequest.Create(myNextURL)

myReq.Headers.Add("Cookie", myAspSession)



-Eria

 

[Travis Bell]

Set the AllowAutoRedirect to false, and then read the redirect from the
your HttpWebResponse object (rsp):

if ((rsp.StatusCode == HttpStatusCode.Found) ||
(rsp.StatusCode == HttpStatusCode.Redirect) ||
(rsp.StatusCode == HttpStatusCode.Moved) ||
(rsp.StatusCode == HttpStatusCode.MovedPermanently))
{
newURI = headers["Location"] ;
newURI = uri.Trim();
}

if (rsp.Headers["Set-Cookie"] != null)
{
newCookie = headers["Set-Cookie"];
}

Now just create a new HttpWebRequest with newURI for your location and
pop in the cookies you retrieved from above. Wash, Rinse, Repeat.

Good luck.