HttpWebRequest times out with ASP.NET_SessionId cookie (response TIMEOUT)


D

davvel

It has been 6 days of re-writing the same code in different ways to try
and avoid a getResponse Timeout which haunted me for much too long now.

I am trying to do a very simple thing which is prooving the most
difficult in my career. Scrape a screen which has several session
variables within it, and send this page to an e-mail.

I have two pages in my application webform1.aspx and webform2.aspx, in
webform1.aspx I set several session variables which when one goes to
webform2.aspx will be displayed in varies places. Going to
webform2.aspx through the normal browser works ok.

Now I write a very simple screen scraper using httpWebRequest pointing
to webform2.aspx, I pass the necessary cookies in the header to
maintain session state and the page times out on me. If I don't pass
the session cookie ( ASP.NET_SessionID ) then the page is scraped but
of course the session variables are all null.


Can anyone tell me if I can scrape a page within my application without
using htppwebrequest or else if there is a known bug / trick on how to
read the current session ID (I read it from this.Context ) and pass it
over to the webform2.aspx.

One version of my code...

HttpWebRequest m_webreq;
HttpWebResponse m_webresp;
string m_response;
System.Net.CookieCollection m_cookies;
string url="webform2.aspx";

m_webreq = (HttpWebRequest) WebRequest.Create(url);
m_webreq.CookieContainer = new CookieContainer();

foreach (String cookieName in
System.Web.HttpContext.Current.Request.Cookies)
{
System.Web.HttpCookie cookie =
System.Web.HttpContext.Current.Request.Cookies[cookieName];
System.Net.Cookie myCookie = new System.Net.Cookie();
myCookie.Name = cookie.Name;
myCookie.Value = cookie.Value;
myCookie.Domain = this.Context.Request.Url.Host;
m_webreq.CookieContainer.Add(myCookie);
}

m_webreq.Method = "POST";
m_webreq.Timeout = 20000;
//Times out in the next line if I use POST and have cookies
m_webresp = (HttpWebResponse)m_webreq.GetResponse();

StreamReader stream = new StreamReader(m_webresp.GetResponseStream(),
System.Text.Encoding.UTF8);

//m_response is never filled with the page that I need
m_response = stream.ReadToEnd();
stream.Close();
m_webresp.Close();
 
Ad

Advertisements


Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top