HttpWebRequest times out with ASP.NET_SessionId cookie (response TIMEOUT)



It has been 6 days of re-writing the same code in different ways to try
and avoid a getResponse Timeout which haunted me for much too long now.

I am trying to do a very simple thing which is prooving the most
difficult in my career. Scrape a screen which has several session
variables within it, and send this page to an e-mail.

I have two pages in my application webform1.aspx and webform2.aspx, in
webform1.aspx I set several session variables which when one goes to
webform2.aspx will be displayed in varies places. Going to
webform2.aspx through the normal browser works ok.

Now I write a very simple screen scraper using httpWebRequest pointing
to webform2.aspx, I pass the necessary cookies in the header to
maintain session state and the page times out on me. If I don't pass
the session cookie ( ASP.NET_SessionID ) then the page is scraped but
of course the session variables are all null.

Can anyone tell me if I can scrape a page within my application without
using htppwebrequest or else if there is a known bug / trick on how to
read the current session ID (I read it from this.Context ) and pass it
over to the webform2.aspx.

One version of my code...

HttpWebRequest m_webreq;
HttpWebResponse m_webresp;
string m_response;
System.Net.CookieCollection m_cookies;
string url="webform2.aspx";

m_webreq = (HttpWebRequest) WebRequest.Create(url);
m_webreq.CookieContainer = new CookieContainer();

foreach (String cookieName in
System.Web.HttpCookie cookie =
System.Net.Cookie myCookie = new System.Net.Cookie();
myCookie.Name = cookie.Name;
myCookie.Value = cookie.Value;
myCookie.Domain = this.Context.Request.Url.Host;

m_webreq.Method = "POST";
m_webreq.Timeout = 20000;
//Times out in the next line if I use POST and have cookies
m_webresp = (HttpWebResponse)m_webreq.GetResponse();

StreamReader stream = new StreamReader(m_webresp.GetResponseStream(),

//m_response is never filled with the page that I need
m_response = stream.ReadToEnd();


Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question