PC Review


Reply
Thread Tools Rate Thread

Screen Scraping a Password Protected Site

 
 
apondu
Guest
Posts: n/a
 
      12th Apr 2007




I'm trying to screen scrape a site that requires a password.

I am using C#.Net, i am new to this and with the information available
around on the internet i just put tht information into the code.

But still i am not able to achieve what i want to.

I have posted the code which i have written, along with the site and
the userid ans password

Can someone take a look at the code and help with the information on
where i am going wrong and guide me across with the correct procedure.
and help to perform screen scrapping of the password word protected
site.

Thnaks for the help

Regards,
Govardhan.

My Code :



public void getContent()
{
UTF8Encoding utf = new UTF8Encoding();

string url = "http://www.bloglines.com/login";

Uri uri = new Uri(url);

string userName = "(E-Mail Removed)";

string userPassword = "password123";

int port = 80;

string proxyUserName = "";

string proxyPassword = "";

string proxyName = "";

CookieCollection Cookies = new CookieCollection();

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);

request.Method = "POST";

request.ContentType = "application/x-www-form-urlencoded";

// request.Credentials = new
NetworkCredential( userName,userPassword );

request.CookieContainer = new CookieContainer();

request.AllowAutoRedirect = true;

if (Cookies != null && Cookies.Count > 0)

request.CookieContainer.Add(Cookies);

// Code Changed to have post data

string postData = "email=(E-Mail Removed)&password=password123";
byte[] postBytes = Encoding.UTF8.GetBytes (postData);

Stream postStream = request.GetRequestStream();
postStream.Write(postBytes, 0, postBytes.Length);
postStream.Close();

// End of Code Changed to have post data

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

if (response.Cookies.Count > 0)
{
if (Cookies == null)

{
Cookies = response.Cookies;
}
else
{
// If we already have cookies update list

foreach (Cookie oRespCookie in response.Cookies)
{
bool bMatch = false;

foreach(Cookie oReqCookie in Cookies)
{
if (oReqCookie.Name == oRespCookie.Name)
{
oReqCookie.Value = oRespCookie.Name;

bMatch = true;

break;
}
}
if (!bMatch)
Cookies.Add(oRespCookie);
}

}
} // End of response.Cookies.Count


request.CookieContainer.Add(Cookies);

request = (HttpWebRequest)WebRequest.Create("http://
www.bloglines.com/myblogs");

response = (HttpWebResponse)request.GetResponse();

Stream strm = response.GetResponseStream();

System.Text.Encoding ec =
System.Text.Encoding.GetEncoding("utf-8");

System.IO.StreamReader reader = new System.IO.StreamReader(strm,
ec);

string str = reader.ReadToEnd();

response.Close();
strm.Close();
reader.Close();

FileStream fs = new FileStream("c:\
\q.htm",FileMode.Create,FileAccess.Write);

StreamWriter sw = new StreamWriter(fs);

sw.Write(str);

sw.Close();
}

 
Reply With Quote
 
 
 
Reply

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Screen Scraping a Password Protected Site apondu Microsoft C# .NET 4 13th Apr 2007 05:40 AM
Screen Scraping a Password Protected Site apondu Microsoft Dot NET Framework 0 12th Apr 2007 08:38 PM
Screen Scraping a Password Protected Site apondu Microsoft Dot NET Framework 0 12th Apr 2007 08:38 PM
Screen Scraping a Password Protected Site Gregory A Greenman Microsoft VB .NET 3 17th Dec 2006 12:28 PM
How to do screen scraping where the site requires a log in Alan Silver Microsoft ASP .NET 2 5th Sep 2006 10:56 PM


Features
 

Advertising
 

Newsgroups
 


All times are GMT +1. The time now is 10:18 PM.