Automate screen scraping: How to programmically "push" a Login button on another web page?

  • Thread starter Thread starter shmulik28
  • Start date Start date
S

shmulik28

I'm hoping to write a utility program that will navigate to another web
site, logon with my username/password, and download data automatically
on a schedule.

When I use HttpWebRequest to get the initial logon screen, there are
multiple embedded forms (Login, Forgot your Password, etc).

I want to be able to pass my username/password information to the web
page's login form and "push" the button (on their webpage, this is done
through some javascript code). When I view the source code of this
page, it appears that several hidden variables are set, so I imagine
the I would need to get/accept/return cookies.

I'm not sure how to "direct" the infomration to a page's particular
"form" - in my case, "pushing" the Login button.

Is there an example to automating this type of login that someone could
share with me?

Thanks!
 
Brock thanks - your utility looks like a great help!.

One last question if I may - when I try and do an HttpWebRequest to the
main logon page, I get back a page with this error message:

"Please enable Javascript in your browser to use [the website]"

I've set the UserAgent to:

...
request.UserAgent = "Mozilla/4.0";

Hoping that it just needed to think it had a compatible browser ...
what do I need set in order to let the page think that javascript is
enabled?

Thanks!
 
Brock thanks - your utility looks like a great help!.

It's not mine -- It's Microsoft's :)
"Please enable Javascript in your browser to use [the website]"

I've set the UserAgent to:

...
request.UserAgent = "Mozilla/4.0";
Hoping that it just needed to think it had a compatible browser ...
what do I need set in order to let the page think that javascript is
enabled?
Click to expand...

I'd try one of these until it works (I'm sure you can guess the ones to try
first):

http://www.zytrax.com/tech/web/browser_ids.htm

-Brock
DevelopMentor
http://staff.develop.com/ballen
 
Back
Top