PC Review


Reply
Thread Tools Rate Thread

Automate screen scraping: How to programmically "push" a Login button on another web page?

 
 
shmulik28@yahoo.com
Guest
Posts: n/a
 
      8th May 2005
I'm hoping to write a utility program that will navigate to another web
site, logon with my username/password, and download data automatically
on a schedule.

When I use HttpWebRequest to get the initial logon screen, there are
multiple embedded forms (Login, Forgot your Password, etc).

I want to be able to pass my username/password information to the web
page's login form and "push" the button (on their webpage, this is done
through some javascript code). When I view the source code of this
page, it appears that several hidden variables are set, so I imagine
the I would need to get/accept/return cookies.

I'm not sure how to "direct" the infomration to a page's particular
"form" - in my case, "pushing" the Login button.

Is there an example to automating this type of login that someone could
share with me?

Thanks!

 
Reply With Quote
 
 
 
 
Brock Allen
Guest
Posts: n/a
 
      8th May 2005
You'll have to use HttpWebRequest to do a HTTP POST passing values for the
IDs of the username and password fields. I'd suggest downloading Fiddler
to see the HTTP traffic and it'll make what's being passed over the HTTP
protocol seem so much more clear.

http://www.fiddlertool.com/fiddler/

-Brock
DevelopMentor
http://staff.develop.com/ballen



> I'm hoping to write a utility program that will navigate to another
> web site, logon with my username/password, and download data
> automatically on a schedule.
>
> When I use HttpWebRequest to get the initial logon screen, there are
> multiple embedded forms (Login, Forgot your Password, etc).
>
> I want to be able to pass my username/password information to the web
> page's login form and "push" the button (on their webpage, this is
> done through some javascript code). When I view the source code of
> this page, it appears that several hidden variables are set, so I
> imagine the I would need to get/accept/return cookies.
>
> I'm not sure how to "direct" the infomration to a page's particular
> "form" - in my case, "pushing" the Login button.
>
> Is there an example to automating this type of login that someone
> could share with me?
>
> Thanks!
>




 
Reply With Quote
 
shmulik28@yahoo.com
Guest
Posts: n/a
 
      8th May 2005
Brock thanks - your utility looks like a great help!.

One last question if I may - when I try and do an HttpWebRequest to the
main logon page, I get back a page with this error message:

"Please enable Javascript in your browser to use [the website]"

I've set the UserAgent to:

...
request.UserAgent = "Mozilla/4.0";

Hoping that it just needed to think it had a compatible browser ...
what do I need set in order to let the page think that javascript is
enabled?

Thanks!

Brock Allen wrote:
> You'll have to use HttpWebRequest to do a HTTP POST passing values

for the
> IDs of the username and password fields. I'd suggest downloading

Fiddler
> to see the HTTP traffic and it'll make what's being passed over the

HTTP
> protocol seem so much more clear.
>
> http://www.fiddlertool.com/fiddler/
>
> -Brock
> DevelopMentor
> http://staff.develop.com/ballen
>
>
>
> > I'm hoping to write a utility program that will navigate to another
> > web site, logon with my username/password, and download data
> > automatically on a schedule.
> >
> > When I use HttpWebRequest to get the initial logon screen, there

are
> > multiple embedded forms (Login, Forgot your Password, etc).
> >
> > I want to be able to pass my username/password information to the

web
> > page's login form and "push" the button (on their webpage, this is
> > done through some javascript code). When I view the source code of
> > this page, it appears that several hidden variables are set, so I
> > imagine the I would need to get/accept/return cookies.
> >
> > I'm not sure how to "direct" the infomration to a page's particular
> > "form" - in my case, "pushing" the Login button.
> >
> > Is there an example to automating this type of login that someone
> > could share with me?
> >
> > Thanks!
> >


 
Reply With Quote
 
Brock Allen
Guest
Posts: n/a
 
      8th May 2005
> Brock thanks - your utility looks like a great help!.

It's not mine -- It's Microsoft's

> "Please enable Javascript in your browser to use [the website]"
>
> I've set the UserAgent to:
>
> ...
> request.UserAgent = "Mozilla/4.0";
> Hoping that it just needed to think it had a compatible browser ...
> what do I need set in order to let the page think that javascript is
> enabled?


I'd try one of these until it works (I'm sure you can guess the ones to try
first):

http://www.zytrax.com/tech/web/browser_ids.htm

-Brock
DevelopMentor
http://staff.develop.com/ballen




 
Reply With Quote
 
 
 
Reply

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Feature Request: "Link to record" Window should come with "push-pin"button to allow for batch linking mduijkers@gmail.com Microsoft Outlook BCM 1 4th Dec 2007 12:17 AM
how can one disable (or click "yes" through automate VB) when printing in MS Word and getting "The margins of section 1 are set outside the printable area of the page" Joe Microsoft VB .NET 0 14th Jun 2007 10:43 PM
when i push the "end" button the curser always goes to cell S47 =?Utf-8?B?bGVv?= Microsoft Excel Misc 1 30th May 2006 09:01 PM
how to "push" button from code Tony C Microsoft Access Form Coding 0 29th Apr 2004 05:11 PM
how to disable "options" button in login screen? Guanrao Microsoft Windows 2000 Security 0 4th Sep 2003 06:24 PM


Features
 

Advertising
 

Newsgroups
 


All times are GMT +1. The time now is 12:21 AM.