Getting data from the Internet

  • Thread starter Thread starter MarkS
  • Start date Start date
M

MarkS

Hi,
I have never done this so this is a new adventure for me. This is more of a
question of where I can get more information about this sort of problem, as I
can see several other jobs requiring this coming up in the near future

1. Start IE/Firefox
2. I need to log on to the web site
3. I need to fill out the search query and press the button for it
4. Copy the data from the web page to a spread sheet
5. If there is another page of data, select the next page and repeat 3
6. Log off from the web site
7. Shut down IE/Firefox

If you could direct me to somewhere I could find out how to do this that
would be great

MarkS
 
There isn't a lot of good information on downloading from the Internet. Each
webpage is different. To downlod web data I use a combination of lots of
different methods and a lot of experience from worksing with lot of webpages
to successuflly acommplish this task. I can help but without a account and
password at the sight it is going to take a lot instruction. If yo get me
the URL I can get your through the login menu without actually having the
password. I will give you generic code and you can modifed the code to put
oin the correct account and password. but then I won't have access to the
remaining pages.

I often look at the source HTML code to help me. going to the webo browser
and using the menu view - Source gets me the HTML code. Also dumping the
data to the worksheet also helps using the macro below.

The login webpage is usually a form.
To get a form or any other tag use this instruction
Set Form = IE.document.getElementsByTagname("Form")

Tags in the htmp source are the string following the angle brackets

<abc 'the html data here /abc>
or
<abc 'the html data here />


the end tag may be just the angle bracket or a forward sla and angle brackt
without the tagname.

To get "id=" items from the html us this instruction. Look at source html
code for the string "id="
Set zip5 = IE.document.getElementById("zip5")

Both of the above formats return multiple occurances of each item. You can
access each item returned by using a for look
for each itm = form

next itm

The items are also an array starting at index 0. So you can get the 3rd
"form" using this instruction

Set Myform = form(2) assuming you used getElementsByTagname method above.


Try this code

Sub GetZipCodes()


ZIPCODE = InputBox("Enter 5 digit zipcode : ")

Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True

URL = "http://zip4.usps.com/zip4/citytown_zip.jsp"

'get web page
IE.Navigate2 URL
Do While IE.readyState <> 4 or
While IE.busy = True

DoEvents
Loop

Set Form = IE.document.getElementsByTagname("Form")

Set zip5 = IE.document.getElementById("zip5")
zip5.Value = ZIPCODE


Set ZipCodebutton = Form(0).onsubmit

Form(0).submit
Do While IE.busy = True
DoEvents
Loop

Set Table = IE.document.getElementsByTagname("Table")
Location = Table(0).Rows(2).innertext
IE.Quit
MsgBox ("Zip code = " & ZIPCODE & " City/State = " & Location)

'test code for dumpling worksheet
RowCount = 1
for each itm in IE.document.all
Range("A" & rowcount) = itm.tagname
Range("B" & rowcount) = itm.classname
Range("C" & rowcount) = left(itm.innertgext,1024) 'filter data to
prevent

'memory errors


RowCount = RowCount + 1
next itm

End Sub
 
There isn't a lot of good information on downloading from the Internet.  Each
webpage is different.  To downlod web data I use a combination of lots of
different methods and a lot of experience from worksing with lot of webpages
to successuflly acommplish this task.  I can help but without a accountand
password at the sight it is going to take a lot instruction.  If yo getme
the URL I can get your through the login menu without actually having the
password.  I will give you generic code and you can modifed the code toput
oin the correct account and password.  but then I won't have access to the
remaining pages.

I often look at the source HTML code to help me.  going to the webo browser
and using the menu view - Source gets me the HTML code.  Also dumping the
data to the worksheet also helps using the macro below.

The login webpage is usually a form.
To get a form or any other tag use this instruction
Set Form = IE.document.getElementsByTagname("Form")

Tags in the htmp source are the string following the angle brackets

<abc 'the html data here                               /abc>
or
<abc 'the html data here                               />

the end tag may be just the angle bracket or a forward sla and angle brackt
without the tagname.

To get "id=" items from the html us this instruction.  Look at sourcehtml
code for the string "id="
Set zip5 = IE.document.getElementById("zip5")

Both of the above formats return multiple occurances of each item.  Youcan
access each item returned by using a for look
for each itm = form

next itm

The items are also an array starting at index 0.  So you can get the 3rd
"form" using this instruction

Set Myform = form(2) assuming you used getElementsByTagname method above.

Try this code

Sub GetZipCodes()

ZIPCODE = InputBox("Enter 5 digit zipcode : ")

Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True

URL = "http://zip4.usps.com/zip4/citytown_zip.jsp"

'get web page
IE.Navigate2 URL
Do While IE.readyState <> 4 or
   While IE.busy = True

   DoEvents
Loop

Set Form = IE.document.getElementsByTagname("Form")

Set zip5 = IE.document.getElementById("zip5")
zip5.Value = ZIPCODE

Set ZipCodebutton = Form(0).onsubmit

Form(0).submit
Do While IE.busy = True
   DoEvents
Loop

Set Table = IE.document.getElementsByTagname("Table")
Location = Table(0).Rows(2).innertext
IE.Quit
MsgBox ("Zip code = " & ZIPCODE & " City/State = " & Location)

'test code for dumpling worksheet
RowCount = 1
for each itm in IE.document.all
   Range("A" & rowcount) = itm.tagname
   Range("B" & rowcount) = itm.classname
   Range("C" & rowcount) = left(itm.innertgext,1024)  'filter data to
prevent            

 'memory errors

   RowCount = RowCount + 1
next itm

End Sub








- Show quoted text -

You are interested in pulling info from the internet to Excel. So, I
don't think this will solve your roblem but it is a way to place info/
passwords to the internet/Google. The blog spot that had this post
might have an answer for you. Go to http://vbadud.blogspot.com/

Here is a simple code that will login to Google accounts with the
provided user-name and password.

The program requires references to the following:

1 Microsoft Internet Controls
2. Microsoft HTML Object Library
 
Joel said:
There isn't a lot of good information on downloading from the Internet. Each
webpage is different. To downlod web data I use a combination of lots of
different methods and a lot of experience from worksing with lot of webpages
to successuflly acommplish this task. I can help but without a account and
password at the sight it is going to take a lot instruction. If yo get me
the URL I can get your through the login menu without actually having the
password. I will give you generic code and you can modifed the code to put
oin the correct account and password. but then I won't have access to the
remaining pages.

I often look at the source HTML code to help me. going to the webo browser
and using the menu view - Source gets me the HTML code. Also dumping the
data to the worksheet also helps using the macro below.

The URL is http://d-cyphatrade.com.au/
 
Back
Top