Possible to "scape" a website?

  • Thread starter Thread starter bbu
  • Start date Start date
B

bbu

Hello,
I need to collect data from a website, everyday. I get an email everyday with 6 to 10 links that is can pick form to get to them. I need to automate as much as possible.

My ideas so far: use an Outlook2000 macro
to automatically open the email when it comes in.
to send the links to a threaded web service
the web service thread
opens the link
opens a notepad of "the source"
scans the source for keywords and
puts the strings into an array
Will this work?
Is this the best way?
Thanks
BB
 
Hi there,

http://www.members.lycos.co.uk/nickpatemanpwp/soft-wtr2.htm

The above link has source of mine for an applicaton that scans HTML for
links, it would be easily modified to do as you suggest. As for opening
Outlook, that shouldn't be too difficult, you could even create a kind of
"Send to" plug-in that will enable you to open the file. One thing you
might want to think about is the "Referer" of the web page, this should be
set before attempting to download the content as you *might* encounter
problems. Getting the referer shouldn't be that difficult, the homepage of
the people who send you the email might be enough?

Nick.

Hello,
I need to collect data from a website, everyday. I get an email everyday
with 6 to 10 links that is can pick form to get to them. I need to automate
as much as possible.

My ideas so far: use an Outlook2000 macro
to automatically open the email when it
comes in.
to send the links to a threaded web service
the web service thread

opens the link

opens a notepad of "the source"

scans the source for keywords and

puts the strings into an array
Will this work?
Is this the best way?
Thanks
BB
 
Back
Top