Need page source for a list of URLs. Anyone know of any program?

[Shani]

I need a program that can take a list of URLs and and save their source
page ( html code) on my computer as a text file. Does anyone know of
anysuch program.

 

[lisztfr]

Shani a écrit :

I need a program that can take a list of URLs and and save their source
page ( html code) on my computer as a text file. Does anyone know of
anysuch program.

htget ?

It exist un cygwin too, IIRC. i never used it...

laurent h

 

[3c273]

If you have Python installed (http://www.python.org), the following script
will do what you want.

-----begin-----
import urllib
urlfile = open('c:\\temp\\url.txt', 'r')
for lines in urlfile:
try:
outfilename = lines.replace('/', '-')
urllib.urlretrieve(lines.strip('/n'), 'c:\\temp\\' +
outfilename.strip('\n')[7:] + '.txt')
except:
pass
-----end-----

Note that the list of urls must be a text file and the urls must begin with
"http://" (without the quotes). I would suggest that you make a folder
called "temp" on your c drive and name your list of urls "url.txt", then you
can use the script unaltered. If you wish to use this and need help, post
back here.
Louis

 

[3c273]

Sorry, that got word wrapped. I'll try again.
-----begin-----
import urllib
urlfile = open(r'c:\temp\url.txt', 'r')
for lines in urlfile:
try:
outfilename = lines.replace('/', '-')
urllib.urlretrieve(lines.strip('/n'), 'c:\\temp\\' \
+ outfilename.strip('\n')[7:] + '.txt')
except:
pass
-----end-----
Louis

If you have Python installed (http://www.python.org), the following script
will do what you want.

-----begin-----
import urllib
urlfile = open('c:\\temp\\url.txt', 'r')
for lines in urlfile:
try:
outfilename = lines.replace('/', '-')
urllib.urlretrieve(lines.strip('/n'), 'c:\\temp\\' +
outfilename.strip('\n')[7:] + '.txt')
except:
pass
-----end-----

Note that the list of urls must be a text file and the urls must begin with
"http://" (without the quotes). I would suggest that you make a folder
called "temp" on your c drive and name your list of urls "url.txt", then you
can use the script unaltered. If you wish to use this and need help, post
back here.
Louis

I need a program that can take a list of URLs and and save their source
page ( html code) on my computer as a text file. Does anyone know of
anysuch program.

 

[Bill Miller]

I need a program that can take a list of URLs and and save their source
page ( html code) on my computer as a text file. Does anyone know of
anysuch program.

Don't know if this will help.

Bookmark Wizard

Bookmark Wizard is a little program with a simple wizard-like
interface that generates an HTML page with all/selected links
from your Internet Favorites folder. The links in the resulting
page are grouped and sorted in ascending order like they are in
your Favorites folder. You can specify colors for links and
backgrounds, title, caption, etc., for your page. Also, you can
exclude certain folders from being listed. Advanced users are
able to edit the entire page template to customize it further.
The resulting web page can be used as a links page on your
homepage; you can send it to a friend or use it just as a startup
page for your web browser etc.

Please note this program is not a HTML to Favorites converter, it
can only create a HTML page from Favorites.

Bookmark Wizard 2.0.1, released on Saturday, July 10, 1999
Download: Bookmark_Wizard_201.exe (401 KB)

http://www.moonsoftware.com/freeware.asp

 

[Susan Bugher]

I need a program that can take a list of URLs and and save their source
page ( html code) on my computer as a text file. Does anyone know of
anysuch program.

Please explain what "take a list of URLs" means. Where is this list? Is
it on a web page? Did someone send you a text file with a list of URLs?
or _____________???

What kind of a list do you want when you're done? A plain text list of
URLs? A text file with a list with clickable links (aka a web page)? or
______________???

There are many simple answers. . . What is the question?

Susan
--
Posted to alt.comp.freeware
Search alt.comp.freeware (or read it online):
http://www.google.com/advanced_group_search?q=+group:alt.comp.freeware
Pricelessware & ACF: http://www.pricelesswarehome.org
Pricelessware: http://www.pricelessware.org (not maintained)

 

[Adam Piggott]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I need a program that can take a list of URLs and and save their source
page ( html code) on my computer as a text file. Does anyone know of
anysuch program.

Wget can do the job.
http://www.gnu.org/software/wget/

Once you've unzipped it you want to:
wget --force-directories -i filename.txt

This will have wget download each URL and put it into a directory generated
from the host name, i.e.:

C:\www.bbc.co.uk\index.html
C:\www.example.com\index.htm

The URLs must be prefixed with http:// for the input file to be understood
correctly.

HTH

Adam Piggott, Proprietor, Proactive Services (Computing).
http://www.proactiveservices.co.uk/

Please replace dot invalid with dot uk to email me.
Apply personally for PGP public key.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (MingW32)

iD8DBQFEhqn57uRVdtPsXDkRAv8NAJ9M4IJfnmOLEQKsjyIjxH7/vGN3xwCeN+8u
X5oNNyRbvY9qxQDaVmblWBw=
=mbni
-----END PGP SIGNATURE-----

 

[Al Klein]

There are many simple answers. . . What is the question?

I often say about programming that the most difficult part is defining
the problem. The next most difficult part is deciding how to solve
the problem. Writing the program is the easy part. Same situation
here.

 

[Lou]

I often say about programming that the most difficult part is defining
the problem. The next most difficult part is deciding how to solve
the problem. Writing the program is the easy part. Same situation
here.

Ah but checking to see that the written program works - that is the real
problem

ALMOST

Then you have to see if the problem that was solved was really the problem
that needed to be solved.

Lou
(30+ years of doing this stuff)