Downloading multiple WebPages and saving to text file help needed!

[Hexman]

Code below ----

I'm trying to save some specific web pages to disk as text files. I searched the Internet and found a basic example which I changed to fit my needs.
I tested it out first on a single URL and it worked fine. Now when I incorporate an array of URL's, it fails to work. The first "responseFromServer"
properly retrieves and displays "http://finance.yahoo.com" (or any other valid URL for the first webrequest.create), but when I use the urls from the
array in the for/next structure, I never get a responseFromServer. My guess is that there must be some kind of initialization to the WebRequest,
Stream, or WebResponse.

A new command for me is "reader.ReadToEnd". In one fell swoop it reads the entire webpage. Very cool. Now I also want to write the entire
"responseFromServer" to disk as a text file. I've seen commands like "File.WriteAllText" and it seems to work well for what I need. Should I look
into any other commands to save the "responseFromServer" to disk as a text file?

Thanks,

Hexman

--------------------------------------------------------------------------------------------
Imports System
Imports System.IO
Imports System.Net
Imports System.Net.WebClient
Imports System.Windows.Forms
Public Class Form1

Private Sub btnProcess_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnProcess.Click
Dim x As Integer
Dim aryUrls(15) As String
aryUrls(0) = "http://finance.yahoo.com/q/bc?s=DOW&t=1d"
aryUrls(1) = "http://finance.yahoo.com/q/bc?s=IBM&t=6m"
aryUrls(2) = "http://finance.yahoo.com/q/bc?s=MSFT&t=6m"
aryUrls(3) = "http://biz.yahoo.com/top.html"

Dim request As WebRequest = WebRequest.Create("http://finance.yahoo.com/")
Dim response As WebResponse = request.GetResponse()
Dim dataStream As Stream = response.GetResponseStream()
Dim reader As New StreamReader(dataStream)
Dim responseFromServer As String = reader.ReadToEnd()

MsgBox(responseFromServer) <=== Shows finance.yahoo.com

For x = 0 To 3
request = WebRequest.Create(aryUrls(x))
dataStream = response.GetResponseStream()
responseFromServer = reader.ReadToEnd()
MsgBox(responseFromServer) <=== Shows nothing
Next

reader.Close()
response.Close()
End Sub

 

[Cor Ligthert [MVP]]

Hexman,

To check this kind of routines is a hell of a job and yours seems to me
incomplete so even not to debug..
It looks like the sample on our webpages but even than I find that always
painfull to debug.

http://www.vb-tips.com/default.aspx?ID=541adf13-d9c0-435c-893f-56dbb63fdf1c

Be aware however that if there are frames used on a webpage, you get with
this kind of routines only the MainFrame.

That was the first that came up into my mind.

Cor