Question about System.Net.WebClient

[jon]

I was recently messing around with WebClient, and was using the following code:

Try
Dim wc As New System.Net.WebClient
Dim strHTML As String = ""
strHTML = wc.DownloadString("http://www.yahoo.com")
TextBox1.Text = strHTML
Catch
MsgBox(Err.Description)
End Try

to download the webpage html and such. Tried it on yahoo, google, and some others.
But when I tried http://www.youtube.com it came back with a
"The remote server returned an error: (400) Bad Request." error.
Why did it give the error instead of returning the html? How would you get the html of
a page like that?

 

[kimiraikkonen]

I was recently messing around with WebClient, and was using the followingcode:

        Try
            Dim wc As New System.Net.WebClient
            Dim strHTML As String = ""
            strHTML = wc.DownloadString("http://www.yahoo.com")
            TextBox1.Text = strHTML
        Catch
            MsgBox(Err.Description)
        End Try

to download the webpage html and such. Tried it on yahoo,  google, and some others.
But when I triedhttp://www.youtube.comit came back with a
"The remote server returned an error: (400) Bad Request." error.
Why did it give the error instead of returning the html? How would you get the html of
a page like that?

Hi,
Using DownloadString, actually, you're getting source code of HTML
output. As it works for other domains rather than Youtube(i can't test
Youtube now, because my ISP blocks it), another way to retrieve the
source code of Youtube might be to navigate http://www.youtube.com
using a Webbrowser, after page is loaded, you can try to get HTML
source code using:
Webbrowser1.DocumentText.

I hope it'll be an alternative,

Onur Güzel

 

[Martin Honnen]

I was recently messing around with WebClient, and was using the following code:

Try
Dim wc As New System.Net.WebClient
Dim strHTML As String = ""
strHTML = wc.DownloadString("http://www.yahoo.com")
TextBox1.Text = strHTML
Catch
MsgBox(Err.Description)
End Try

to download the webpage html and such. Tried it on yahoo, google, and some others.
But when I tried http://www.youtube.com it came back with a
"The remote server returned an error: (400) Bad Request." error.
Why did it give the error instead of returning the html? How would you get the html of
a page like that?

Try setting the User-Agent header to the one of a known browser e.g.
wc.Headers("User-Agent") = _
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)"

 

[jon]

Ah, thanks for the suggestions, setting the user-agent fixed the problem.
Yeah, I could of used the Webbrowser and gotten source from that, but no need now
since webclient works fine. Good to keep in mind as an alt for webclient tho.