Screen Scraping With VB.NET

N

Nick

What i wanted to do is make a webgame easyer, by getting
the page, and in stead of having multiple click actions,
make it all just 1 click and record stats for my profile.

For Example:

there is an arm wrestle part, you have to select oponent,
and click Challenge, because of the high traffic, once
you click challenge, the match is allready over, and u
have to refresh the page and try another.

The problem however is: when i make a webrequest, i only
get:
[Response]

<script type="text/javascript">
if (self.name != 'right')
top.location.replace('index.php');
</script>

<META HTTP-EQUIV="Refresh" CONTENT="0; URL=index.php"
TARGET="_top">

[/Response]

added a response bracket to show what i got from the page.

I wonder what headers i need to set to get the full page,
when i use IE, and do Show Source, i get much more (will
show in the bottom of the post.)

Here's the code i had to get the response:

Code: 
Private Sub Form1_Load(ByVal sender As System.Object,
ByVal e As System.EventArgs) Handles MyBase.Load
txtAW.Text = getpage
("http://www.bootleggers.us/index.php")
End Sub

Private Function getpage(ByVal url As String) As
String
Dim awPR As System.Net.HttpWebRequest = CType
(System.Net.WebRequest.Create(url),
System.Net.HttpWebRequest)
Dim awResp As System.Net.WebResponse
awPR.Headers.Add("name", "right")
Try
awResp = awPR.GetResponse
Catch ex As Exception
MsgBox(ex.Message)
Exit Function
End Try
Dim r As New System.IO.StreamReader
(awResp.GetResponseStream())
Dim strTmp As String = r.ReadToEnd
r.Close()
Return strTmp
End Function

Thanx in Advance,

Nick

Response i should get:



<script type="text/javascript">
if (self.name != 'right')
top.location.replace('index.php');
</script>


<html>
<head>
<link REL="stylesheet" TYPE="text/css"
HREF="includes/main.css">
</head>
<body bgcolor=#333333>
<head>
<link REL=\"stylesheet\" TYPE=\"text/css\"
HREF=\"http://www.bootleggers.us/includes/main.css\">
</head>

<body topmargin=0>
<table width="100%" border=0 cellpadding=0 cellspacing=0
bgcolor="#5C5C5C">
<tr><td><img src="http://www.bootleggers.us/l.gif"></td>
<td width="100%"
background="http://www.bootleggers.us/m.gif"></td>
<td><img src="http://www.bootleggers.us/r.gif"></td>
</tr>
</table>
<br><br>

<center>
<table width=90% cellspacing=0 cellpadding=0 border=0>
<tr>
<td width=50% valign=top align=center>

<form method=post action=>
<table border=1 cellspacing=0 cellpadding=2
bordercolor=black width=90% class=sub2>
<tr>
<td class=header align=center colspan=3>Train</td>
</tr>
<tr>
<td class=header align=center>Activity</td>
<td class=header align=center>Rest</td>
<td class=header align=center>Gain</td>
</tr>
<tr>
<td width=50%><input type=radio name=train value=1> 10
Pullups</td>
<td>10 Mins</td>
<td>13%</td>
</tr>
<tr>
<td width=50%><input type=radio name=train value=2> 15
Dumbbells</td>
<td>5 Mins</td>
<td>7%</td>
</tr>
<tr>
<td width=50%><input type=radio name=train value=3> 25
Pushups</td>
<td>2 Mins</td>
<td>3%</td>
</tr>
<tr>
<td colspan=3 align=right><input type=submit
value="Work Out!"></td>
</tr>
</table>
</form>

<br>

<table border=1 cellspacing=0 cellpadding=2
bordercolor=black width=90% class=sub2>
<tr>
<td class=header align=center colspan=3>Current
Strength</td>
</tr>
<tr>
<td width=45%>Wins/Losses Ratio:</td><td>466:90</td>
</tr>
<tr>
<td width=45%>Current Level:</td><td>29</td>
</tr>
<tr>
<td width=45%>Percent to Next Level:</td>
<td>
<table border=0 cellspacing=0 cellpadding=0
width=100% class=sub2>
<tr><td width=36% bgcolor=green
align=right> </td><td>36%</td>
</tr>
</table>
</td>
</tr>
</table>

<br><br>

<form method=post action=>
<table border=1 cellspacing=0 cellpadding=2
bordercolor=black width=90% class=sub2>
<tr>
<td class=header align=center colspan=3>Create
Match</td>
</tr>
<tr>
<td>Bet:</td><td><input type=text name=matchbet></td>
</tr>
<tr>
<td> </td><td align=right><input type=submit
value="Start!"></td>
</tr>
</table>
</form>


</td>
<td width=50% valign=top align=center>

<form method=post action=>
<table border=1 cellspacing=0 cellpadding=2
bordercolor=black width=90% class=sub2>
<tr>
<td class=header align=center colspan=4>Current
Matches</td>
</tr>
<tr>
<td class=header align=center width=1%> </td>
<td class=header align=center>Opponent (W:L)</td>
<td class=header align=center>Bet</td>
</tr>
<tr>
<td width=1%><input type=radio name=challengematch
value="7470"></td>
<td><a href="viewprofile.php?
viewuser=Aronovic">Aronovic</a> (193:571)</td>
<td>$150</td>
</tr>
<tr>
<td width=1%><input type=radio name=challengematch
value="7471"></td>
<td><a href="viewprofile.php?
viewuser=luckchris">luckchris</a> (45:153)</td>
<td>$111</td>
</tr>
<tr>
<td width=1%><input type=radio name=challengematch
value="5531"></td>
<td><a href="viewprofile.php?
viewuser=Rikskie">Rikskie</a> (1421:110)</td>
<td>$1,624,032</td>
</tr>
<tr>
<td width=1%><input type=radio name=challengematch
value="7392"></td>
<td><a href="viewprofile.php?viewuser=swipe">swipe</a>
(2838:399)</td>
<td>$150</td>
</tr>
<tr>
<td colspan=4 align=right><input type=submit
value="Challenge!"></td>
</tr>
</table>
</form>

</td>
</tr>
</table>
</center><br><br><center>
<table border=1 cellspacing=0 cellpadding=2
bordercolor=black class=sub2>
<tr>
<td><b>Crew:</b> None</td>
<td><b>Rank:</b> Boss</td>
<td><b>Cash:</b> $333,161</td>
<td><b>Health:</b><td bgcolor=green width=100
align=center>100%</td><td class=sub2 width=0
align=center> </td></td>
</tr>
<tr>
<td colspan=2><b>Gun:</b> HighStandard .22 (0
Bullets)</td>
<td colspan=4><b>Protection:</b> None </td>
</tr>
</table>

<br>

<a
href="http://www.bootleggers.us/agreement.php">Terms of
Service</a>
</center>
 
N

Nick

i only added this below here, to try if that made anny
difference, but as u might be able to tell, i am kindda
lost...
awPR.Headers.Add("name", "right")
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Need assistance with Detailsview and passing values to a textbox 0
HTML, CSS and JavaScript in a Gridview 2
External data from HTML document 2
IHTMLELement2 4
Displaying XML 1
The trouble with my code ? 6
Making a domain lookup in asp.net 2
Regular Expressions 10

Top