Unable to Retrieve Complete Web Page

[Alan]

I am trying to completely retrieve a web page and search it using
VBScript regular expressions. However, I do not get the complete web
page.

Am I running into some VBA string length limit or what? Is there some
way around it?

My Sub may be found below.

Alan

Sub GetGoogleHomePage()

Dim oIE As SHDocVw.InternetExplorer
Dim sPage As String

' Create a new (hidden) instance of IE
Set oIE = New SHDocVw.InternetExplorer

' Open the web page
oIE.Navigate "http://www.google.com"

' Wait for the page to complete loading
Do Until oIE.ReadyState = READYSTATE_COMPLETE
DoEvents
Loop

' Retrieve the text of the web page
sPage = oIE.Document.body.InnerHTML

' Display the HTML
Debug.Print sPage

End Sub

 

[Alan]

Leith,
It seems like I will run into the same problem (too long)
or have problems with text broken over multiple lines when I read the
data back from the file.

But, I'll give this a try.

Could you please explain the "WinHttp.WinHttpRequest.5.1" vs.
"WinHttp.WinHttpRequest.5"?

Thanks, Alan

 

[Alan]

I do seem to have the same truncation problem when I read the firl
back in with this code:

' Read each line of the file, looking for the description
Dim myFileName As String
Dim myLine As String
Dim FileNum As Long

myFileName = ThisWorkbook.Path & "\URL.txt"
FileNum = FreeFile
Close FileNum
Open myFileName For Input As FileNum
count = 0
Do While Not EOF(FileNum)
count = count + 1
Line Input #FileNum, myLine
Debug.Print myLine
Debug.Print "===========================================" &
vbCrLf
Debug.Print count & vbCrLf
Debug.Print vbCrLf &
"===========================================" & vbCrLf
myLine = ExtractCoDescr(myLine)
If Len(myLine) > 0 Then
GetCoDescription = myLine
Close FileNum
Exit Do
End If
Loop
Close FileNum

 

[Alan]

I am trying to extract text following a series of HTML tags and
keywords.

If you can explain how I get started on properly reading it, I
would appreciate it.

Alan

 

[Alan]

I used a VBScript TextStream object. That worked.

Alan

 

[Don Guillett]

Care to post it for the archives

 

[Alan]

Here is the code that worked:

Sub ReadWebFileTextStream()
Dim fs As Object ' scripting.filesystemobject
Dim txtin As Object ' scripting.textstream
Dim strline As String

Set fs = CreateObject("scripting.filesystemobject")
Set txtin = fs.opentextfile(ThisWorkbook.Path & "\URL.txt", 1) '
1 is for Reading

Do While Not txtin.atendofstream
strline = txtin.readline
'
' Process data here . . .
'
Loop
txtin.Close
Set txtin = Nothing
Set fs = Nothing
End Sub

 

[Ron Rosenfeld]

Am I running into some VBA string length limit or what? Is there some
way around it?

I believe there is a limit as to how much data the immediate window can
display.

However, I've had no problems parsing long documents using the innerHTML or
innerTEXT property.
--ron