Internet Explorer object...

[VBSome]

As far as I understand there is no native VB.NET equivalent of the VB6 Web
browser control. There are some things I can play with HTML wise, but none
do what I really need to do (which is ultimately walk the DOM &
programatically click on elements in the DOM).

I am trying to rebuild some of the stuff I've done in VB6 in dotnet by using
the web browser control thru the com interop. HELP! It behaves different
then I am used to and I am not sure where to go from here...

One of the most important events in what I am trying to accomplish is the
DocumentComplete event. The DocumentComplete fires (or at least should fire)
every time a document in a frame or an iFrame is done loading in the web
browser.

For what I need to do (and I have this working perfectly in VB6) I need to
wait an X amount of docComplete events before continuing with the rest of
the code. In VB6 this is no problem. The method I write that navigates
simply looks at a property I declared that gets upped by one every time the
doComplete fires. VB6 is lovely in its stupidity that my method will
actually halt for as long as the property is not X (Do while).

This same logic does not work in dotnet. I have a navigate method and at the
class level I declare a withevents variable. If I put a Do... While
(encapsulating the check for the property value) in my navigate method, it
never releases and the docComplete event never fires. If I take the Do...
While out, then the doComplete fires but not until my navigate method is
already finished...

Obviously I am a dotnetnewbie.... What am I doing wrong??

 

[Cor Ligthert]

VBSome

I do not know what you do wrong, however here a very simple way to use the
AXwebbrowser (the same as in VB6).

Open a new windows application project

In the toolbox rightclick and select add/Remove items

In the customize toolbox select Com and in that Microsoft Webbrowser

When that is in the toolbox drag it to your form
Drag also a button to your form.

Then this code and you have a mini Webbrowser.

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As
System.EventArgs) Handles Button1.Click
Me.AxWebBrowser1.Navigate2("www.google.com")
End Sub


I hope this helps a little bit?

Cor

 

[Tim Anderson]

This same logic does not work in dotnet. I have a navigate method and at
the class level I declare a withevents variable. If I put a Do... While
(encapsulating the check for the property value) in my navigate method, it
never releases and the docComplete event never fires. If I take the Do...
While out, then the doComplete fires but not until my navigate method is
already finished...

Try to redo your code to avoid polling for a condition in Do ... While. For
example, have the DocComplete event call a routine that increments a
counter, then takes the action when the counter reaches the right value
(though I have to say this technique sounds rather frail).

It's also worth looking at timers. A timer won't block anything in your
application, so if you can't avoid polling, have a timer do it.

Tim
..NET pros and cons
http://www.itwriting.com/phorum/list.php?f=6

 

[VBSome]

Cor:
No, sorry, that doesn't help. I know how to create a web browser. I am
actually creating an application (or rather recreating) that walks the DOM
and then can perform actions (such as "click on link", "fill in this
textbox") based on the HTML Element it is presented with. I, unfortunately,
already know all the stuff you talk about.

Tim:
That's exactly where it fails. How do I respond to an event outside the
method if I can't respond to it inside the method? Like I said, VB6 was
brilliantly stupid in that respect. I guess timers have some merit. I
should try that. As for it being "frail".... Sometimes you have no choice.

Basically what I am doing is writing a "script language" that can visually
navigate through web sites. Sometimes you have to perform an action that can
only be executed after something else happens first. I can't have the code
click on a button (or try) if that button is not loaded yet. For reasons
that might not be obvious at first we have opted to count docCompletes
rather then just trying to keep hitting a control until succes or time out.

I will try a timer based model, thanks for the responses and if anyone has
anything to add, let me know.

 

[Cor Ligthert]

VBSome,

And you know as well everything about MSHTML?
(That is the DOM)
mshtml
http://msdn.microsoft.com/library/default.asp?url=/workshop/browser/hosting/hosting.asp

When not,
Start with not to import it however reference everything that because of the
terrible amount of interfaces
Set option strict of in your program.

Beneath a piece of code as I use it.
\\\
For Each iDocument As mshtml.IHTMLDocument2 In pDocuments
For i As Integer = 0 To iDocument.all.length - 1
Dim hrefname As String
Dim hElm As mshtml.IHTMLElement = _
DirectCast(iDocument.all.item(i), mshtml.IHTMLElement)
Dim tagname As String = hElm.tagName.ToLower
///

However probably you know that as well?

Cor

 

[VBSome]

And you know as well everything about MSHTML?

(That is the DOM)
mshtml
http://msdn.microsoft.com/library/default.asp?url=/workshop/browser/hosting/hosting.asp

However probably you know that as well?

Cor, if I somehow offended you, my apologies. I do know a fair bit about
MSHTML, I have been writing web scraping tools in VB6 for 4 years. What I
would really like to know, besides from the obvious "Why did MS decide not
to make this native to dotnet" (answer: because they would have to rewrite
IE6), is how to wait inside a sub or function on an event that happens
outside of that sub or function. In VB6 (because of it not being the
marvelous language vb.net is) the shortcomings worked to my advantage. in
dotnet this seems more complicated then it has to be....

 

[Cor Ligthert]

VBSome,

If I somehow offended you, my apoligies, you did that not to me. Only
that mshtml is such a terrible thing that I could not resist writing that,
because when you know that well you should have told that. Only the ones who
should know better can offend me.

However to your message, AFAIK is One of the benefits of Net 2.0 a new
wraper around the webbrowser.

And let us not forget that IE6 is (again AFAIK) deep into the OS to avoid
double and/or conflicting handling

However with that you cannot do anything now.

Problem is that I don't know what you know and as well not what you want, so
one of the problems what can be with the webbrowser is when you are
dowloading a page with frames. It are than more documents. You see that in
my code below, where I read a document. This code I got basicly ones from
Charles when I started with the webbrowser. However I have used more events
to be sure that all is downloaded and as well a timer to prevent that there
are other errors.


eDocuments is an Arraylist.
\\\
Private Sub AxWebBrowser1_DocumentComplete(ByVal sender As System.Object, _
ByVal e As AxSHDocVw.DWebBrowserEvents2_DocumentCompleteEvent) _
Handles AxWebBrowser1.DocumentComplete
Try
Dim wb As SHDocVw.WebBrowser = DirectCast(e.pDisp,
SHDocVw.WebBrowser)
eDocuments.Add(wb.Document)
Catch ex As Exception
Dim a As String = ex.Message
End Try
End Sub
///

I hope this helps something?

Cor

 

[VBSome]

Nah, not offended at all (ik heb een dikke huid hahahahaha)

I understand the code below... but the fact of the matter is, I cannot get
anything to execute synchronously (spelling?).

I put some debug statements in my tester app and look:


Thread Start11/24/2004 9:30:43 PM <-- from a class I start a thread that
does the wb.Navigate2 inside another class

Inside the Thre. 11/24/2004 9:30:43 PM <-- here I am in the thread

The thread '<No Name>' (0x620) has exited with code 0 (0x0). <-- it exits

Almost out of Thre. 11/24/2004 9:30:43 PM <-- this is right before I leave
the sub

Thread alive? False11/24/2004 9:30:43 PM <-- back in the main class the
thread is dead OF COURSE! I just did a thread.join

Thread done11/24/2004 9:30:43 PM <-- After I do a thread.join

Num of Docs. 211/24/2004 9:30:45 PM <-- AND HERE COME THE DOC COMPLETES!!!!

Num of Docs. 111/24/2004 9:30:45 PM

Num of Docs. 311/24/2004 9:30:45 PM

Num of Docs. 211/24/2004 9:30:45 PM


Why is that??? Why do the doc completes fire after the thread exits? Why are
the doc complete events not fired while the thread is still active??

What am I missing??

Thanks for all the help!

 

[Cor Ligthert]

VBSome,

Before I go on, are you using multithreading. I use a lot of multithreading,
however especially not in this case.

It seems to me very much work to control the different docs and as well all
the wrong URL's.

So can you tell this first to me?

Cor

 

[VBSome]

Before I go on, are you using multithreading. I use a lot of

multithreading, however especially not in this case.

Yes! I am.

 

[Cor Ligthert]

What do you expect as benefit from that

"VBSome" <[email protected]

 

[JS]

On my first try (all on same thread) when I did (pseudocode follows):

wb.Navigate
Do while DocCompleteCount < X
Loop

The Navigate got executed but never returned, the loop locked up the
application never finishing the Navigate.

Then I tried a timer as Tim suggested but that had the same result.
Now that I am firing the Navigate on its own thread, the
documentcomplete event comes back AFTER the thread.join which
surprises me but must be (or so I reckon) because even though the sub
I call to navigate and the doc complete event handler are in the same
class they probably execute on different threads and not on the same
thread I use for the navigate.

The reason I can't use the code sample you provided is that I *HAVE*
to know how many complete events fire. In the work that we do, we know
that sometimes we have to wait for a certain number of docCompletes
versus ALL. It's a speed issue. If a web page has 10 frames, but the
control I need to click on appears in the second frame (or iFrame)
there is no point in waiting for all 10 of them. It's that counting
that has got me baffled. I know how to do it in VB6. But like I said,
in dotnet the same dll doesn't seem to work the same way. Most likely
this is due to my lack of knowledge of vb.net...

 

[Cor Ligthert]

JS,

That do I as well, however to use the axbrowser, I use almost all events
from it to keep it going by instance the dowloandbegin, the
downloadcomplete, the navigatecomplete2, the navigateerror, the
beforenavigate2 and even more and than still a timer. It is a lot of tunning
in my opinion, what I cannot do for you.

Cor

 

[Larry Serflaten]

On my first try (all on same thread) when I did (pseudocode follows):

wb.Navigate
Do while DocCompleteCount < X
Loop

The Navigate got executed but never returned, the loop locked up the
application never finishing the Navigate.

Did you try counting the DocumentComplete events from within the
DocumentComplete event? When you get to X number of events,
abort (Stop) the page and call a sub that has the code you need
executed.

Also I saw no mention of using DoEvents to let the WB process
its own events on your thread, but counting in the DocumentComplete
event itself would force you to remove the loop and let that sub end,
so DoEvents should not be needed (here).

LFS

 

[JS]

Did you try counting the DocumentComplete events from within the
DocumentComplete event? When you get to X number of events,
abort (Stop) the page and call a sub that has the code you need
executed.

Also I saw no mention of using DoEvents to let the WB process
its own events on your thread, but counting in the DocumentComplete
event itself would force you to remove the loop and let that sub end,
so DoEvents should not be needed (here).

LFS

My problem is that I have the navigate and documentcomplete are in
separate subs (as is usal) and that the doc.complete fires AFTER my
code in the navigate method is already done execusting.

I can't believe I am this stupid... I mean, come on, I have this
working in VB 6.... Djeeeez!

 

[Larry Serflaten]

My problem is that I have the navigate and documentcomplete are in
separate subs (as is usal) and that the doc.complete fires AFTER my
code in the navigate method is already done execusting.

I can't believe I am this stupid... I mean, come on, I have this
working in VB 6.... Djeeeez!

But this isn't VB6, and old hacks may not work in .Net. (Waiting
in a loop for a variable to change is a hack for 'event driven' methodology)

Re-code your solution to let the navigate sub end, and call whatever
else you need done from the event that tests when it is time to continue
processing....

LFS

 

[JS]

Thanks!
I have it working finally.... In a way that halfway makes sense
It involves a timer! It's even better then I had expected. With this
hurdle out of the way I think I can quickly come up with the rest of
the code.... I only need to code 57 more methods of performing an
action in a Browser