Open a URL, fill in fields and submit using .net code

G

Guest

Hi,

I want to write a simple .net program to open a URL, fill in fields, and
click on a button to submit it using .net 1.1 framework.

Can someone help in suggesting the libraries I should use?

I tried using javascript, however, I am not able to make javascript wait for
the page to completely load, before trying to access and fill fields on the
page. Hence resorted to .net, but not finding the right library to use.

Please suggest.
 
G

Guest

Hi there

The reference to shdocvw was definitely useful. However I am running into
further problems - the InternetExplorerClass / WebBrowser objects are not
calling the even handlers registered for NavigateComplete / NavigateComplete2
events.
In the code below ie_NavigateComplete is not getting called.

Here is a relevant part of my code:
static void Main(string[] args)
{
MainNeoLister n = new MainNeoLister();

}
InternetExplorerClass ie;
MainNeoLister()
{
ie = new InternetExplorerClass();

object flags = null;
object targetFrameName = "_BLANK";
object postData = null;
object headers = null;
object url = "http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx";

ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx",
ref flags, ref targetFrameName, ref postData, ref headers);
ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx?catalog=game&itemID=new",
ref flags, ref targetFrameName, ref postData, ref headers);
ie.NavigateComplete +=new
DWebBrowserEvents_NavigateCompleteEventHandler(ie_NavigateComplete);

Console.ReadLine();
}

private void ie_NavigateComplete(string URL)
{
IHTMLDocument3 doc = (IHTMLDocument3)ie.Document;
doc.getElementById("ctl00_cphContentSection_itemDetailsForm_txtName101").innerText = "NewName";
Console.WriteLine(doc.getElementById("ctl00_cphContentSection_itemDetailsForm_hasItemChanged").innerText);
}

What can be the problem why ie_NavigateComplete is not getting called?
--
Thanks,
Sameeksha
MCAD.Net


Dave Sexton said:
Hi Sameeksha,

You can embed IE in your application and work with that. Add a reference to
SHDocVw.dll from the COM tab and read the following article for some
guidance:

Reusing the WebBrowser Control
http://msdn.microsoft.com/library/default.asp?url=/workshop/browser/webbrowser/WebBrowser.asp

(In the 2.0 framework use the new managed WebBrowser control instead)
 
G

Guest

Please read the code for this example as given below. (Ignore the code in
previous comment)
static void Main(string[] args)
{
MainNeoLister n = new MainNeoLister();

}
InternetExplorerClass ie;
MainNeoLister()
{
ie = new InternetExplorerClass();

object flags = null;
object targetFrameName = "_BLANK";
object postData = null;
object headers = null;
object url = "http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx";

ie.NavigateComplete +=new
DWebBrowserEvents_NavigateCompleteEventHandler(ie_NavigateComplete);
ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx",
ref flags, ref targetFrameName, ref postData, ref headers);
ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx?catalog=game&itemID=new",
ref flags, ref targetFrameName, ref postData, ref headers);


Console.ReadLine();
}

private void ie_NavigateComplete(string URL)
{
if (URL ==
"http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx?catalog=game&itemID=new")
{
IHTMLDocument3 doc = (IHTMLDocument3)ie.Document
doc.getElementById("ctl00_cphContentSection_itemDetailsForm_txtName101").innerText = "NewName"
doc.getElementById Console.WriteLine(doc.getElementById("ctl00_cphContentSection_itemDetailsForm_hasItemChanged").innerText);
}
}
--
Thanks,
Sameeksha
MCAD.Net


Sameeksha said:
Hi there

The reference to shdocvw was definitely useful. However I am running into
further problems - the InternetExplorerClass / WebBrowser objects are not
calling the even handlers registered for NavigateComplete / NavigateComplete2
events.
In the code below ie_NavigateComplete is not getting called.

Here is a relevant part of my code:
static void Main(string[] args)
{
MainNeoLister n = new MainNeoLister();

}
InternetExplorerClass ie;
MainNeoLister()
{
ie = new InternetExplorerClass();

object flags = null;
object targetFrameName = "_BLANK";
object postData = null;
object headers = null;
object url = "http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx";

ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx",
ref flags, ref targetFrameName, ref postData, ref headers);
ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx?catalog=game&itemID=new",
ref flags, ref targetFrameName, ref postData, ref headers);
ie.NavigateComplete +=new
DWebBrowserEvents_NavigateCompleteEventHandler(ie_NavigateComplete);

Console.ReadLine();
}

private void ie_NavigateComplete(string URL)
{
IHTMLDocument3 doc = (IHTMLDocument3)ie.Document;
doc.getElementById("ctl00_cphContentSection_itemDetailsForm_txtName101").innerText = "NewName";
Console.WriteLine(doc.getElementById("ctl00_cphContentSection_itemDetailsForm_hasItemChanged").innerText);
}

What can be the problem why ie_NavigateComplete is not getting called?
 
D

Dave Sexton

Hi,

The Navigate method is not synchronous, IIRC. Calling it twice in a row
will just be wasteful.

I'm not sure why the NavigateComplete event isn't working though. It could
have something to do with the browser not being visible. Have you tried
using the DocumentComplete event, which is probably better if you need to
parse the document anyway?

An alternative to using an invisible browser to submit a form is to use the
WebClient class to make the request and then parse the HTML yourself,
possibly using Regular Expressions depending upon the complexity of your
needs. If the source is valid XHTML then you can load it into an
XmlDocument as a fragment and use XPath to find the interesting fields
instead of parsing the document via patterns. The latter would be the
preferred approach if it's something you can use, IMO.

--
Dave Sexton
http://davesexton.com/blog

Sameeksha said:
Please read the code for this example as given below. (Ignore the code in
previous comment)
static void Main(string[] args)
{
MainNeoLister n = new MainNeoLister();

}
InternetExplorerClass ie;
MainNeoLister()
{
ie = new InternetExplorerClass();

object flags = null;
object targetFrameName = "_BLANK";
object postData = null;
object headers = null;
object url = "http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx";

ie.NavigateComplete +=new
DWebBrowserEvents_NavigateCompleteEventHandler(ie_NavigateComplete);
ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx",
ref flags, ref targetFrameName, ref postData, ref headers);
ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx?catalog=game&itemID=new",
ref flags, ref targetFrameName, ref postData, ref headers);


Console.ReadLine();
}

private void ie_NavigateComplete(string URL)
{
if (URL ==
"http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx?catalog=game&itemID=new")
{
IHTMLDocument3 doc = (IHTMLDocument3)ie.Document;
doc.getElementById("ctl00_cphContentSection_itemDetailsForm_txtName101").innerText
= "NewName";
doc.getElementById
Console.WriteLine(doc.getElementById("ctl00_cphContentSection_itemDetailsForm_hasItemChanged").innerText);
}
}
--
Thanks,
Sameeksha
MCAD.Net


Sameeksha said:
Hi there

The reference to shdocvw was definitely useful. However I am running into
further problems - the InternetExplorerClass / WebBrowser objects are not
calling the even handlers registered for NavigateComplete /
NavigateComplete2
events.
In the code below ie_NavigateComplete is not getting called.

Here is a relevant part of my code:
static void Main(string[] args)
{
MainNeoLister n = new MainNeoLister();

}
InternetExplorerClass ie;
MainNeoLister()
{
ie = new InternetExplorerClass();

object flags = null;
object targetFrameName = "_BLANK";
object postData = null;
object headers = null;
object url =
"http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx";

ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx",
ref flags, ref targetFrameName, ref postData, ref headers);
ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx?catalog=game&itemID=new",
ref flags, ref targetFrameName, ref postData, ref headers);
ie.NavigateComplete +=new
DWebBrowserEvents_NavigateCompleteEventHandler(ie_NavigateComplete);

Console.ReadLine();
}

private void ie_NavigateComplete(string URL)
{
IHTMLDocument3 doc = (IHTMLDocument3)ie.Document;
doc.getElementById("ctl00_cphContentSection_itemDetailsForm_txtName101").innerText
= "NewName";
Console.WriteLine(doc.getElementById("ctl00_cphContentSection_itemDetailsForm_hasItemChanged").innerText);
}

What can be the problem why ie_NavigateComplete is not getting called?
--
Thanks,
Sameeksha
MCAD.Net


Dave Sexton said:
Hi Sameeksha,

You can embed IE in your application and work with that. Add a
reference to
SHDocVw.dll from the COM tab and read the following article for some
guidance:

Reusing the WebBrowser Control
http://msdn.microsoft.com/library/default.asp?url=/workshop/browser/webbrowser/WebBrowser.asp

(In the 2.0 framework use the new managed WebBrowser control instead)

--
Dave Sexton
http://davesexton.com/blog

Hi,

I want to write a simple .net program to open a URL, fill in fields,
and
click on a button to submit it using .net 1.1 framework.

Can someone help in suggesting the libraries I should use?

I tried using javascript, however, I am not able to make javascript
wait
for
the page to completely load, before trying to access and fill fields
on
the
page. Hence resorted to .net, but not finding the right library to
use.

Please suggest.
 
G

Guest

Thanks, Dave.

I tried using DocumentComplete event, however it also does not succeed.
However the point to note is that it opens a new IE7 instance and navigates
to the 2 URLs correctly.

Now I will try implementing the WebClient class as you suggested. In the
meantime here is the code with documentcomplete event which is not working:
MainNeoLister()
{
ie = new InternetExplorerClass();

object flags = null;
object targetFrameName = "_BLANK";
object postData = null;
object headers = null;
object url = "http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx";

ie.DocumentComplete += new
DWebBrowserEvents2_DocumentCompleteEventHandler(ie_DocumentComplete)
ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx", ref flags, ref targetFrameName, ref postData, ref headers);
ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx?catalog=game&itemID=new",
ref flags, ref targetFrameName, ref postData, ref headers);

f = new StreamWriter(@"c:\neolister-output.txt", false);
Console.ReadLine();
}

private void ie_DocumentComplete(object pDisp, ref object URL)
{
f.Write(URL);
if ((string)URL ==
"http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx?catalog=game&itemID=new")
{
IHTMLDocument3 doc = (IHTMLDocument3)ie.Document
doc.getElementById("ctl00_cphContentSection_itemDetailsForm_txtName101").innerText = "NewName"
f.Write(doc.getElementById("ctl00_cphContentSection_itemDetailsForm_hasItemChanged").innerText);
f.Close();
}
}

--
Thanks,
Sameeksha
MCAD.Net


Dave Sexton said:
Hi,

The Navigate method is not synchronous, IIRC. Calling it twice in a row
will just be wasteful.

I'm not sure why the NavigateComplete event isn't working though. It could
have something to do with the browser not being visible. Have you tried
using the DocumentComplete event, which is probably better if you need to
parse the document anyway?

An alternative to using an invisible browser to submit a form is to use the
WebClient class to make the request and then parse the HTML yourself,
possibly using Regular Expressions depending upon the complexity of your
needs. If the source is valid XHTML then you can load it into an
XmlDocument as a fragment and use XPath to find the interesting fields
instead of parsing the document via patterns. The latter would be the
preferred approach if it's something you can use, IMO.

--
Dave Sexton
http://davesexton.com/blog

Sameeksha said:
Please read the code for this example as given below. (Ignore the code in
previous comment)
static void Main(string[] args)
{
MainNeoLister n = new MainNeoLister();

}
InternetExplorerClass ie;
MainNeoLister()
{
ie = new InternetExplorerClass();

object flags = null;
object targetFrameName = "_BLANK";
object postData = null;
object headers = null;
object url = "http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx";

ie.NavigateComplete +=new
DWebBrowserEvents_NavigateCompleteEventHandler(ie_NavigateComplete);
ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx",
ref flags, ref targetFrameName, ref postData, ref headers);
ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx?catalog=game&itemID=new",
ref flags, ref targetFrameName, ref postData, ref headers);


Console.ReadLine();
}

private void ie_NavigateComplete(string URL)
{
if (URL ==
"http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx?catalog=game&itemID=new")
{
IHTMLDocument3 doc = (IHTMLDocument3)ie.Document;
doc.getElementById("ctl00_cphContentSection_itemDetailsForm_txtName101").innerText
= "NewName";
doc.getElementById
Console.WriteLine(doc.getElementById("ctl00_cphContentSection_itemDetailsForm_hasItemChanged").innerText);
}
}
--
Thanks,
Sameeksha
MCAD.Net


Sameeksha said:
Hi there

The reference to shdocvw was definitely useful. However I am running into
further problems - the InternetExplorerClass / WebBrowser objects are not
calling the even handlers registered for NavigateComplete /
NavigateComplete2
events.
In the code below ie_NavigateComplete is not getting called.

Here is a relevant part of my code:
static void Main(string[] args)
{
MainNeoLister n = new MainNeoLister();

}
InternetExplorerClass ie;
MainNeoLister()
{
ie = new InternetExplorerClass();

object flags = null;
object targetFrameName = "_BLANK";
object postData = null;
object headers = null;
object url =
"http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx";

ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx",
ref flags, ref targetFrameName, ref postData, ref headers);
ie.Navigate("http://ng-neo-srv1/neoedge.neolister/Secure/MyContent.aspx?catalog=game&itemID=new",
ref flags, ref targetFrameName, ref postData, ref headers);
ie.NavigateComplete +=new
DWebBrowserEvents_NavigateCompleteEventHandler(ie_NavigateComplete);

Console.ReadLine();
}

private void ie_NavigateComplete(string URL)
{
IHTMLDocument3 doc = (IHTMLDocument3)ie.Document;
doc.getElementById("ctl00_cphContentSection_itemDetailsForm_txtName101").innerText
= "NewName";
Console.WriteLine(doc.getElementById("ctl00_cphContentSection_itemDetailsForm_hasItemChanged").innerText);
}

What can be the problem why ie_NavigateComplete is not getting called?
--
Thanks,
Sameeksha
MCAD.Net


:

Hi Sameeksha,

You can embed IE in your application and work with that. Add a
reference to
SHDocVw.dll from the COM tab and read the following article for some
guidance:

Reusing the WebBrowser Control
http://msdn.microsoft.com/library/default.asp?url=/workshop/browser/webbrowser/WebBrowser.asp

(In the 2.0 framework use the new managed WebBrowser control instead)

--
Dave Sexton
http://davesexton.com/blog

Hi,

I want to write a simple .net program to open a URL, fill in fields,
and
click on a button to submit it using .net 1.1 framework.

Can someone help in suggesting the libraries I should use?

I tried using javascript, however, I am not able to make javascript
wait
for
the page to completely load, before trying to access and fill fields
on
the
page. Hence resorted to .net, but not finding the right library to
use.

Please suggest.
 
D

Dave Sexton

Hi,

I didn't realize that it was opening IE - I thought it remained hidden. If
you require IE to be visible then WebClient won't help you at all since it's
not associated with a browser.

I tried your code, with some slight modifications to get it to build, and
none of the events were raised for me either. I tested with IE7,
exclusively.

The fix shown in the following article didn't work either, but it's
interesting to note and supplies some code that supposedly fixes a problem
that seems similar to what you are experiencing:

BUG: The BeforeNavigate2 Event of WebBrowser Control Does Not Fire If Hosted
in a Visual C# .NET Application
http://support.microsoft.com/kb/325079/EN-US/

I tested my own version of the code found in the article above using the
DWebBrowserEvents2 interface instead of DWebBrowserEvents. The code appears
after my signature. Some of the events are raised, so I suspect that this
may have something to do with the fact that IE7 uses tabs now but I'm not
sure how to solve the problem. Maybe my code will give you somewhere to
start :)

--
Dave Sexton
http://davesexton.com/blog

class Program : Component, DWebBrowserEvents2
{
private InternetExplorerClass ie;
private IConnectionPoint icp;
private int cookie = -1;

static void Main(string[] args)
{
Program program = new Program();
program.Run();
}

private void Run()
{
ie = new SHDocVw.InternetExplorerClass();

IConnectionPointContainer icpc = (IConnectionPointContainer) ie;

Guid g = typeof(DWebBrowserEvents2).GUID;
icpc.FindConnectionPoint(ref g, out icp);
icp.Advise(this, out cookie);

object flags = null;
object targetFrameName = "_BLANK";
object postData = null;
object headers = null;
object url = "http://localhost:1652/NewsGroupsWebsite/Default.aspx";

ie.Navigate2(ref url, ref flags, ref targetFrameName, ref postData, ref
headers);
}

protected override void Dispose(bool disposing)
{
if (disposing)
{
if (-1 != cookie)
{
try
{
icp.Unadvise(cookie);
}
catch (COMException ex)
{
Trace.WriteLine("Unadvise error: " + Environment.NewLine +
ex.ToString());
}
}

cookie = -1;
}

base.Dispose(disposing);
}

#region DWebBrowserEvents2 Members
void DWebBrowserEvents2.BeforeNavigate2(object pDisp, ref object URL, ref
object Flags, ref object TargetFrameName, ref object PostData, ref object
Headers, ref bool Cancel)
{
Console.WriteLine("BeforeNavigate2");
}

void DWebBrowserEvents2.ClientToHostWindow(ref int CX, ref int CY)
{
Console.WriteLine("ClientToHostWindow");
}

void DWebBrowserEvents2.CommandStateChange(int Command, bool Enable)
{
Console.WriteLine("CommandStateChange");
}

void DWebBrowserEvents2.DocumentComplete(object pDisp, ref object URL)
{
Console.WriteLine("DocumentComplete");
}

void DWebBrowserEvents2.DownloadBegin()
{
Console.WriteLine("DownloadBegin");
}

void DWebBrowserEvents2.DownloadComplete()
{
Console.WriteLine("DownloadComplete");
}

void DWebBrowserEvents2.FileDownload(bool ActiveDocument, ref bool Cancel)
{
Console.WriteLine("FileDownload");
}

void DWebBrowserEvents2.NavigateComplete2(object pDisp, ref object URL)
{
Console.WriteLine("NavigateComplete2");
}

void DWebBrowserEvents2.NavigateError(object pDisp, ref object URL, ref
object Frame, ref object StatusCode, ref bool Cancel)
{
Console.WriteLine("NavigateError");
}

void DWebBrowserEvents2.NewWindow2(ref object ppDisp, ref bool Cancel)
{
Console.WriteLine("NewWindow2");
}

void DWebBrowserEvents2.NewWindow3(ref object ppDisp, ref bool Cancel, uint
dwFlags, string bstrUrlContext, string bstrUrl)
{
Console.WriteLine("NewWindow3");
}

void DWebBrowserEvents2.OnFullScreen(bool FullScreen)
{
Console.WriteLine("OnFullScreen");
}

void DWebBrowserEvents2.OnMenuBar(bool MenuBar)
{
Console.WriteLine("OnMenuBar");
}

void DWebBrowserEvents2.OnQuit()
{
Console.WriteLine("OnQuit");
}

void DWebBrowserEvents2.OnStatusBar(bool StatusBar)
{
Console.WriteLine("OnStatusBar");
}

void DWebBrowserEvents2.OnTheaterMode(bool TheaterMode)
{
Console.WriteLine("OnTheaterMode");
}

void DWebBrowserEvents2.OnToolBar(bool ToolBar)
{
Console.WriteLine("OnToolBar");
}

void DWebBrowserEvents2.OnVisible(bool Visible)
{
Console.WriteLine("OnVisible");
}

void DWebBrowserEvents2.PrintTemplateInstantiation(object pDisp)
{
Console.WriteLine("PrintTemplateInstantiation");
}

void DWebBrowserEvents2.PrintTemplateTeardown(object pDisp)
{
Console.WriteLine("PrintTemplateTeardown");
}

void DWebBrowserEvents2.PrivacyImpactedStateChange(bool bImpacted)
{
Console.WriteLine("PrivacyImpactedStateChange");
}

void DWebBrowserEvents2.ProgressChange(int Progress, int ProgressMax)
{
Console.WriteLine("ProgressChange");
}

void DWebBrowserEvents2.PropertyChange(string szProperty)
{
Console.WriteLine("PropertyChange");
}

void DWebBrowserEvents2.SetPhishingFilterStatus(int PhishingFilterStatus)
{
Console.WriteLine("SetPhishingFilterStatus");
}

void DWebBrowserEvents2.SetSecureLockIcon(int SecureLockIcon)
{
Console.WriteLine("SetSecureLockIcon");
}

void DWebBrowserEvents2.StatusTextChange(string Text)
{
Console.WriteLine("StatusTextChange");
}

void DWebBrowserEvents2.TitleChange(string Text)
{
Console.WriteLine("TitleChange");
}

void DWebBrowserEvents2.UpdatePageStatus(object pDisp, ref object nPage,
ref object fDone)
{
Console.WriteLine("UpdatePageStatus");
}

void DWebBrowserEvents2.WindowClosing(bool IsChildWindow, ref bool Cancel)
{
Console.WriteLine("WindowClosing");
}

void DWebBrowserEvents2.WindowSetHeight(int Height)
{
Console.WriteLine("WindowSetHeight");
}

void DWebBrowserEvents2.WindowSetLeft(int Left)
{
Console.WriteLine("WindowSetLeft");
}

void DWebBrowserEvents2.WindowSetResizable(bool Resizable)
{
Console.WriteLine("WindowSetResizable");
}

void DWebBrowserEvents2.WindowSetTop(int Top)
{
Console.WriteLine("WindowSetTop");
}

void DWebBrowserEvents2.WindowSetWidth(int Width)
{
Console.WriteLine("WindowSetWidth");
}

void DWebBrowserEvents2.WindowStateChanged(uint dwWindowStateFlags, uint
dwValidFlagsMask)
{
Console.WriteLine("WindowStateChanged");
}
#endregion
}
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top