XML and WebProxy

[Andrew]

Morning everybody,

I have an XML document located on a web address and I wish to create a
C# program to read that XML document and process it - which shouldn't
be that difficult yet I'm running into all kinds of problems because
I'm behind a firewall.

After much searching I found using WebProxy, NetworkCredentials and
HttpWebResponse/Request to be quite useful but again it's not really
working the way I'd like. So I decided to setup a prototype using the
three classes above so I could get them working.

For example...

HttpWebRequest HttpWReq =
(HttpWebRequest)WebRequest.Create("http://homepage.com/");
WebProxy objProxy = new WebProxy("1.2.3.4");
NetworkCredential myCred = new NetworkCredential("user", "pass",
"domain");
// a few more lines of code here
Console.WriteLine(HttpWebResp.StatusDescription);

which outputs "OK" and that's fantastic - at least I know my proxy is
setup right.

But if I change the WebRequest.Create from "http://homepage.com" to
"http://homepage.com/filename.xml" it generates a 404 error; but the
file is there.

Any ideas?

 

[Joerg Jooss]

Thus wrote Andrew,

Morning everybody,

I have an XML document located on a web address and I wish to create a
C# program to read that XML document and process it - which shouldn't
be that difficult yet I'm running into all kinds of problems because
I'm behind a firewall.

After much searching I found using WebProxy, NetworkCredentials and
HttpWebResponse/Request to be quite useful but again it's not really
working the way I'd like. So I decided to setup a prototype using the
three classes above so I could get them working.

Did you have a look at WebClient? It may make your life a lot easier compared
to what you need to do with HttpWebRequest.

For example...

HttpWebRequest HttpWReq =
(HttpWebRequest)WebRequest.Create("http://homepage.com/");
WebProxy objProxy = new WebProxy("1.2.3.4");
NetworkCredential myCred = new NetworkCredential("user", "pass",
"domain");
// a few more lines of code here
Console.WriteLine(HttpWebResp.StatusDescription);
which outputs "OK" and that's fantastic - at least I know my proxy is
setup right.

But if I change the WebRequest.Create from "http://homepage.com" to
"http://homepage.com/filename.xml" it generates a 404 error; but the
file is there.

Any ideas?

Are you sure that the file isn't protected or created dynamically by some
web application? Can you download it using a browser?

Cheers,

 

[Andrew]

Hey, thanks for the response!

The test file is just a bog standard xml file but I would like to hope
that I could use a dynamically generated xml file - does it matter in
the future if it is? But yeah, I can access the file via browser. I'll
check how WebClient etc works in the morning when I'm back at my labs
computer.

* Note to self: Look at WebClient class.

Is there anything else you could suggest?

Andy

 

[Joerg Jooss]

Thus wrote Andrew,

Hey, thanks for the response!

The test file is just a bog standard xml file but I would like to hope
that I could use a dynamically generated xml file - does it matter in
the future if it is?

Only if the web application that creates that file requires authentication,
requires some sort of session tracked by cookies, or wants to identify your
browser... in other word, if it requires some piece of information who or
what the client is.

But yeah, I can access the file via browser. I'll
check how WebClient etc works in the morning when I'm back at my labs
computer.

* Note to self: Look at WebClient class.

Is there anything else you could suggest?

If all fails, use Fiddler to compare the HTTP traffic between your application
and the server to the one between your browser and the server.

Cheers,

 

[Andrew]

Hi Joerg,

I tried using WebClient and I found something out probably to do with
internal settings. Like I originally said, I was having trouble reading
any file that wasn't a domain name. It turns out using
WebClient.OpenRead and Stream/StreamReader classes that I'm not even
connecting to the domain name.

Instead what I'm getting is some kind of error page saying the website
has not been constructed.

<HTML>
blah blah..
The site you were trying to reach does not currently have a default
page. It may be in the process of being upgraded.<br><br>
Please try this site again later. If you still experience the problem,
try contacting the website administrator.
blah blah
</HTML>

I'm beginning to think it's more of a configuration problem with IIS on
the server machine, or a problem with the proxy.

Can you add any further suggestions? At least WebClient was able to
help track the error better.

Andy