XmlDocument.Load() Protocol

[Jonathan Wood]

I'm using XmlDocument.Load to retrieve a PAD (XML) file. My code looks
something like this:

XmlDocument xml = new XmlDocument();

// Non-HTTP prefix not supported
if (!url.StartsWith("http://", StringComparison.OrdinalIgnoreCase))
throw new Exception("PAD URL contains an unsupported protocol
prefix");

// Load XmlDocument
xml.Load(url);

When url is set to "http://www.mobiletutor.org/guitar/jar.xml", I get the
following error:

The remote server returned an error: (400) Bad Request.

I notified this site about the error and here is their response:

"FYI, this is failing because your file fetcher is not sending HTTP
Accept headers and is thus considered insecure by our Web server. You
may want to fix that..."

I don't fully understand this response but, sure enough, if I load the URL
above using Internet Explorer, it appears to work just fine.

Does XmlDocument have some protocol problems, or is there something I can do
to make my code work in this case?

Thanks for any tips.

 

[Peter Duniho]

[...]
I notified this site about the error and here is their response:

"FYI, this is failing because your file fetcher is not sending HTTP
Accept headers and is thus considered insecure by our Web server. You
may want to fix that..."

I don't fully understand this response but, sure enough, if I load the
URL above using Internet Explorer, it appears to work just fine.

Does XmlDocument have some protocol problems, or is there something I
can do to make my code work in this case?

I'm not clear on why not including an Accept header is a security
problem, but that's probably just lack of knowledge on my part (though a
quick Google didn't turn up anything).

The way the response describes it, "Accept headers" plural, suggests
maybe it's not the Accept header per se, but one of the variants (e.g.
Accept-Charset, Accept-Encoding, etc....and not that that assumption
makes the insecurity potential any more obvious to me ) that they
consider important. You may want to ask them to clarify, so that you
know _exactly_ what header you need to include and what the value for
the header should be.

As for dealing with the requirement, AFAIK there's nothing in
XmlDocument you can use to force the header to be included when calling
Load() with a URL. But, you can use the WebRequest class to resolve
your URL, and for HttpWebRequest instances, include the necessary header
(if it's not by default...I don't recall which headers are and it's not
entirely clear exactly what header is needed).

With the WebRequest instance in hand, you then just get the response,
and from that, the response stream. You can then pass the Stream
instance to the XmlDocument.Load() method instead.

Pete

 

[Tim Williams]

If you follow Peter's suggestion and you need to determine which headers to
add, you can use Fiddler to monitor the headers sent by your browser
(assuming that is able to successfully fetch the file).

http://www.fiddler2.com/fiddler2/ (this is a must have tool if you're
doing anything web-related)

Tim

 

[Jonathan Wood]

The way the response describes it, "Accept headers" plural, suggests maybe
it's not the Accept header per se, but one of the variants (e.g.
Accept-Charset, Accept-Encoding, etc....and not that that assumption makes
the insecurity potential any more obvious to me ) that they consider
important. You may want to ask them to clarify, so that you know
_exactly_ what header you need to include and what the value for the
header should be.

And what seems really odd to me is that his email caused Windows Live Mail
to show some sort of security warning that I had to override in order to see
the message (I eventually just looked at the message source and did not view
the message normally.)

As for dealing with the requirement, AFAIK there's nothing in XmlDocument
you can use to force the header to be included when calling Load() with a
URL. But, you can use the WebRequest class to resolve your URL, and for
HttpWebRequest instances, include the necessary header (if it's not by
default...I don't recall which headers are and it's not entirely clear
exactly what header is needed).

With the WebRequest instance in hand, you then just get the response, and
from that, the response stream. You can then pass the Stream instance to
the XmlDocument.Load() method instead.

Yeah, unfortunately this is not an area I'm strong in. And this is the first
time I've seen this complaint so I'll probably Google it a bit more and just
drop it if I don't find anything more specific. (I have no idea what headers
to add.) And, because of the strange warning with his email, I'm not certain
it's worth even writing him back either.

Thanks.

Jonathan