Authentication not working on HTTP-POST using NetworkCredential

[Patrick Fogarty]

I am programming what is to be a web service client that will use an
HTTP-POST to request and retrieve data. The remote server (written in java
for what it's worth) requires basic authentication as per RFC 2617
(http://www.faqs.org/rfcs/rfc2617.html). My attempts to authenticate are
failing. The server requires the header to be present with the request.
For security reasons, it will not reply in any way if the header is not
present.

More specifically, my attempts fail when attempting to attach a
'NetworkCredential' object to the 'Credentials' property of a
'HttpWebRequest' object. If I create the header manually, everything works
fine. When attempting to do it 'the Microsoft Way' no authentication
information is sent in the header, even if I set 'PreAuthenticate' = true.

What am I missing? Below are two examples. Each has the code to send the
request followed by the captured request header.


- Patrick

------------------------------------------------------------
<< the code that fails >>

(( assume reqBytes and SomeURI already set ))

request = (HttpWebRequest) WebRequest.Create(SomeURI);

request.PreAuthenticate = true;
request.Credentials = new NetworkCredential("JoeBlow","MountainHo");

request.Timeout = 20 * 1000;
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = reqBytes.Length;

Stream reqStream = request
reqStream.Write(reqBytes,0,reqBytes.Length);
reqStream.Close();

------------------------------
POST / HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Content-Length: 1718
Expect: 100-continue
Connection: Keep-Alive
Host: me:10000



------------------------------------------------------------
<< the code that works>>

(( assume reqBytes and SomeURI already set ))

request = (HttpWebRequest) WebRequest.Create(SomeURI);

// 'GetManualAuthorization' written by me to generate RFC2617-compliant
basic authentication header
request.Headers.Add("Authorization", GetManualAuthorization("JoeBlow",
"MountainHo"));


request.Timeout = 20 * 1000;
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = reqBytes.Length;

Stream reqStream = request
reqStream.Write(reqBytes,0,reqBytes.Length);
reqStream.Close();

------------------------------
POST / HTTP/1.1
Authorization: BASIC Sm9lQmxvdzpNb3VudGFpbkhv
Content-Type: application/x-www-form-urlencoded
Content-Length: 1718
Expect: 100-continue
Connection: Keep-Alive
Host: me:10000

 

[Peter Huang [MSFT]]

Hi Patrick,

The reason you are not seeing the credentials passed on the
inital request to the web server is because Microsoft is following
section 2 of RFC 2617(http://www.faqs.org/rfcs/rfc2617.html)

Here’s the main benefit of using pre-authenticate. Suppose I’m going to
make 50
requests to <http://server/path/> and this URL is protected with Basic
authentication. On the first request, the client gets challenged by the
server and
sends back a second request which contains information that the server
accepts
(assuming auth succeeds) so it can send back the requested resource.
With the pre-authenticate property set to true:
The remaining 49 requests will include the authorization information in the
first
request they send to the server so the server will not challenge the client
and
force it to do another round trip before getting the resource.
The total number of roundtrips between client and server will be 51.
With the pre-authenticate property set to false:
The remaining 49 requests will not include the authorization information in
the
first request and will therefore be challenged by the server on each first
request
and will only get the desired resource after sending the authorization
header in
the second request.
The total number of roundtrips between client and server will be 100.
In other words, pre-authenticate=true is one request shy of taking half the
time of
pre-authenticate=false. Note that pre-authentication only works for Digest
and
Basic in v1.0. It can’t work for NTLM because it is connection-based
however the
fact that it is connection based means that you’ll only get challenged once
per
connection so it isn’t an issue if you are caching connections. In the
Whidbey
release of the .NET Framework we’ll also support pre-authentication for
Kerberos.

In order to get the inital request to send credentials, you will need to
use the
workaround of overriding the GetWebRequest method in the proxy code.

(Hack code obtained from the Internet)
The PreAuthenticate property on .NET's
System.Web.Services.Protocols.SoapHttpClientProtocol is supposed to force
the SOAP
client proxy to send credentials with the first request, rather than doing
the
challenge/response exchange. If you add the following code to your SOAP
Client
proxy, you can make PreAuthenticate work (this example is for basic
authentication):
protected override System.Net.WebRequest
GetWebRequest(Uri uri) {
System.Net.HttpWebRequest request =
(System.Net.HttpWebRequest)base.GetWebRequest(uri);
if (this.PreAuthenticate) {
System.Net.NetworkCredential nc =
this.Credentials.GetCredential(uri,"Basic");
if (nc != null) {
byte[] credBuf =
new System.Text.UTF8Encoding().
GetBytes(nc.UserName + ":" + nc.Password);
request.Headers["Authorization"] =
"Basic " + Convert.ToBase64String(credBuf);
}
}
return request;
}

This work around modifies the web service proxy class which is
automatically generated. This means every time someone updates a "web
reference" in
Dev Studio, they would need to reinsert the "hack" code.

Let me know if you have any questions or conerns.

Regards,
Peter Huang
Microsoft Online Partner Support
Get Secure! www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.
--------------------

 

[Patrick Fogarty]

Peter -

I want to thank you for that thorough response.

I kind of suspected that was the case. Sometimes you have to read a RFC a
few hundred times to translate from theory to practical use. I had made
mention (to the authors of the server) that no challenge was being issued.
Unfortunately, especially in the industry that I am in, not responding (and
just closing the connection) in the absence of proper credentials is very
common. It prevents an accidental or deliberate probe of a URL from
divulging information that can be used to mount a subsequent attack.

The hack that you included below is similar to the one I did myself. I
merely put it in a method of a utility class rather than one of a derived
class.


- Patrick




| Hi Patrick,
|
| The reason you are not seeing the credentials passed on the
| inital request to the web server is because Microsoft is following
| section 2 of RFC 2617(http://www.faqs.org/rfcs/rfc2617.html)
|
| Here’s the main benefit of using pre-authenticate. Suppose I’m going to
| make 50
| requests to <http://server/path/> and this URL is protected with Basic
| authentication. On the first request, the client gets challenged by the
| server and
| sends back a second request which contains information that the server
| accepts
| (assuming auth succeeds) so it can send back the requested resource.
| With the pre-authenticate property set to true:
| The remaining 49 requests will include the authorization information in
the
| first
| request they send to the server so the server will not challenge the
client
| and
| force it to do another round trip before getting the resource.
| The total number of roundtrips between client and server will be 51.
| With the pre-authenticate property set to false:
| The remaining 49 requests will not include the authorization information
in
| the
| first request and will therefore be challenged by the server on each first
| request
| and will only get the desired resource after sending the authorization
| header in
| the second request.
| The total number of roundtrips between client and server will be 100.
| In other words, pre-authenticate=true is one request shy of taking half
the
| time of
| pre-authenticate=false. Note that pre-authentication only works for Digest
| and
| Basic in v1.0. It can’t work for NTLM because it is connection-based
| however the
| fact that it is connection based means that you’ll only get challenged
once
| per
| connection so it isn’t an issue if you are caching connections. In the
| Whidbey
| release of the .NET Framework we’ll also support pre-authentication for
| Kerberos.
|
| In order to get the inital request to send credentials, you will need to
| use the
| workaround of overriding the GetWebRequest method in the proxy code.
|
| (Hack code obtained from the Internet)
| The PreAuthenticate property on .NET's
| System.Web.Services.Protocols.SoapHttpClientProtocol is supposed to force
| the SOAP
| client proxy to send credentials with the first request, rather than doing
| the
| challenge/response exchange. If you add the following code to your SOAP
| Client
| proxy, you can make PreAuthenticate work (this example is for basic
| authentication):
| protected override System.Net.WebRequest
| GetWebRequest(Uri uri) {
| System.Net.HttpWebRequest request =
| (System.Net.HttpWebRequest)base.GetWebRequest(uri);
| if (this.PreAuthenticate) {
| System.Net.NetworkCredential nc =
| this.Credentials.GetCredential(uri,"Basic");
| if (nc != null) {
| byte[] credBuf =
| new System.Text.UTF8Encoding().
| GetBytes(nc.UserName + ":" + nc.Password);
| request.Headers["Authorization"] =
| "Basic " + Convert.ToBase64String(credBuf);
| }
| }
| return request;
| }
|
| This work around modifies the web service proxy class which is
| automatically generated. This means every time someone updates a "web
| reference" in
| Dev Studio, they would need to reinsert the "hack" code.
|
| Let me know if you have any questions or conerns.
|
| Regards,
| Peter Huang
| Microsoft Online Partner Support
| Get Secure! www.microsoft.com/security
| This posting is provided "as is" with no warranties and confers no rights.

 

[Peter Huang [MSFT]]

Hi Patrick,

I am glad that you have gotten a workaround yourself.


Regards,
Peter Huang
Microsoft Online Partner Support
Get Secure! www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.
--------------------

From: "Patrick Fogarty" <[email protected]>
References: <[email protected]>

Subject: Re: Authentication not working on HTTP-POST using NetworkCredential
Date: Tue, 26 Aug 2003 09:13:29 -0400
Lines: 118
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Message-ID: <[email protected]>
Newsgroups: microsoft.public.dotnet.general
NNTP-Posting-Host: ool-18ba9dd9.dyn.optonline.net 24.186.157.217
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP10.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.general:106081
X-Tomcat-NG: microsoft.public.dotnet.general

Peter -

I want to thank you for that thorough response.

I kind of suspected that was the case. Sometimes you have to read a RFC a
few hundred times to translate from theory to practical use. I had made
mention (to the authors of the server) that no challenge was being issued.
Unfortunately, especially in the industry that I am in, not responding (and
just closing the connection) in the absence of proper credentials is very
common. It prevents an accidental or deliberate probe of a URL from
divulging information that can be used to mount a subsequent attack.

The hack that you included below is similar to the one I did myself. I
merely put it in a method of a utility class rather than one of a derived
class.


- Patrick




| Hi Patrick,
|
| The reason you are not seeing the credentials passed on the
| inital request to the web server is because Microsoft is following
| section 2 of RFC 2617(http://www.faqs.org/rfcs/rfc2617.html)
|
| Here’s the main benefit of using pre-authenticate. Suppose I’m going to
| make 50
| requests to <http://server/path/> and this URL is protected with Basic
| authentication. On the first request, the client gets challenged by the
| server and
| sends back a second request which contains information that the server
| accepts
| (assuming auth succeeds) so it can send back the requested resource.
| With the pre-authenticate property set to true:
| The remaining 49 requests will include the authorization information in
the
| first
| request they send to the server so the server will not challenge the
client
| and
| force it to do another round trip before getting the resource.
| The total number of roundtrips between client and server will be 51.
| With the pre-authenticate property set to false:
| The remaining 49 requests will not include the authorization information
in
| the
| first request and will therefore be challenged by the server on each first
| request
| and will only get the desired resource after sending the authorization
| header in
| the second request.
| The total number of roundtrips between client and server will be 100.
| In other words, pre-authenticate=true is one request shy of taking half
the
| time of
| pre-authenticate=false. Note that pre-authentication only works for Digest
| and
| Basic in v1.0. It can’t work for NTLM because it is connection-based
| however the
| fact that it is connection based means that you’ll only get challenged
once
| per
| connection so it isn’t an issue if you are caching connections. In the
| Whidbey
| release of the .NET Framework we’ll also support pre-authentication for
| Kerberos.
|
| In order to get the inital request to send credentials, you will need to
| use the
| workaround of overriding the GetWebRequest method in the proxy code.
|
| (Hack code obtained from the Internet)
| The PreAuthenticate property on .NET's
| System.Web.Services.Protocols.SoapHttpClientProtocol is supposed to force
| the SOAP
| client proxy to send credentials with the first request, rather than doing
| the
| challenge/response exchange. If you add the following code to your SOAP
| Client
| proxy, you can make PreAuthenticate work (this example is for basic
| authentication):
| protected override System.Net.WebRequest
| GetWebRequest(Uri uri) {
| System.Net.HttpWebRequest request =
| (System.Net.HttpWebRequest)base.GetWebRequest(uri);
| if (this.PreAuthenticate) {
| System.Net.NetworkCredential nc =
| this.Credentials.GetCredential(uri,"Basic");
| if (nc != null) {
| byte[] credBuf =
| new System.Text.UTF8Encoding().
| GetBytes(nc.UserName + ":" + nc.Password);
| request.Headers["Authorization"] =
| "Basic " + Convert.ToBase64String(credBuf);
| }
| }
| return request;
| }
|
| This work around modifies the web service proxy class which is
| automatically generated. This means every time someone updates a "web
| reference" in
| Dev Studio, they would need to reinsert the "hack" code.
|
| Let me know if you have any questions or conerns.
|
| Regards,
| Peter Huang
| Microsoft Online Partner Support
| Get Secure! www.microsoft.com/security
| This posting is provided "as is" with no warranties and confers no rights.