get file details for HTTP location

[baker_tony]

Hi, is there any way of getting details (such as last modified/created
date/time) of a file located on the web?

E.g I'd like to know when the file "http://www.myWebSite.com/
update.txt" was last created/modified.

Cheers,

Tony.

 

[Vadym Stetsiak]

Hello, (e-mail address removed)!

With general HTTP access there is no way to get these values.
If that file will be located on FTP server then some of the mentioned
properties will be available.

If only HTTP access is available, you can add special page on the server
that will return update.txt upon request and specify necessary attributes in
the HTTP headers.

HTH
--
With best regards, Vadym Stetsiak.
Blog: http://vadmyst.blogspot.com

You wrote on Tue, 18 Sep 2007 10:49:47 -0000:

bt> Hi, is there any way of getting details (such as last
bt> modified/created date/time) of a file located on the web?

bt> E.g I'd like to know when the file "http://www.myWebSite.com/
bt> update.txt" was last created/modified.

bt> Cheers,

bt> Tony.

 

[Chris Shepherd]

Hello, (e-mail address removed)!

With general HTTP access there is no way to get these values.

This is incorrect. Normal webservers will return an HTTP header
Last-Modified that contains this information.

To the OP: See "14.29 Last-Modified" in
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html for information
on what to expect in response.

Some webservers may not respond properly with Last-Modified, so it's
best to ensure your target uses proper headers before relying on it (and
even then, it's subject to change).

Chris.

 

[gwell]

This is incorrect. Normal webservers will return anHTTPheader
Last-Modified that contains this information.

To the OP: See "14.29 Last-Modified" inhttp://www.w3.org/Protocols/rfc2616/rfc2616-sec14.htmlfor information
on what to expect in response.

Some webservers may not respond properly with Last-Modified, so it's
best to ensure your target uses proper headers before relying on it (and
even then, it's subject to change).

Chris.

Hi,

I am tryng to write a windows service that will retrieve csv files and
other files over HTTP. I only want to get those files that have been
updated recently. Considering the discussion above do you think this
is an unreliable way to go? Maybe I should try look at ftp?

I cant seem to get the last-modified property off the header... here
is some sample code I am testing:

Any help would be greatly appreciated... thanks!

*************************************************************************************************************************************************************

System.Net.WebRequest req = System.Net.HttpWebRequest.Create("http://
uri/TestHTML.htm");
string last = req.Headers["If-Modified-Since"];
IEnumerator cacheEnum = req.Headers.GetEnumerator();
while (cacheEnum.MoveNext())
{
string strKey = cacheEnum.Current.ToString();
}

 

[Jon Skeet [C# MVP]]

I cant seem to get the last-modified property off the header... here
is some sample code I am testing:

Any help would be greatly appreciated... thanks! *************************************************************************************************************************************************************

System.Net.WebRequest req = System.Net.HttpWebRequest.Create("http://
uri/TestHTML.htm");
string last = req.Headers["If-Modified-Since"];
IEnumerator cacheEnum = req.Headers.GetEnumerator();
while (cacheEnum.MoveNext())
{
string strKey = cacheEnum.Current.ToString();

}

You need to *add* the If-Modified-Since header yourself. The idea is
that you tell the server when you last fetched the data, and it can
either give you back the data or tell you that it hasn't changed.

Jon

 

[Vadym Stetsiak]

Hello, gwell!

Ftp can be a better option here.
Another way, if you have access to the remote HTTP server (where CSVs are
hosted) and that server has, say, ASP.NET. Then you can code ASP.NET
web page that will return requested file along with necessary headers.

--
With best regards, Vadym Stetsiak.
Blog: http://vadmyst.blogspot.com

You wrote on Wed, 19 Sep 2007 01:16:47 -0700:


g> Hi,

g> I am tryng to write a windows service that will retrieve csv files
g> and other files over HTTP. I only want to get those files that have
g> been updated recently. Considering the discussion above do you think
g> this is an unreliable way to go? Maybe I should try look at ftp?

g> I cant seem to get the last-modified property off the header... here
g> is some sample code I am testing:

g> Any help would be greatly appreciated... thanks!

g> **********************************************************************
g> **********************************************************************
g> *****************

g> System.Net.WebRequest req = System.Net.HttpWebRequest.Create("http://
g> uri/TestHTML.htm");
g> string last = req.Headers["If-Modified-Since"];
g> IEnumerator cacheEnum = req.Headers.GetEnumerator();
g> while (cacheEnum.MoveNext())
g> {
g> string strKey = cacheEnum.Current.ToString();
g> }

 

[Chris Shepherd]

Hello, gwell!

Ftp can be a better option here.
Another way, if you have access to the remote HTTP server (where CSVs are
hosted) and that server has, say, ASP.NET. Then you can code ASP.NET
web page that will return requested file along with necessary headers.

FTP *can* be a better option, but it's a lot to implement when the
webserver's already there and it already pumps out the information
requested.

I also wouldn't use a webpage for this alternate solution, go with an
XML document.

Chris.

 

[Chris Shepherd]

I am tryng to write a windows service that will retrieve csv files and
other files over HTTP. I only want to get those files that have been
updated recently. Considering the discussion above do you think this
is an unreliable way to go? Maybe I should try look at ftp?

Here's the thing. I think FTP could be a good way to go if you had to do
this in multiple locations, where reliability may be in question. The
problem is most hosts nowadays do properly return the Last-Modified
header -- at the very least, all installs of Apache and IIS CAN return
the Last-Modified header, and do by default.

The major con with using an FTP connection is that you're doing quite a
bit more work to do what you could do with a simple HTTP request. The
overhead might be minimal, but in code, tracking FTP sites, logins,
etc., that's going to be far and away more work than simply tracking the
URL and checking its Last-Modified header.

Now, given your goal here of grabbing remote files via a service, I
think you should definitely stick to HTTP until such time as it becomes
an issue for you. Last-Modified is part of the RFC, so RFC-friendly
webservers will understand what you're looking for (which, just with the
two mentioned above accounts for something like 96%+ of all webservers
in use publicly on the Internet), and it's fairly trivial to implement.

I cant seem to get the last-modified property off the header... here
is some sample code I am testing:

Any help would be greatly appreciated... thanks!

[...]

Jon's response already covered the code stuff.

Chris.

 

[gwell]

I am tryng to write a windows service that will retrieve csv files and
other files overHTTP. I only want to get those files that have been
updated recently. Considering the discussion above do you think this
is an unreliable way to go? Maybe I should try look at ftp?

Here's the thing. I think FTP could be a good way to go if you had to do
this in multiple locations, where reliability may be in question. The
problem is most hosts nowadays do properly return the Last-Modified
header -- at the very least, all installs of Apache and IIS CAN return
the Last-Modified header, and do by default.

The major con with using an FTP connection is that you're doing quite a
bit more work to do what you could do with a simpleHTTPrequest. The
overhead might be minimal, but in code, tracking FTP sites, logins,
etc., that's going to be far and away more work than simply tracking the
URL and checking its Last-Modified header.

Now, given your goal here of grabbing remote files via a service, I
think you should definitely stick toHTTPuntil such time as it becomes
an issue for you. Last-Modified is part of the RFC, so RFC-friendly
webservers will understand what you're looking for (which, just with the
two mentioned above accounts for something like 96%+ of all webservers
in use publicly on the Internet), and it's fairly trivial to implement.

I cant seem to get the last-modified property off the header... here
is some sample code I am testing:

Any help would be greatly appreciated... thanks!

[...]

Jon's response already covered the code stuff.

Chris.

Hi,

Thanks for all the feedback. I have just got a proof of concept
working using HTTP and I'm going to start building the windows
service. I found the following article useful:

http://www.clariusconsulting.net/blogs/kzu/archive/2007/07/30/29105.aspx

Here is my updated code that I will work with -

Thanks for all your comments... much appreciated!

*********************************************************************************************************************************************************

// Create a request for the URL.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://uri/
TestHTML.htm");

// If required by the server, set the credentials.
request.Credentials = CredentialCache.DefaultCredentials;
request.IfModifiedSince = DateTime.Parse("23-07-2007");

try
{
using (WebResponse response = request.GetResponse())
{
// Do something if the resource has changed.
}
}
catch (WebException wex)
{
HttpWebResponse httpResponse = wex.Response as HttpWebResponse;
if (httpResponse.StatusCode == HttpStatusCode.NotModified)
{
// resource was not modified.
}
// Something else happened. Rethrow or log.
throw;
}

 

[baker_tony]

Fantastic stuff! I managed to construct the following code which gives
me the last modified date time:

HttpWebRequest request =
(HttpWebRequest)WebRequest.Create("http://www.teslaeurope.com/
default.htm");

// If required by the server, set the credentials.
//request.Credentials =
CredentialCache.DefaultCredentials;
//request.IfModifiedSince = DateTime.Parse("01-01-1990");

try
{
using (WebResponse response = request.GetResponse())
{
string lastModified = response.Headers["Last-
Modified"].ToString();
}
}
catch (WebException wex)
{
....