Bug In NTFS Date/Time in Windows XP SP3 Fully Patched

[Stewart Berman]

I am in the Eastern time zone (New York).

I use the following:
XCOPY "M:\My TiVo Recordings\Part1\*.*" "P:\My TiVo Recordings\Part1\" /D /E /V /C /I /Y >>
C:\Build\Backup\MediaBackup1.log
to backup video files from an internal SATA drive to an external USB drive. Both drives use NTFS.

Normally the above would only copy one or two files at the most.

When this kicked off this morning at 5:00 AM this morning (3/8/2009) XCOPY decided that all of the
files on the SATA drive were an hour newer than the files on the USB drive. For example a file on
the SATA drive that showed a modified date/time stamp of 7/18/2007 1:51 AM on Saturday showed a
modified date/time stamp of 7/18/2007 2:51 AM on Sunday. However the same file on the USB drive
still showed a modified data/time stamp of 7/18/2007 1:51 AM. XCOPY therefore decided that the
version on the SATA was newer than the version on the USB drive and copied it.

The differences in the date/time stamped also showed up in Explorer -- until the files on the USB
drive were overwritten. So this is not an XCOPY bug but a bug in the NTFS file system daylight
savings time interface for SATA drives.

The bug is on the SATA side. Date/Time stamps should not change when daylight savings time occurs.
If a file was last modified on 7/18/2007 at 1:51 AM that is what should be visible in Explorer
whether daylight savings time is in effect or not.

 

[John Wunderlich]

I am in the Eastern time zone (New York).

I use the following:
XCOPY "M:\My TiVo Recordings\Part1\*.*" "P:\My TiVo
Recordings\Part1\" /D /E /V /C /I /Y >>
C:\Build\Backup\MediaBackup1.log to backup video files from an
internal SATA drive to an external USB drive. Both drives use
NTFS.

Normally the above would only copy one or two files at the most.

When this kicked off this morning at 5:00 AM this morning
(3/8/2009) XCOPY decided that all of the files on the SATA drive
were an hour newer than the files on the USB drive. For example
a file on the SATA drive that showed a modified date/time stamp of
7/18/2007 1:51 AM on Saturday showed a modified date/time stamp of
7/18/2007 2:51 AM on Sunday. However the same file on the USB
drive still showed a modified data/time stamp of 7/18/2007 1:51
AM. XCOPY therefore decided that the version on the SATA was
newer than the version on the USB drive and copied it.

The differences in the date/time stamped also showed up in
Explorer -- until the files on the USB drive were overwritten. So
this is not an XCOPY bug but a bug in the NTFS file system
daylight savings time interface for SATA drives.

The bug is on the SATA side. Date/Time stamps should not change
when daylight savings time occurs. If a file was last modified on
7/18/2007 at 1:51 AM that is what should be visible in Explorer
whether daylight savings time is in effect or not.

No bug. System operating normally.
FAT file systems store file date/time as a local date/time.
NTFS file systems store date/time in UTC format.
This can cause a problem when the computer automatically adjusts for
Daylight Savings time (restarting your computer should adjust for
this difference)

For more info, see:

"File Times"
<http://msdn.microsoft.com/en-us/library/ms724290(VS.85).aspx>
Skip down halfway to the "File Times and Daylight Saving Time"
paragraph.

Personally, I recommend that you reformat/convert your USB drive to
NTFS format, which should permanently eliminate this problem.

HTH,
John

 

[Stewart Berman]

No bug. System operating normally.
FAT file systems store file date/time as a local date/time.
NTFS file systems store date/time in UTC format.
This can cause a problem when the computer automatically adjusts for
Daylight Savings time (restarting your computer should adjust for
this difference)

For more info, see:

"File Times"
<http://msdn.microsoft.com/en-us/library/ms724290(VS.85).aspx>
Skip down halfway to the "File Times and Daylight Saving Time"
paragraph.

Personally, I recommend that you reformat/convert your USB drive to
NTFS format, which should permanently eliminate this problem.

HTH,
John

You are correct -- the USB drive does have a FAT32 file system not NTFS. However, it is still a
bug. If I modified a file at 1:15 AM on the 23rd of February it should show that time even if it is
the 8th of March. Otherwise the audit trail is not reliable.

In addition. Microsoft could easy fix the time stamp breaks between NTFS and FAT32 by flushing the
FAT32 cache when the daylight savings time switch is done instead of requiring the user to reboot at
3:00 AM. At least the file time stamps would be consistent even if they are wrong.

 

[John Wunderlich]

You are correct -- the USB drive does have a FAT32 file system not
NTFS. However, it is still a bug. If I modified a file at 1:15
AM on the 23rd of February it should show that time even if it is
the 8th of March. Otherwise the audit trail is not reliable.

On a NTFS-formatted drive, the file date is physically stored in UTC
time. There is no daylight savings time here. It is strictly GMT. The
file date/time is fixed. What does change is how time is _displayed_.
If you tell Windows you want the time displayed as Pacific Daylight
Time, then the system will subtract 7 hours from the actual recorded
time and show you that. It only _appears_ to shift by one hour because
you implicitly tell Windows that you want it displayed as Daylight time
instead of Standard time. The file date/time itself is recorded on
disk in GMT and does not change, thus 'audit trail' is physically
unbroken.

The FAT method of storing Local Time is much more problematic. If you
store a file on a laptop while on the East coast, then fly to the west
coast (or step across the Georgia-Alabama border), how does Windows
know which files on the disk should be time-shifted? Should it do a
complete disk scan?

It is not a 'bug' because Microsoft has designed, built, and documented
it that way and isn't likely to fix it. (actually, they probably have
fixed it by creating NTFS). I would catagorize it as undesirable
behavior, though.

In addition. Microsoft could easy fix the time stamp breaks
between NTFS and FAT32 by flushing the FAT32 cache when the
daylight savings time switch is done instead of requiring the user
to reboot at 3:00 AM. At least the file time stamps would be
consistent even if they are wrong.

There's no second-guessing Microsoft.
Complain all you want -- the bottom line is still the same: If you're
at all dependent on file date/time, you're _much_ better off switching
all of your drives to NTFS.

Cheers,
John

 

[odp]

On a NTFS-formatted drive, the file date is physically stored in UTC
time. There is no daylight savings time here. It is strictly GMT. The
file date/time is fixed. What does change is how time is _displayed_.
If you tell Windows you want the time displayed as Pacific Daylight
Time, then the system will subtract 7 hours from the actual recorded
time and show you that. It only _appears_ to shift by one hour because
you implicitly tell Windows that you want it displayed as Daylight time
instead of Standard time. The file date/time itself is recorded on
disk in GMT and does not change, thus 'audit trail' is physically
unbroken.

I have had the same problem, between my NTFS internal hard disk, and my
FAT32 esternal hard disk. You answer was very explanatory. You say that

On a NTFS-formatted drive, the file date is physically stored in UTC
time. There is no daylight savings time here. It is strictly GMT. The
file date/time is fixed. What does change is how time is _displayed_.

What has confused me is that when time switched from ST to DST, why were the
timestamps on all of my internal hard disk files changed at all? All that
needed to change was the system clock time, so that when "new" files are
created or old files "modified" they will have the correct "current"
timestamp. None of the pre-existing files were modified - so why were their
timestamps altered? Files need to "display" the true create/modify
timestamp, and not display create/modify time adjusted for current time. If
I go and talk with someone about changing a file, I need to remember if it
was changed during ST or DST, so I can adjust in my mind, what hour I should
ask about? Even if both the FAT32 and NTFS file systems changed the files
the same - to my lights it would still not be accurate. How can we keep
accurate date and timestamps, if ms just comes in and willy nilly changes
them?

deb

 

[Bill Blanton]

I have had the same problem, between my NTFS internal hard disk, and my
FAT32 esternal hard disk. You answer was very explanatory. You say that


What has confused me is that when time switched from ST to DST, why were the
timestamps on all of my internal hard disk files changed at all? All that
needed to change was the system clock time, so that when "new" files are
created or old files "modified" they will have the correct "current"
timestamp. None of the pre-existing files were modified - so why were their
timestamps altered? Files need to "display" the true create/modify
timestamp, and not display create/modify time adjusted for current time. If
I go and talk with someone about changing a file, I need to remember if it
was changed during ST or DST, so I can adjust in my mind, what hour I should
ask about? Even if both the FAT32 and NTFS file systems changed the files
the same - to my lights it would still not be accurate. How can we keep
accurate date and timestamps, if ms just comes in and willy nilly changes
them?

The point is that the timestamp did not change. Just the "displayed time".

 

[odp]

The point is that the timestamp did not change. Just the "displayed time".

I understand the point. I may not have written it clearly, when I wrote
'changed' what I meant to write, was "displayed'. Even if the FAT32 and
NTFS file systems both displayed in sync using DST, why would we want files
created on 1/1/2006 at noon to display 1/1/2006 01:00 pm? One pm is not the
'true' time it was created.

Fiels created prior to the DST change should not display differently after
the switch. The new DST system time can be used to reflect timestamps for
new files and files moidfied after the switch. Am I the only one who thinks
this is a screwy way to treat files?

 

[John Wunderlich]

I understand the point. I may not have written it clearly, when I
wrote 'changed' what I meant to write, was "displayed'. Even if
the FAT32 and NTFS file systems both displayed in sync using DST,
why would we want files created on 1/1/2006 at noon to display
1/1/2006 01:00 pm? One pm is not the 'true' time it was created.

Files created prior to the DST change should not display
differently after the switch. The new DST system time can be used
to reflect timestamps for new files and files moidfied after the
switch. Am I the only one who thinks this is a screwy way to
treat files?

Yes, your way is screwy.

What if it were Fall and Daylight Savings time was just about to end.
You create a file at 1:30 AM then at 2:00 AM the time shifts back one
hour to 1:00 AM at which time you see a mistake in your file and make a
quick correction. Now, which file is the newer file? The one with the
1:05 AM time displayed or the one with the 1:30 AM time displayed?

As it is currently, and should be, after the time rollback, the 1:30
file would then display a 12:30AM time and your 1:05AM file would be
the newest.

-- John

 

[Jim]

Unix does it right: the time is always GMT on the disc, and is converted
for you to
eyeballt when you want to eyeball.

That is the way XP works as well.
Jim