Hi Slobodan,
I did all previous tests with NTFS with compression.
Now I prepared a CF disk with FAT and tested again
(The test is: recursively copy a directory tree
from network drive to disk, then (optionally) do
FlushFileBuffers, then cut power off, reboot, compare
files on disk to files on network. Autochk is enabled.)
With FAT things are even worse:
Without FlushFileBuffers, chkdsk complains about lost
clusters and creates some *.chk files, but the original
files are completely missing.
With FlushFileBuffers, everything is OK.
With NTFS, and without FlushFileBuffers, the situation
is as follows:
chkdsk detects no errors, all files look OK at first glance,
(file length, modification time etc. are OK),
but usually one file (not always the same!) consists of
binary zeros from some offset on. That means that the
NTFS "metadata" is OK, but the user data (file contents)
is corrupted...
So it seems that our only rescue is FlushFileBuffers!
The problem is not with our own application which we
can modify easily, it's with 3rd party SW (e.g. FTP
server, ZIP file extractor, ...) where it would be best
if flushing occurs at least if a file is closed.
Regards,
Bernhard
Slobodan Brcin (eMVP) wrote:
Hi Bernhard,
First do you use NTFS or FAT?
I think that NTFS was not created to work on removable disks so it
might not flush data on its own. FAT actually flushed data on
removable disks.
Now when you mention this, I remembered that there was a discussion
on this subject although I can't find the right thread :-(
Anyhow you can't do anything to disk internal caching trough MS
registry switches, but rather trough sending proprietary SCSI codes
to disk device to disable caching or to alter internal caching
behavior.
If you get corruption when you press reset button then this is
windows software issue.
If you get corruption when you cut power then this might be internal
disk caching problem.
Optimize for quick removal will try to pass all write request to disk
controller as they come, with minimum or no delay. Also it
should have different behavior with regards on dirty bit.
http://groups-beta.google.com/group..._frm/thread/0a00a67f882086c7#4ee475b3e0c63a48
http://groups-beta.google.com/group..._frm/thread/a692df61fb93fb8d#4a434eecc6c89102
Regards,
Slobodan
Hi Slobodan,
the registry value UserWriteCacheSetting=0x0 is automatically
created when I un-check the "Enable write caching on the disk"
check-box in the properties GUI.
That was of course what I tried first, but it had no effect on
the corrupted file problems after power down.
I don't know much about caching issues, but I conclude from the
explanation for the "Optimize for quick removal" switch in the GUI
("This setting disables write caching on the disk and in Windows,
...")
that there are two levels of write caching: hardware level and
Windows
level. The UserWriteCacheSetting value seems to have effect only on
hardware level caching. (But this is actually only speculation...)
At the moment I think I have to fall back on a tool that calls
FlushFileBuffers every 5 seconds or so - not very satisfactory,
but I hope this will reduce the errors considerably.
Regards,
Bernhard
Slobodan Brcin (eMVP) wrote:
Try:
UserWriteCacheSetting:
http://groups-beta.google.com/groups?q=UserWriteCacheSetting&qt_s=Search+Groups
I have no idea where you dig out "SpecialFlags" key but look at
the
following that only mention value that you use, this is
hardcoded hack for specific disk models.
Following call set entries that you mention:
ClassScanForSpecial(fdoExtension, DiskBadControllers,
DiskSetSpecialHacks);
And as you can see DiskBadControllers is hardcoded.
CLASSPNP_SCAN_FOR_SPECIAL_INFO DiskBadControllers[] = {
{ "COMPAQ" , "PD-1" , NULL, 0x02
},
{ "CONNER" , "CP3500" , NULL, 0x02
},
{ "FUJITSU" , "M2652S-512" , NULL, 0x01
},
{ "HP ", "C1113F " , NULL, 0x20
},
// iomegas require START_UNIT commands so be sure to match
all of
them.
{ "iomega" , "jaz" , NULL, 0x30
},
{ "iomega" , NULL , NULL, 0x20
},
{ "IOMEGA" , "ZIP" , NULL, 0x27
},
{ "IOMEGA" , NULL , NULL, 0x20
},
{ "MAXTOR" , "MXT-540SL" , "I1.2", 0x01
},
{ "MICROP" , "1936-21MW1002002" , NULL, 0x03
},
{ "OLIVETTI", "CP3500" , NULL, 0x02
},
{ "SEAGATE" , "ST41601N" , "0102", 0x02
},
{ "SEAGATE" , "ST3655N" , NULL, 0x08
},
{ "SEAGATE" , "ST3390N" , NULL, 0x08
},
{ "SEAGATE" , "ST12550N" , NULL, 0x08
},
{ "SEAGATE" , "ST32430N" , NULL, 0x08
},
{ "SEAGATE" , "ST31230N" , NULL, 0x08
},
{ "SEAGATE" , "ST15230N" , NULL, 0x08
},
{ "SyQuest" , "SQ5110" , "CHC", 0x03
},
{ "TOSHIBA" , "MK538FB" , "60", 0x01
},
{ NULL , NULL , NULL, 0x0
}
};
Regards,
Slobodan
// HackDisableWriteCache
// CLASS_SPECIAL_DISABLE_WRITE_CACHE
Hi Slobodan,
- Setting HackMask=1 (in "...\Device Parameters\Classpnp" key)
did not help.
- Setting SpecialFlags=8 (in "...\Device Parameters\disk" key)
had the effect that I could not enable write caching in the
properties dialog any more (which would be OK), but did not
help either.
(This idea stems from the following DDK source line
#define HackDisableWriteCache (0x08)
- I then played around with IOCTL_STORAGE_SET_HOTPLUG_INFO:
I was able to persistently change the DeviceHotplug value to 1
for the CF device, but this did not help, too.
The MediaRemovable value is still 0 (it is 1 for the USB
stick), and I it seems I just can't let Windows treat the CF
card like a removable hotplug device :-(
Thanks for your help anyway
Bernhard
Slobodan Brcin (eMVP) wrote:
Hi Bernhard,
Few facts from DDK:
1. RemovalPolicyExpectSurpriseRemoval = 3
2.
#define CLASSP_REG_HACK_VALUE_NAME (L"HackMask")
#define CLASSP_REG_WRITE_CACHE_VALUE_NAME
(L"WriteCacheEnableOverride")
#define CLASSP_REG_REMOVAL_POLICY_VALUE_NAME
(L"UserRemovalPolicy")
3. There is WriteCacheEnableOverride but this is override
that is
used to enable and not to disable caching in filesystems.
4. UserRemovalPolicy of 3 with override and set flag
DeviceHotplug to
true.
5. This is from DDK classpnp implementation. Comment in code
speak
for itself:
//
// this refers to devices which, for reasons not yet
understood,
// do not fail PREVENT_MEDIA_REMOVAL requests even though
they
// have no way to lock the media into the drive. this
allows
// the filesystems to turn off delayed-write caching for
these
// devices as well.
//
if (TEST_FLAG(FdoExtension->PrivateFdoData->HackFlags,
FDO_HACK_CANNOT_LOCK_MEDIA)) {
fdoData->HotplugInfo.MediaHotplug = TRUE;
} else {
fdoData->HotplugInfo.MediaHotplug = FALSE;
}
6. #define FDO_HACK_CANNOT_LOCK_MEDIA
(0x00000001)
To conclude this try playing with registry value "HackMask"
bit
0.
Let us know if this helped.
Regards,
Slobodan
Hi Slobodan,
yes, I used regedit, restarted the computer, and the value
remained
unchanged.
Here is how the regedit export looks like:
