Hi,

I'm currently working on a project with some specific storage which
internally does a lot of write buffering. The whole thing is working
perfect except of a single problem: when shutting down/suspending/hibernating/
rebooting or whatever the power is cut too fast and the storage
(it is USB bus powered) is not able to complete the cached pending writes
thus we are loosing/corrupting data. Is there any way to delay the
power off. I mean, can I insert an artificial delay after the system
completed the disk I/O (i.e. closed files, flushed registry hives etc)
but before the power is turned off. It is a dirty solution, I know,
and therefore our Linux colleagues had no problem to workaround the
problem this way. Can we do the same on windows? The project is multi-
platform and we need to support at least XP, 2K3 and Vista.

Perhaps some customized shutdown script, a simple kernel mode driver
blocking power IOCTL, anything else. Any ideas are most welcome.

Thank you!

voidcoder wrote:
> Hi,
>
> I'm currently working on a project with some specific storage which
> internally does a lot of write buffering. The whole thing is working
> perfect except of a single problem: when shutting
> down/suspending/hibernating/
> rebooting or whatever the power is cut too fast and the storage
> (it is USB bus powered) is not able to complete the cached pending writes
> thus we are loosing/corrupting data. Is there any way to delay the
> power off. I mean, can I insert an artificial delay after the system
> completed the disk I/O (i.e. closed files, flushed registry hives etc)
> but before the power is turned off. It is a dirty solution, I know,
> and therefore our Linux colleagues had no problem to workaround the
> problem this way. Can we do the same on windows? The project is multi-
> platform and we need to support at least XP, 2K3 and Vista.
>
> Perhaps some customized shutdown script, a simple kernel mode driver
> blocking power IOCTL, anything else. Any ideas are most welcome.
>
> Thank you!
>

There are a couple ways you can address this.

1) Understand how it is supposed to work. For example, with a
USB protocol analyser, see what command Windows is sending
to the device, as part of flushing the device cache, before
shutdown. Then, study the command itself. Is the command synchronous
or asynchronous ? Does the command to flush, require an
acknowledgement that flushing is complete, or instead, is the
status from the device ignored, and assumed to be completed ?
You should understand the mechanics, before using a
sledgehammer to fix it.

2) Based on the results of your studies of (1), then
you can look for a workaround, of which there are a few.
Holding up Windows shutdown might work, if the interval is not
too long - there might be timers used in the shutdown sequence,
that will prevent your method from working if you are looking
for extremely long intervals. (Windows might assume the driver
has stopped responding, and kill it anyway.)

USB device ports have a couple options for internal computer
power. Either +5V or +5VSB can be used. +5VSB sticks around,
until you flip off the switch on the back of the computer.
Older motherboards sometimes had a header and a jumper plug,
to set the voltage source (provided for things like waking a
computer by pressing a key on a USB keyboard). Some recent
motherboards, have +5VSB as the only source. The only limitation
of the +5VSB thing, is if you have so many USB devices connected,
that the power supply doesn't have enough current for them all.
If the USB port was +5VSB powered, then you should have time
to finish flushing.

The next level of fix, would be an act of desperation. It is
possible, if your product provided a 20 pin to 20 pin power
cable extension, for your product to fool around with the
PS_ON# signal going to the power supply. You could artificially
keep the power supply running longer than normal, by altering
the protocol to the power supply. PS_ON# is open collector, and
by having a third party device ground PS_ON# for 30 seconds,
that would keep the power supply running for 30 seconds, giving
time for your device to complete its operation. But if anyone
sees such a solution used, naturally they would scoff at it :-)

Another act of desperation, is a power storage device inside
the USB storage thing. There are things like "supercaps", which
can store charge for a number of seconds. These are not
rechargeable batteries, but are capacitors with fairly large
farad ratings. There are rules to their usage, and I'd have
to go back and look that stuff up - there are probably some
rules about rates of current flow and the like. By storing
enough power, for any possible flushing operation, then it
would not matter how curt the OS was.

If you think about it, it would make sense that the issue had
been considered, when the OS was designed. And that spec wise,
there is a mismatch of expectations. I understand, that one
workaround applied at one time, was for Microsoft to delay
shutdown for a couple seconds, to fix a problem like this.
Whether that is an admission that the problem is an OS one,
or was done to help some hardware implementations that are
less than perfect, is something you could research.

Paul

"voidcoder" <(E-Mail Removed)> wrote in message
news:#(E-Mail Removed)...
> Hi,
>
> I'm currently working on a project with some specific storage which
> internally does a lot of write buffering. The whole thing is working
> perfect except of a single problem: when shutting
down/suspending/hibernating/
> rebooting or whatever the power is cut too fast and the storage
> (it is USB bus powered) is not able to complete the cached pending writes
> thus we are loosing/corrupting data. Is there any way to delay the
> power off. I mean, can I insert an artificial delay after the system
> completed the disk I/O (i.e. closed files, flushed registry hives etc)
> but before the power is turned off. It is a dirty solution, I know,
> and therefore our Linux colleagues had no problem to workaround the
> problem this way. Can we do the same on windows? The project is multi-
> platform and we need to support at least XP, 2K3 and Vista.
>
> Perhaps some customized shutdown script, a simple kernel mode driver
> blocking power IOCTL, anything else. Any ideas are most welcome.
>
> Thank you!
>

RemoveDrive V1.8 - prepares drives for safe removal
Freeware by Uwe Sieber - www.uwe-sieber

Or

Disable write caching for the USB drive.
Computer management - Disk management. In the lower right hand pane, scroll
down to the
flash drive. Right click on the gray box at the left of the drive's entry,
then on properties.
On the Policies tab, select "Optimize for quick removal".

Hi Paul,

thanks for your advices! The problem is definitely hardware
and has to (will) be fixed in hardware by the manufacturer.
Thats for sure. However, at the current stage we just need
to pass a couple of demos and I'd really prefer a simple
fix with delaying the power off. So my question is: how do
I delay the power off in windows? Is there a way to execute
a custom script AFTER the OS completes all the pending disk
I/O. I'm 100% sure there is something like this, e.g. the
way windows installs some updates before shutdown or some
clone/backup disk programs can execute some code to
clone the disk after the OS finished its job but before
the power is removed completely. Any ideas?

Thank you.



Paul wrote:
> voidcoder wrote:
>> Hi,
>>
>> I'm currently working on a project with some specific storage which
>> internally does a lot of write buffering. The whole thing is working
>> perfect except of a single problem: when shutting
>> down/suspending/hibernating/
>> rebooting or whatever the power is cut too fast and the storage
>> (it is USB bus powered) is not able to complete the cached pending writes
>> thus we are loosing/corrupting data. Is there any way to delay the
>> power off. I mean, can I insert an artificial delay after the system
>> completed the disk I/O (i.e. closed files, flushed registry hives etc)
>> but before the power is turned off. It is a dirty solution, I know,
>> and therefore our Linux colleagues had no problem to workaround the
>> problem this way. Can we do the same on windows? The project is multi-
>> platform and we need to support at least XP, 2K3 and Vista.
>>
>> Perhaps some customized shutdown script, a simple kernel mode driver
>> blocking power IOCTL, anything else. Any ideas are most welcome.
>>
>> Thank you!
>>
>
> There are a couple ways you can address this.
>
> 1) Understand how it is supposed to work. For example, with a
> USB protocol analyser, see what command Windows is sending
> to the device, as part of flushing the device cache, before
> shutdown. Then, study the command itself. Is the command synchronous
> or asynchronous ? Does the command to flush, require an
> acknowledgement that flushing is complete, or instead, is the
> status from the device ignored, and assumed to be completed ?
> You should understand the mechanics, before using a
> sledgehammer to fix it.
>
> 2) Based on the results of your studies of (1), then
> you can look for a workaround, of which there are a few.
> Holding up Windows shutdown might work, if the interval is not
> too long - there might be timers used in the shutdown sequence,
> that will prevent your method from working if you are looking
> for extremely long intervals. (Windows might assume the driver
> has stopped responding, and kill it anyway.)
>
> USB device ports have a couple options for internal computer
> power. Either +5V or +5VSB can be used. +5VSB sticks around,
> until you flip off the switch on the back of the computer.
> Older motherboards sometimes had a header and a jumper plug,
> to set the voltage source (provided for things like waking a
> computer by pressing a key on a USB keyboard). Some recent
> motherboards, have +5VSB as the only source. The only limitation
> of the +5VSB thing, is if you have so many USB devices connected,
> that the power supply doesn't have enough current for them all.
> If the USB port was +5VSB powered, then you should have time
> to finish flushing.
>
> The next level of fix, would be an act of desperation. It is
> possible, if your product provided a 20 pin to 20 pin power
> cable extension, for your product to fool around with the
> PS_ON# signal going to the power supply. You could artificially
> keep the power supply running longer than normal, by altering
> the protocol to the power supply. PS_ON# is open collector, and
> by having a third party device ground PS_ON# for 30 seconds,
> that would keep the power supply running for 30 seconds, giving
> time for your device to complete its operation. But if anyone
> sees such a solution used, naturally they would scoff at it :-)
>
> Another act of desperation, is a power storage device inside
> the USB storage thing. There are things like "supercaps", which
> can store charge for a number of seconds. These are not
> rechargeable batteries, but are capacitors with fairly large
> farad ratings. There are rules to their usage, and I'd have
> to go back and look that stuff up - there are probably some
> rules about rates of current flow and the like. By storing
> enough power, for any possible flushing operation, then it
> would not matter how curt the OS was.
>
> If you think about it, it would make sense that the issue had
> been considered, when the OS was designed. And that spec wise,
> there is a mismatch of expectations. I understand, that one
> workaround applied at one time, was for Microsoft to delay
> shutdown for a couple seconds, to fix a problem like this.
> Whether that is an admission that the problem is an OS one,
> or was done to help some hardware implementations that are
> less than perfect, is something you could research.
>
> Paul

Hi,

That was my first idea and unfortunately it failed.
This not an ordinary USB mass storage device, this
is very specific device and it does a lot of write
buffering internally (not artificial buffering in
s/w on OS side).



Jack B. Pollack wrote:
> "voidcoder" <(E-Mail Removed)> wrote in message
> news:#(E-Mail Removed)...
>> Hi,
>>
>> I'm currently working on a project with some specific storage which
>> internally does a lot of write buffering. The whole thing is working
>> perfect except of a single problem: when shutting
> down/suspending/hibernating/
>> rebooting or whatever the power is cut too fast and the storage
>> (it is USB bus powered) is not able to complete the cached pending writes
>> thus we are loosing/corrupting data. Is there any way to delay the
>> power off. I mean, can I insert an artificial delay after the system
>> completed the disk I/O (i.e. closed files, flushed registry hives etc)
>> but before the power is turned off. It is a dirty solution, I know,
>> and therefore our Linux colleagues had no problem to workaround the
>> problem this way. Can we do the same on windows? The project is multi-
>> platform and we need to support at least XP, 2K3 and Vista.
>>
>> Perhaps some customized shutdown script, a simple kernel mode driver
>> blocking power IOCTL, anything else. Any ideas are most welcome.
>>
>> Thank you!
>>
>
> RemoveDrive V1.8 - prepares drives for safe removal
> Freeware by Uwe Sieber - www.uwe-sieber
>
> Or
>
> Disable write caching for the USB drive.
> Computer management - Disk management. In the lower right hand pane, scroll
> down to the
> flash drive. Right click on the gray box at the left of the drive's entry,
> then on properties.
> On the Policies tab, select "Optimize for quick removal".
>
>

voidcoder wrote:
> Hi,
>
> That was my first idea and unfortunately it failed.
> This not an ordinary USB mass storage device, this
> is very specific device and it does a lot of write
> buffering internally (not artificial buffering in
> s/w on OS side).
>
>
>
> Jack B. Pollack wrote:
>> "voidcoder" <(E-Mail Removed)> wrote in message
>> news:#(E-Mail Removed)...
>>> Hi,
>>>
>>> I'm currently working on a project with some specific storage which
>>> internally does a lot of write buffering. The whole thing is working
>>> perfect except of a single problem: when shutting
>> down/suspending/hibernating/
>>> rebooting or whatever the power is cut too fast and the storage
>>> (it is USB bus powered) is not able to complete the cached pending
>>> writes
>>> thus we are loosing/corrupting data. Is there any way to delay the
>>> power off. I mean, can I insert an artificial delay after the system
>>> completed the disk I/O (i.e. closed files, flushed registry hives etc)
>>> but before the power is turned off. It is a dirty solution, I know,
>>> and therefore our Linux colleagues had no problem to workaround the
>>> problem this way. Can we do the same on windows? The project is multi-
>>> platform and we need to support at least XP, 2K3 and Vista.
>>>
>>> Perhaps some customized shutdown script, a simple kernel mode driver
>>> blocking power IOCTL, anything else. Any ideas are most welcome.
>>>
>>> Thank you!
>>>
>>
>> RemoveDrive V1.8 - prepares drives for safe removal
>> Freeware by Uwe Sieber - www.uwe-sieber
>>
>> Or
>>
>> Disable write caching for the USB drive.
>> Computer management - Disk management. In the lower right hand pane,
>> scroll
>> down to the
>> flash drive. Right click on the gray box at the left of the drive's
>> entry,
>> then on properties.
>> On the Policies tab, select "Optimize for quick removal".
>>
>>

Paul wrote:
> voidcoder wrote:
>> Hi,
>>
>> That was my first idea and unfortunately it failed.
>> This not an ordinary USB mass storage device, this
>> is very specific device and it does a lot of write
>> buffering internally (not artificial buffering in
>> s/w on OS side).
>>
>>
>>
>> Jack B. Pollack wrote:
>>> "voidcoder" <(E-Mail Removed)> wrote in message
>>> news:#(E-Mail Removed)...
>>>> Hi,
>>>>
>>>> I'm currently working on a project with some specific storage which
>>>> internally does a lot of write buffering. The whole thing is working
>>>> perfect except of a single problem: when shutting
>>> down/suspending/hibernating/
>>>> rebooting or whatever the power is cut too fast and the storage
>>>> (it is USB bus powered) is not able to complete the cached pending
>>>> writes
>>>> thus we are loosing/corrupting data. Is there any way to delay the
>>>> power off. I mean, can I insert an artificial delay after the system
>>>> completed the disk I/O (i.e. closed files, flushed registry hives etc)
>>>> but before the power is turned off. It is a dirty solution, I know,
>>>> and therefore our Linux colleagues had no problem to workaround the
>>>> problem this way. Can we do the same on windows? The project is multi-
>>>> platform and we need to support at least XP, 2K3 and Vista.
>>>>
>>>> Perhaps some customized shutdown script, a simple kernel mode driver
>>>> blocking power IOCTL, anything else. Any ideas are most welcome.
>>>>
>>>> Thank you!
>>>>
>>>
>>> RemoveDrive V1.8 - prepares drives for safe removal
>>> Freeware by Uwe Sieber - www.uwe-sieber
>>>
>>> Or
>>>
>>> Disable write caching for the USB drive.
>>> Computer management - Disk management. In the lower right hand pane,
>>> scroll
>>> down to the
>>> flash drive. Right click on the gray box at the left of the drive's
>>> entry,
>>> then on properties.
>>> On the Policies tab, select "Optimize for quick removal".
>>>
>>>

I was going to suggest you change the USB port to running from
+5VSB. If you can tell me the computer make and model, or the
motherboard make and model, I may be able to look it up and suggest
a method for doing that.

If a USB mouse remains lit, after the computer goes into standby, then
the port is already running from +5VSB, and so you do have power after
soft shutdown. Then your problem would be elsewhere, such as a failure
protocol-wise, to shut things down properly.

If there is some software stack, other than something like USB Mass
Storage, maybe the necessary flushing mechanism isn't being sent
to the drive ?

Another option, is to connect the USB drive, to a powered USB hub,
where again, the idea would be that power is always present.

SYBA CL-U2HUB-7 USB 2.0 7 Ports Hub - Retail
http://c1.neweggimages.com/NeweggIma...801-021-03.jpg

(Power supply that comes with that USB hub)
http://c1.neweggimages.com/NeweggIma...801-021-05.jpg

Paul

I would try a powered usb hub. By having it powered from same AC as the
storage unit it would stay up after computer power is gone.
I don't know that that would really work but powered hubs are not very
expensive.

Michael


"Paul" <(E-Mail Removed)> wrote in message
news:g6kkqa$3a9$(E-Mail Removed)...
> voidcoder wrote:
>> Hi,
>>
>> That was my first idea and unfortunately it failed.
>> This not an ordinary USB mass storage device, this
>> is very specific device and it does a lot of write
>> buffering internally (not artificial buffering in
>> s/w on OS side).
>>
>>
>>
>> Jack B. Pollack wrote:
>>> "voidcoder" <(E-Mail Removed)> wrote in message
>>> news:#(E-Mail Removed)...
>>>> Hi,
>>>>
>>>> I'm currently working on a project with some specific storage which
>>>> internally does a lot of write buffering. The whole thing is working
>>>> perfect except of a single problem: when shutting
>>> down/suspending/hibernating/
>>>> rebooting or whatever the power is cut too fast and the storage
>>>> (it is USB bus powered) is not able to complete the cached pending
>>>> writes
>>>> thus we are loosing/corrupting data. Is there any way to delay the
>>>> power off. I mean, can I insert an artificial delay after the system
>>>> completed the disk I/O (i.e. closed files, flushed registry hives etc)
>>>> but before the power is turned off. It is a dirty solution, I know,
>>>> and therefore our Linux colleagues had no problem to workaround the
>>>> problem this way. Can we do the same on windows? The project is multi-
>>>> platform and we need to support at least XP, 2K3 and Vista.
>>>>
>>>> Perhaps some customized shutdown script, a simple kernel mode driver
>>>> blocking power IOCTL, anything else. Any ideas are most welcome.
>>>>
>>>> Thank you!
>>>>
>>>
>>> RemoveDrive V1.8 - prepares drives for safe removal
>>> Freeware by Uwe Sieber - www.uwe-sieber
>>>
>>> Or
>>>
>>> Disable write caching for the USB drive.
>>> Computer management - Disk management. In the lower right hand pane,
>>> scroll
>>> down to the
>>> flash drive. Right click on the gray box at the left of the drive's
>>> entry,
>>> then on properties.
>>> On the Policies tab, select "Optimize for quick removal".
>>>
>>>

Hi guys,

thank you, the powered hub thing has been already
tested and works OK (at least we found a couple of
hubs which do the work). It is my fault for not providing
the full details on what we've tried so far, sorry.

For various reasons we would prefer a temporary s/w
solution with a delayed shut down, hence the only
reason for this post. If anyone of you windows gurus
know a way for customizing the power down sequence
to delay the power cut please share your thoughts.

I believe it is pretty simple, as I mentioned before
there is plenty s/w around capable of doing smth.
after the OS completed its job but before the power
off (e.g. acronis clone image and so on). I just need
to add a simple app/script to block the power off for
n seconds. Nothing fancy or complex. The only question
is how do I insert this extra step into the power down
sequence?


Thanks.


Michael Walraven wrote:
> I would try a powered usb hub. By having it powered from same AC as the
> storage unit it would stay up after computer power is gone.
> I don't know that that would really work but powered hubs are not very
> expensive.
>
> Michael
>
>
> "Paul" <(E-Mail Removed)> wrote in message
> news:g6kkqa$3a9$(E-Mail Removed)...
>> voidcoder wrote:
>>> Hi,
>>>
>>> That was my first idea and unfortunately it failed.
>>> This not an ordinary USB mass storage device, this
>>> is very specific device and it does a lot of write
>>> buffering internally (not artificial buffering in
>>> s/w on OS side).
>>>
>>>
>>>
>>> Jack B. Pollack wrote:
>>>> "voidcoder" <(E-Mail Removed)> wrote in message
>>>> news:#(E-Mail Removed)...
>>>>> Hi,
>>>>>
>>>>> I'm currently working on a project with some specific storage which
>>>>> internally does a lot of write buffering. The whole thing is working
>>>>> perfect except of a single problem: when shutting
>>>> down/suspending/hibernating/
>>>>> rebooting or whatever the power is cut too fast and the storage
>>>>> (it is USB bus powered) is not able to complete the cached pending
>>>>> writes
>>>>> thus we are loosing/corrupting data. Is there any way to delay the
>>>>> power off. I mean, can I insert an artificial delay after the system
>>>>> completed the disk I/O (i.e. closed files, flushed registry hives etc)
>>>>> but before the power is turned off. It is a dirty solution, I know,
>>>>> and therefore our Linux colleagues had no problem to workaround the
>>>>> problem this way. Can we do the same on windows? The project is multi-
>>>>> platform and we need to support at least XP, 2K3 and Vista.
>>>>>
>>>>> Perhaps some customized shutdown script, a simple kernel mode driver
>>>>> blocking power IOCTL, anything else. Any ideas are most welcome.
>>>>>
>>>>> Thank you!
>>>>>
>>>>
>>>> RemoveDrive V1.8 - prepares drives for safe removal
>>>> Freeware by Uwe Sieber - www.uwe-sieber
>>>>
>>>> Or
>>>>
>>>> Disable write caching for the USB drive.
>>>> Computer management - Disk management. In the lower right hand
>>>> pane, scroll
>>>> down to the
>>>> flash drive. Right click on the gray box at the left of the drive's
>>>> entry,
>>>> then on properties.
>>>> On the Policies tab, select "Optimize for quick removal".
>>>>
>>>>

voidcoder wrote:
>
> Hi guys,
>
> thank you, the powered hub thing has been already
> tested and works OK (at least we found a couple of
> hubs which do the work). It is my fault for not providing
> the full details on what we've tried so far, sorry.
>
> For various reasons we would prefer a temporary s/w
> solution with a delayed shut down, hence the only
> reason for this post. If anyone of you windows gurus
> know a way for customizing the power down sequence
> to delay the power cut please share your thoughts.
>
> I believe it is pretty simple, as I mentioned before
> there is plenty s/w around capable of doing smth.
> after the OS completed its job but before the power
> off (e.g. acronis clone image and so on). I just need
> to add a simple app/script to block the power off for
> n seconds. Nothing fancy or complex. The only question
> is how do I insert this extra step into the power down
> sequence?
>
>
> Thanks.
>

Have you considered posting the question in a Windows
driver group ? Or a group involving PC storage ?
Programmers may not hang out in the groups this
is posted in.

microsoft.public.development.device.drivers
comp.sys.ibm.pc.hardware.storage

There is an example of a protocol issue here.

http://support.microsoft.com/kb/286733

If the device works while you use a powered hub,
that should be sufficient for a demonstration of
the technology. If you're honest, you can always
explain you are working on the cache flushing protocol,
and ensuring that Windows knows the device "supports
caching" and thus needs some kind of command sent
to it to flush caches.

I used to do demos, like the kind you're trying to
do, and if you use two guys, a slick guy to talk
fast, with a second rodent-like individual who runs
around and fiddles the hardware (that is me), you
can cover up just about any sin the hardware may have :-)
Been there, done that.

Paul