write cache (win/Storage)

[Ingo]

According to technet the option to "enable write cache" in the policies
window (w2k3) always has an impact on the setting of the underlying
storage, no matter if it's a single dumb sata disk or a high-end
storage-subsystem with integrated RAID-controller.
I do understand that this setting always implies windows OS caching.
Or is it a driver's setting?
All tests I did confirmed that behaviour regarding IBM DS4700 storage.
When you tick the option the IBM storage manager shows an altered
setting as well. So obviously SCSI commands trigger the setting on the DS.
If hard disk caching is disabled, performance drops to 15% (SATA, duh).
Now I want to DISABLE windows OS caching completely, but leave caching
on for the underlying storage.
Would that sound reasonable (and possible)?
Thanks.

 

[Arno Wagner]

According to technet the option to "enable write cache" in the policies
window (w2k3) always has an impact on the setting of the underlying
storage, no matter if it's a single dumb sata disk or a high-end
storage-subsystem with integrated RAID-controller.
I do understand that this setting always implies windows OS caching.
Or is it a driver's setting?
All tests I did confirmed that behaviour regarding IBM DS4700 storage.
When you tick the option the IBM storage manager shows an altered
setting as well. So obviously SCSI commands trigger the setting on the DS.
If hard disk caching is disabled, performance drops to 15% (SATA, duh).
Now I want to DISABLE windows OS caching completely, but leave caching
on for the underlying storage.
Would that sound reasonable (and possible)?

Not really. UNless you do not trust your OS. For Windows, this
may still be a reasonable idea, as the technology is not up
to the standards of other OSes.

As to whether it is possible, you can only disable OS buffering.
I thinkt his wirks by marking the device as "removable".
You will still get application buffering. What you do not want to
disable is caching. Caching only influences read access, by
delivering already read stuff from memory (MS here again
confuses the terminologuy: There is no write caching. It is
either read caching or write buffering).

Arno

 

[Ingo]

Not really. UNless you do not trust your OS. For Windows, this
may still be a reasonable idea, as the technology is not up
to the standards of other OSes.

As to whether it is possible, you can only disable OS buffering.
I thinkt his wirks by marking the device as "removable".
You will still get application buffering. What you do not want to
disable is caching. Caching only influences read access, by
delivering already read stuff from memory (MS here again
confuses the terminologuy: There is no write caching. It is
either read caching or write buffering).

Arno

Thanks for the answer (and the side blow to Win, of course The reason
I want to do this is having two Win that share one LUN in a SAN. One
machine dies, the other seamlessly takes over (write sessions, it's a
backup app). The app requires having the mentioned Win setting turned
off (or using your term write buffering). Alas, as described above, that
also disables caching on the storage controller for that specific LUN.
I'm not sure every storage vendor does this. I think HDS Thunder leaves
its caching setting on.
I want to be sure that write operations (acknowledged "back" from OS to
the app) have found their way to the storage (leaving out windows
buffering) in case of desaster. I cannot set "remov. device", it's
greyed out. Oh yeah, there's a app driver (similar to Win cluster
service for quorum) that disallows all settings for the LUN on the
passive node. Maybe that's what "trusting your OS" is all about and the
reason the backup sw vendor also requires that setting to be off.
Am I having it right? Do I lose data when my active machine dies?
Thanks.

 

[Arno Wagner]

Thanks for the answer (and the side blow to Win, of course The reason
I want to do this is having two Win that share one LUN in a SAN. One
machine dies, the other seamlessly takes over (write sessions, it's a
backup app). The app requires having the mentioned Win setting turned
off (or using your term write buffering). Alas, as described above, that
also disables caching on the storage controller for that specific LUN.
I'm not sure every storage vendor does this. I think HDS Thunder leaves
its caching setting on.
I want to be sure that write operations (acknowledged "back" from OS to
the app) have found their way to the storage (leaving out windows
buffering) in case of desaster. I cannot set "remov. device", it's
greyed out. Oh yeah, there's a app driver (similar to Win cluster
service for quorum) that disallows all settings for the LUN on the
passive node. Maybe that's what "trusting your OS" is all about and the
reason the backup sw vendor also requires that setting to be off.
Am I having it right? Do I lose data when my active machine dies?
Thanks.

Ok, so you wan a device accessed directly from multiple installations
of Windows. Sorry, but for that you need an OS that can actually
handle this. Windows cannot. Linux, incidentially, also cannot
really, as far as I know. This suggests that your approach is wrong.

What could be possible under Linux is to multi-host the SCSI and
to only load the SCSI drive on one machine. When that one fails,
you can switch it off (using a remote power swicth) and then load
the SCSI drive on the second box, reset the bus and maybe access the
disk. This strikes me as overly complex though.

Thisrd approach: Use Linux and mirror the drive remotely to the
second machine. This means you need two instances of your storage.
The mirroring schould be possible wth the righ filesystem, maybe GFS.

This needs more research, but having one storage device under
the control of two OSes is not normally possible and done.

Arno

 

[Arno Wagner]

Ok, so you wan a device accessed directly from multiple installations
of Windows. Sorry, but for that you need an OS that can actually
handle this. Windows cannot. Linux, incidentially, also cannot
really, as far as I know. This suggests that your approach is wrong.

What could be possible under Linux is to multi-host the SCSI and
to only load the SCSI drive on one machine. When that one fails,
you can switch it off (using a remote power swicth) and then load
the SCSI drive on the second box, reset the bus and maybe access the
disk. This strikes me as overly complex though.

Thisrd approach: Use Linux and mirror the drive remotely to the
second machine. This means you need two instances of your storage.
The mirroring schould be possible wth the righ filesystem, maybe GFS.

This needs more research, but having one storage device under
the control of two OSes is not normally possible and done.

P.S.: Here is the main problem: Both OSes need a constistent
picture of the device at any time. Since one does not know
when the other wtote somewhere, all on disk data and metadata
would have to be read at each time it was used, no caching at all.
And the device would need to be locked against accesses by the
other one at times. While theoretically possible, you would get likely
<1% of the performance you get with ordinary accessess. Therefore
you do this either with one machine controlling the storage and
use remote access, or you replicate not only the machine, but
also the storage and mirror the data.

Arno

 

[Ingo]

Ok, so you wan a device accessed directly from multiple installations
of Windows. Sorry, but for that you need an OS that can actually
handle this. Windows cannot. Linux, incidentially, also cannot
really, as far as I know. This suggests that your approach is wrong.

What could be possible under Linux is to multi-host the SCSI and
to only load the SCSI drive on one machine. When that one fails,
you can switch it off (using a remote power swicth) and then load
the SCSI drive on the second box, reset the bus and maybe access the
disk. This strikes me as overly complex though.

Thisrd approach: Use Linux and mirror the drive remotely to the
second machine. This means you need two instances of your storage.
The mirroring schould be possible wth the righ filesystem, maybe GFS.

This needs more research, but having one storage device under
the control of two OSes is not normally possible and done.

Arno

Hello Arno,
you got me wrong Of course the storage is only configured in a SAN
to be accessible by both OSs (once again see MSCS quorum disk, Shared
Storage). Going down to Miliseconds you are right. Of course it's EITHER
one OS (machine) OR the other accessing the LUN. You're apt to lose data
when both do it REALLY SIMULTANEOUSLY (signatures, inodes etc.) As long
as both systems are alive the Failover is handled fine. There are
cluster-aware Apps (one of which is my Backup app). So now I've come
back to my OP again. What happens when the active OS dies?
Obviously I haven't made myself clear here.
Have a nice weekend

 

[Ingo]

Hello Arno,
you got me wrong Of course the storage is only configured in a SAN
to be accessible by both OSs (once again see MSCS quorum disk, Shared
Storage). Going down to Miliseconds you are right. Of course it's EITHER
one OS (machine) OR the other accessing the LUN. You're apt to lose data
when both do it REALLY SIMULTANEOUSLY (signatures, inodes etc.) As long
as both systems are alive the Failover is handled fine. There are
cluster-aware Apps (one of which is my Backup app). So now I've come
back to my OP again. What happens when the active OS dies?
Obviously I haven't made myself clear here.
Have a nice weekend

p.s. I think I know the misunderstanding now... the application is not
constantly "failing over" between the two OSs!!! Of course that would be
performance=0. No no, one machine writes for weeks, then dies. That's
the situation.

 

[Arno Wagner]

Hello Arno,
you got me wrong Of course the storage is only configured in a SAN
to be accessible by both OSs (once again see MSCS quorum disk, Shared
Storage). Going down to Miliseconds you are right. Of course it's EITHER
one OS (machine) OR the other accessing the LUN. You're apt to lose data
when both do it REALLY SIMULTANEOUSLY (signatures, inodes etc.) As long
as both systems are alive the Failover is handled fine. There are
cluster-aware Apps (one of which is my Backup app). So now I've come
back to my OP again. What happens when the active OS dies?
Obviously I haven't made myself clear here.
Have a nice weekend

So your question is what happens to the dat on disk when the OS dies
while there still is information in the buffer not yet flushed to
disk? Well, that depends on the filesystem and the application.
Conceptually this is not really different from the machine dying while
that application is in the process of writing and there is no
write-buffer. Turning off the buffer does not really help that
much. It just reduces the risk a bit and reduces the amount of data
''in flight'' (i.e. where the app thinks it has written, but it is not
on disk). This only matters if there is outside communication that
then could be told the wrong thing. Unless your application does
single sector writes with disc-buffer flushes after them, you
allways need to be prepared for aborted writes, combined
with a system crash.

Arno

 

[Squeeze]

Arno Wagner wrote in news:[email protected]

Not really. UNless you do not trust your OS. For Windows, this
may still be a reasonable idea, as the technology is not up
to the standards of other OSes.

As to whether it is possible, you can only disable OS buffering.
I thinkt his wirks by marking the device as "removable".
You will still get application buffering. What you do not want to
disable is caching. Caching only influences read access, by
delivering already read stuff from memory (MS here again
confuses the terminologuy: There is no write caching.

It is either read caching or write buffering).

Utter nonsense, as always from the babblebot

 

[Ingo]

Arno Wagner wrote in news:[email protected]


Utter nonsense, as always from the babblebot

well Squeeze... since I'm really stuck here, you might want to share
what you think?!
Problem in a nutshell again:
- cluster aware app requires "enable write caching on disk" to be
disabled in Win 2k3 OS policy pane of disk mgmt (my guess is that is
because power failure of SERVER)
- doing that disables caching in storage subsystem as well
(these being SATA disks, performance drops)
- i don't want that
- so question: is there a way to disable OS caching (server failure no
problem) without disabling underlying storage caching mechanism? If not,
wtf. Option "optimze for quick removal" is of course not available
Thanks
Ingo

 

[Arno Wagner]

well Squeeze... since I'm really stuck here, you might want to share
what you think?!
Problem in a nutshell again:
- cluster aware app requires "enable write caching on disk" to be
disabled in Win 2k3 OS policy pane of disk mgmt (my guess is that is
because power failure of SERVER)
- doing that disables caching in storage subsystem as well
(these being SATA disks, performance drops)
- i don't want that
- so question: is there a way to disable OS caching (server failure no
problem) without disabling underlying storage caching mechanism? If not,
wtf. Option "optimze for quick removal" is of course not available
Thanks
Ingo

Don't expect anything from that person.

Arno

 

[Squeeze]

Ingo wrote in news:[email protected]

well Squeeze... since I'm really stuck here, you might want to share
what you think?!

What makes you think that if I knew a solution I wouldn't share it with
you first time but I would second time?

I think that you should go to the storage newsgroup, comp.arch.storage .

Problem in a nutshell again:
- cluster aware app requires "enable write caching on disk" to be
disabled in Win 2k3 OS policy pane of disk mgmt (my guess is that is
because power failure of SERVER)
- doing that disables caching in storage subsystem as well
(these being SATA disks, performance drops)

- i don't want that

Tough luck. That's what it takes to make sure that data read from the drive
is actually on the platters and not in some cache or buffer, drive or otherwise.