J
Jerry Peters
If you don't know which ones don't lie, what else can you assumemiso said:
safely?
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
safely?
A
Arno
Jerry Peters said:If you don't know which ones don't lie, what else can you assume
safely?
And that is the second problem. But the lying itself is pretty
bad. If a disk claims data has been flushed to surface, but has
not, I would say that this is a pretty severe product defect.
The only known fix is to switch off drive buffer-cache completely,
but that does have negatice performance impact. Nonetheless,
"enterprise" RAID controllers do this and do the access reordering
themselves, supported by a small cache in battery-packed RAM.
Arno
D
DevilsPGD
Arno said:And that is the second problem. But the lying itself is pretty
bad. If a disk claims data has been flushed to surface, but has
not, I would say that this is a pretty severe product defect.
The only known fix is to switch off drive buffer-cache completely,
but that does have negatice performance impact. Nonetheless,
"enterprise" RAID controllers do this and do the access reordering
themselves, supported by a small cache in battery-packed RAM.
And that's the key: If drives were to lie but back it up with
battery-powered cache, all would be well. Unfortunately, they don't.
M
miso
And that is the second problem. But the lying itself is pretty
bad. If a disk claims data has been flushed to surface, but has
not, I would say that this is a pretty severe product defect.
The only known fix is to switch off drive buffer-cache completely,
but that does have negatice performance impact. Nonetheless,
"enterprise" RAID controllers do this and do the access reordering
themselves, supported by a small cache in battery-packed RAM.
Arno
Can you bypass the drive cache on the fly? That is, in the sync command,
make the last step a read of the last byte of data directly from the
drive rather than drive cache.
My opinion here is if this problem was as widespread as you say, errors
relating to this would be showing up all the time. Yet this is the first
time I have heard about the cache not being written.
Now I can believe some drives got out in the wild with poorly written
firmware, but I can't believe every drive on the planet has this problem.
D
DevilsPGD
miso said:Can you bypass the drive cache on the fly? That is, in the sync command,
make the last step a read of the last byte of data directly from the
drive rather than drive cache.
My opinion here is if this problem was as widespread as you say, errors
relating to this would be showing up all the time. Yet this is the first
time I have heard about the cache not being written.
The cache is eventually written, but not if there is a power failure
right between when the drive confirms a write is complete, but before
the write is actually complete.
Even then, if the drive tends to write in the same order the OS did, it
will tend to fail safely anyway most often than not, and will look like
a normal failure, that the file system can recover from as normal.
It's an imperfect system.
J
Jerry Peters
How often do you get power failures with the drive having cached data?miso said:Can you bypass the drive cache on the fly? That is, in the sync command,
make the last step a read of the last byte of data directly from the
drive rather than drive cache.
My opinion here is if this problem was as widespread as you say, errors
relating to this would be showing up all the time. Yet this is the first
time I have heard about the cache not being written.
Now I can believe some drives got out in the wild with poorly written
firmware, but I can't believe every drive on the planet has this problem.
IIRC some of the kernel developers actually *forced* power failures to
test drives, quite a few lied as I remember. Also, just as an example,
my two main machines are essentially immune to this problem, one (this
one) is a netbook and the other is a laptop, both have working
batteries.
A
Arno
DevilsPGD said:
And that's the key: If drives were to lie but back it up with
battery-powered cache, all would be well. Unfortunately, they don't.
Maybe, just maybe we are going to see them doing this with flash.
When power fails, they have a few ms left to preserve buffered data.
Arno
A
Arno
Can you bypass the drive cache on the fly? That is, in the sync command,
make the last step a read of the last byte of data directly from the
drive rather than drive cache.
No, you cannot. You can just tell the drive to switch off
buffering and cache completely.
My opinion here is if this problem was as widespread as you say, errors
relating to this would be showing up all the time. Yet this is the first
time I have heard about the cache not being written.
They do not show up all the time because ordinarily computers
are switched off cleanly which means the drive does not receive
data in the last second or so. But if you unpower HDD immediately
after it has claimed to have flushed its buffer, the problem
crops up reliably.
Now I can believe some drives got out in the wild with poorly written
firmware, but I can't believe every drive on the planet has this problem.
Nobody ever talked about "every drive on the planet" but you.
It is just that unless you know you have to assume the drive does
it wrong.
Arno
A
Arno
Jerry Peters said:How often do you get power failures with the drive having cached data?
IIRC some of the kernel developers actually *forced* power failures to
test drives, quite a few lied as I remember. Also, just as an example,
my two main machines are essentially immune to this problem, one (this
one) is a netbook and the other is a laptop, both have working
batteries.
I believe laptops/netbooks are where the problem originally was
found: Some people (me for example) run these until the battery
is really empty and the power-logic does a hard switch off. If
the drive flushes reliably, a journalling filesystem will prevent
damage reliably for this scenario. If it does not, your filesystem
may get damaged with permanent data loss. The problem is made
worse by drives that lie and reorder commands. Then "commited"
journal writes can be delayed significantly.
Arno
M
miso
Nobody ever talked about "every drive on the planet" but you.
It is just that unless you know you have to assume the drive does
it wrong.
Arno
You said:
The point is precisely that "sync" does not work reliably because
the drives are lying.
This implies all drives. Proper English is:
The point is precisely that "sync" does not work reliably because
some of the drives are lying.
The bitch about usenet is it never dies, so it is hard to change your
statements.
A
Arno
miso said:On 11/29/2013 10:30 PM, Arno wrote:
You said:
The point is precisely that "sync" does not work reliably because
the drives are lying.
This implies all drives. Proper English is:
The point is precisely that "sync" does not work reliably because
some of the drives are lying.
No, the full statement is "... the drives we are talking about here
are lying...". Pretty obvious. Context matters. If you require
people to always state the full context in each step of the
communication, nobody will be willing to talk to you anymore
and rightfully so.
And don't talk to me about proper english.
The bitch about usenet is it never dies, so it is hard to change your
statements.
Indeed.