If a drive has reallocated sectors, did it lose data?

[Guest]

I recently checked the SMART data for a 1-year old hard drive, and it showed 15 reallocated sectors. If a drive has reallocated sectors, does that mean it definitely lost data, or is it possible to reallocate sectors without losing any data?

I wonder if any of the other SMART attributes might indicate if data was lost or not. There is another one called "Uncorrectable Sector Count" and that has a value of zero - does that mean no data has been lost on the drive?

Here's a screenshot of 4 different SMART programs showing the SMART data for my drive:
http://img11.imageshack.us/img11/9241/mbpsmart.png

 

[Arno]

I recently checked the SMART data for a 1-year old hard drive, and it
showed 15 reallocated sectors. If a drive has reallocated sectors, does
that mean it definitely lost data, or is it possible to reallocate sectors
without losing any data?

Reallocated sectors are not data loss, but successfully recovered
and moved. Whether the recovery was due to a write or a successful
read after trying hard is not recorded.

Data-loss is listed as "pending" sectors. Once they get written
the drive carefully evaluates them and either reallocates or if
the sector is fine (and the problem was, e.g., vibration during
the original unreadable write) write the to the original sector.

I wonder if any of the other SMART attributes might indicate if data was
lost or not. There is another one called "Uncorrectable Sector Count" and
that has a value of zero - does that mean no data has been lost on the
drive?

Yes. Not every drive has that AFAIK though. And see above.

Here's a screenshot of 4 different SMART programs showing the
SMART data for my drive:
http://img11.imageshack.us/img11/9241/mbpsmart.png

A small number (and 15 is small) of reallocated sectors does
not necessarily indicate a problem on modern drives.

The disk looks fine, possibly except for the hardware ECC
recovered. I think this is not one of the hard to intepret
values (but somebody here will doubtlessly correct me if it
is), and the cooked value of "1" actually indicates a problem,
namely that the drive has trouble reading. That could also
be the reason for the reallocated sectors.

According to Google, this attribute was only ever discussed
cursorily in this group. Acronis says "If no other (critical) parameters
report a problem, hardware replacement is recommended on mission critical
systems only." on http://kb.acronis.com/content/9131

For home users I would interpret that as "if backup exists
or the data is of low value, do not replace".

Arno

 

[Rod Speed]

I recently checked the SMART data for a 1-year old hard
drive, and it showed 15 reallocated sectors. If a drive has
reallocated sectors, does that mean it definitely lost data,

No, sectors will only be reallocated on a write.

or is it possible to reallocate sectors without losing any data?

Yes, when the write fails, the sector is reallocated, and then the write
succeeds.

I wonder if any of the other SMART attributes
might indicate if data was lost or not.

That one does.

There is another one called "Uncorrectable Sector Count"

That's sectors that can not be reallocated.

and that has a value of zero - does that
mean no data has been lost on the drive?
Yes.

Here's a screenshot of 4 different SMART
programs showing the SMART data for my drive:
http://img11.imageshack.us/img11/9241/mbpsmart.png

You haven't lost any data.

15 reallocated sectors is getting up a bit, so its
important that you backup the drive right now.

 

[Guest]

No, sectors will only be reallocated on a write.


Yes, when the write fails, the sector is reallocated, and then the write
succeeds.

What if I have a spreadsheet that is sitting on some bad sectors, and
I open the spreadsheet and the drive can't read from those bad sectors
- does the drive just return bad data? Then if I save the
spreadsheet, the writes to the bad sectors fail and the sectors are
reallocated, but then I did lose data because it wasn't originally
read correctly.

That's sectors that can not be reallocated.

If sectors are only reallocated on writes, then are you saying that
the "Uncorrectable Sector Count" can only be greater than zero if the
drive has run out of spare sectors?

 

[mike]

What if I have a spreadsheet that is sitting on some bad sectors, and
I open the spreadsheet and the drive can't read from those bad sectors
- does the drive just return bad data? Then if I save the
spreadsheet, the writes to the bad sectors fail and the sectors are
reallocated, but then I did lose data because it wasn't originally
read correctly.

I'd also like to hear the details on this.
You're mixing your concepts.
As I understand it, a spreadsheet program should never read the hard drive.
The program asks the OS for data. The OS figgers out where it is and
presents it to the program. The program should get GOOD data or
a message that the data cannot be recovered. It's up to the error
correcting/handling/reporting
capabilities of the disk the OS and the program to decide what
to present to the user.

I don't recall ever seeing WRONG data. But I've seen plenty of "data
not found", "read error", CRC error, etc.

 

[Noob]

What if I have a spreadsheet that is sitting on some bad sectors, and
I open the spreadsheet and the drive can't read from those bad sectors
- does the drive just return bad data?

A fraction of the area on HDDs is dedicated to ECC
(50-bytes per 512-byte sector on legacy drives, circa 100-bytes
per 4K sector on AF drives, according to Wikipedia)
https://en.wikipedia.org/wiki/Advanced_Format#Advanced_Format_overview

Therefore, it is VERY unlikely (yet not impossible) for an error
to go undetected at the time of read.

 

[Rod Speed]

What if I have a spreadsheet that is sitting on some bad sectors,
and I open the spreadsheet and the drive can't read from those
bad sectors - does the drive just return bad data?

Nope, the drive returns an error saying that the sector is unreadable.

Then if I save the spreadsheet,

You wouldn't do that because you got an error message
when you tried to read it, there is nothing to save.

the writes to the bad sectors fail and the sectors are reallocated,
but then I did lose data because it wasn't originally read correctly.

Nope, the original file is still there and it may be possible to
read it later if the bad sectors are marginally readable or if
the failure to read them was because the drive had go stinking
hot and you allowed the drive to cool down by turning it off etc.

If sectors are only reallocated on writes, then are you saying that
the "Uncorrectable Sector Count" can only be greater than zero
if the drive has run out of spare sectors?

No. They are uncorrectable if they cannot be read. That's an entirely
separate matter to whether the drive has run out of spare sectors.
It only considers how many spare sectors there are when it trys to
reallocate a sector on a write.

 

[Arno]

What if I have a spreadsheet that is sitting on some bad sectors, and
I open the spreadsheet and the drive can't read from those bad sectors
- does the drive just return bad data?

No. Drives have checksums that are _very_ good as recognizing errors.
I have had bad data returned on an SSD and a flash drive, but that
is likely firmware bugs tue to immaturity. It is not something
you need to expect with a HDD, at least not due to disk surface
problems. You can get bad data due to broken mainboards, bad RAM,
overclocked CPUs and bad PUS though.

Then if I save the
spreadsheet, the writes to the bad sectors fail and the sectors are
reallocated, but then I did lose data because it wasn't originally
read correctly.

If sectors are only reallocated on writes, then are you saying that
the "Uncorrectable Sector Count" can only be greater than zero if the
drive has run out of spare sectors?

Sectors can be reallocated on reading. As usual, Rod does not
know what he is talking about. If on a read, the drive has
to try multiple times, or the ECC is needed to a certain
degree, the drive will test the sector and if found bad will
reallocated. Now if the drive fails to read a secor, then
you have an "uncorrectable sector", obviously.

Arno

 

[Arno]

I'd also like to hear the details on this.
You're mixing your concepts.
As I understand it, a spreadsheet program should never read the hard drive.
The program asks the OS for data. The OS figgers out where it is and
presents it to the program. The program should get GOOD data or
a message that the data cannot be recovered. It's up to the error
correcting/handling/reporting
capabilities of the disk the OS and the program to decide what
to present to the user.

I don't recall ever seeing WRONG data. But I've seen plenty of "data
not found", "read error", CRC error, etc.

I have had wrong data without error message on one early OCZ SSD
and two flash sticks, one by Kingston and one by PQI. I guess the
SSD is a firmware bug and the Flash drives are bad engineering
decisions.

And you can of course get data corruption at other points than
the drive.

Arno

 

[Rod Speed]

No. Drives have checksums that are _very_ good as recognizing errors.
I have had bad data returned on an SSD and a flash drive, but that
is likely firmware bugs tue to immaturity. It is not something
you need to expect with a HDD, at least not due to disk surface
problems. You can get bad data due to broken mainboards, bad RAM,
overclocked CPUs and bad PUS though.

Sectors can be reallocated on reading.

Wrong, as always.

As usual, Rod does not know what he is talking about.

We'll see...

If on a read, the drive has to try multiple times,
or the ECC is needed to a certain degree, the drive
will test the sector and if found bad will reallocated.

Nope, because it hasn't got the data from that sector.

Now if the drive fails to read a secor, then
you have an "uncorrectable sector", obviously.

Utterly mangled all over again.

 

[mike]

How does the drive know that the write failed?
Waiting a whole revolution to read/verify each sector seems like
an unacceptable performance penalty.

 

[mike]

Same way it knows that the read failed.

You don't have to verify it like that before doing anything else, you
can just do that later when the platter had rotated under the heads to
there again.

Like when moving a 20GB video file?
Are you saying that disk writes are always verified by reading/comparing?
automagically?
By the drive?
Without cutting the throughput by at least half...maybe a zillion
if seeks are involved?

 

[Rod Speed]

Like when moving a 20GB video file?

That can be done, but isnt usually.

Are you saying that disk writes are always verified by reading/comparing?
automagically?
By the drive?

No, that that can be done by the drive if you want it to do that.

The Maxtors used to have that automatically enabled when
they were new and that automatically got disabled after
enough power on hours, not very many, tens of hours.

It was done like that so that the drive would auto
reallocate the worst sectors when it was first being used.

Without cutting the throughput by at least half...

The effect wasn't very noticeable at all with those drives.

maybe a zillion if seeks are involved?

Nothing like as much as that with modern fast seeking drives.

 

[Noob]

Drives have checksums that are _very_ good as recognizing errors.
I have had bad data returned on an SSD and a flash drive, but that
is likely firmware bugs tue to immaturity. It is not something
you need to expect with a HDD, at least not due to disk surface
problems. You can get bad data due to broken main boards, bad RAM,
overclocked CPUs and bad PUS though.

My dying IDE controller returned whatever was requested, minus a
5-byte offset (so instead of getting bytes 0-511 of the sector,
I'd get bytes 5-511 + random garbage).

Needless to say, the boot sector looked "funny" to the BIOS,
and the PC refused to boot!

 

[Guest]

That can be done, but isnt usually.






No, that that can be done by the drive if you want it to do that.

I was looking through this thread again, and would like a clarification on one thing. You said the drive will reallocate a sector when a write fails.But how exactly does it know the write fails if you say it usually doesn't read the data to verify it was written correctly?

 

[Arno]

I was looking through this thread again, and would like a clarification on
one thing. You said the drive will reallocate a sector when a write
fails. But how exactly does it know the write fails if you say it usually
doesn't read the data to verify it was written correctly?

A drive checks a sector when a read is very difficult. If that
check fails, it reallocates. If the check succeeds, it just
re-writes the sector. The latter happens for example when there
was vibration during the original write that resulted in a
marginal write quality.

If a read fails, the drive marks the sector as "pending" and
reports a read error to the sytem. On subsequent reads to it,
it will try again very hard to read, with either success (then
see above) or failure, the sector stays "pending" and a
read error is reported to the system.

Wne a "pending" sector is written to, the drive checks the
target sector.If the check succeeds, it just overwrites it.
If the check fails, it re-allocates. In both cases, the
"pending" mark on the sector is deleted.

Arno

 

[Rod Speed]

I was looking through this thread again, and would like a clarification on
one thing.
You said the drive will reallocate a sector when a write fails. But how
exactly does it know
the write fails if you say it usually doesn't read the data to verify it
was written correctly?

The sector will normally be flagged as pending because it has been hard
to read, so when you write to one of those sectors, and the drive does
verify that it was written to successfully, because it was a pending sector,
it will know that it need to reallocate it when the write has been seen to
fail.

 

[Guest]

The sector will normally be flagged as pending because it has been hard
to read, so when you write to one of those sectors, and the drive does
verify that it was written to successfully,

Does it read the sector to verify it was written successfully?

because it was a pending sector,
it will know that it need to reallocate it when the write has been seen to
fail.

What about a sector that hasn't been marked pending - the drive tries
to write to it and it fails. Does the drive read the sector
immediately after the write to find out that it failed (the data
doesn't match)?

 

[Guest]

A drive checks a sector when a read is very difficult. If that
check fails, it reallocates.

If it fails, how can it reallocate without the data?

If the check succeeds, it just
re-writes the sector. The latter happens for example when there
was vibration during the original write that resulted in a
marginal write quality.

If a read fails, the drive marks the sector as "pending" and
reports a read error to the sytem. On subsequent reads to it,
it will try again very hard to read, with either success (then
see above) or failure, the sector stays "pending" and a
read error is reported to the system.

Wne a "pending" sector is written to, the drive checks the
target sector.If the check succeeds, it just overwrites it.
If the check fails, it re-allocates. In both cases, the
"pending" mark on the sector is deleted.

When you say the drive "checks" the sector, how exactly does that
work?

 

[Arno]

On Wednesday, December 19, 2012 10:48:35 PM UTC-5, Rod Speed wrote: [...]
No, that that can be done by the drive if you want it to do that.
I was looking through this thread again, and would like a clarification on
one thing.  You said the drive will reallocate a sector when a write
fails.  But how exactly does it know the write fails if you say it usually
doesn't read the data to verify it was written correctly?

A drive checks a sector when a read is very difficult. If that
check fails, it reallocates.

If it fails, how can it reallocate without the data?

When a read is very difficult but succeeds eventually.
Then it checks the sector whether it can be cleanly rewrittwen.
If the read fails despite hogher effort, the sector
is set as pending.

When you say the drive "checks" the sector, how exactly does that
work?

Write a number of patterns, see whether they can be read cleanly.
How exactly this is done is vendor dependent and they are
not giving out descriptions.

Arno