Formatting a hard disk and handling of suspicious bad sectors

[Yousuf Khan]

When I right-clicked a hard disk in Window$ and hit Format, was it the
hard disk controller that took over to format a particular sector on the
disk? Or was it the OS?

It's an OS-level operation. It doesn't extend out to beyond the
partition borders, and partitions are organizational constructs
maintained by the OS'es. If there is a separate partition maintained by
a different OS, it won't be touched.

Today, a format is simply rewriting the file allocation and directory
information on the partition. A long format might involve a cursory
read-test of the surface of the disk before finalizing.

I wanted to reduce the fault tolerance level of the formatting process
such that it would mark a sector as bad when there was one single
read/write failure.

Right now, the Format process would retry again and again for long time
when a bad sector was hit. I don't want the process to retry, and just
mark it as bad.

As others have mentioned, just run a disk wipe utility on the partition.
Disk Redactor is a free-ware freespace wipe utility that runs under
Windows. It even has a feature to do just a write test on the disk, if
you don't want to actually wipe the freespace.


Yousuf Khan

 

[Yousuf Khan]

Yes, but why can't customers do a low-level format again AFTER YEARS of
use? Why should customers rely on SMART?

SMART is actually quite useful these days, but it requires that you get
the modern disk monitoring software that can interpret it for you. A
good one is Hard Disk Sentinel, but its most advanced SMART disk testing
features require a registration fee. But its overall disk health report
is completely free, and that's all you really need.

Other good ones seem to be HDScan (completely free), and Seatools (free
from Seagate, but requires a Seagate or Maxtor drive to be installed on
your system, but it'll work with all other makes of drives, as long as
you have at least one Seagate in there).

SMART does its own internal disk testing that is quite thorough.

Yousuf Khan

 

[Arno]

Yes, but why can't customers do a low-level format again AFTER YEARS of
use? Why should customers rely on SMART?

There really is no need for that feature anymore. Reading the
complete DISK (e.g. bu long SMART selftest) and then doing
one complete overwrite serves the same purpose. One problem
is that a real LL format requires the disk to expose its
internal structure and encoding of secotrs, IDs, etc. over
the interface. While in the old days this format was standardized
and the connection between disk and controller actually had
analog signal lines, today disks are far more complicated
internally with ZBR, advances error correciotn, no-ID sectors,
etc.. This means writing working formatting software would
be a nightmare. Not even the disk itself carries that around.
In addition, you would also need to write the servo information,
and AFIAK most (all?) modern disks cannot write that at all
because their heads are not suitable for it.

On the other hand, what purpose did LL format serve? Not rally
any, besides adjusting the disk to the controller, e.g. by
selecting MFM or RLL. Defect mapping was not done in LL format,
but the verify read afterwards. You can still do that. But
today disks will do the defect management themselves, because
it did cause numerous problems, e.g. that OSes were unable to
dynamically handle defects.

Incidentially, with advanced magnetic coatings, you do not
need to format the disk during its lifetime, it will not lose
data due to weakened magnetization, except in areas with
surface defects. And these need to be mapped out permanently,
another reason to let the disk do it.

So, you do not need to rely on SMART. But you do need to rely
on the disks own defective secor management. And one way to
do a complete check is a long SMART selftest and then a
complete overwrite. However you get the same effect by doing
a complete read and then a complete overwrite. I have done this.

Arno

 

[Yousuf Khan]

How could I tell the utility not to retry a bad sector to save time? I
want it to mark a sector as bad when it fails to read it on the FIRST
TIME (aka, NO MERCY)!

There's always little errors happening with modern drives, e.g. line
errors. There could be 100's of these per minute. If it weren't for the
fact that the drive's own electronics understood this situation, under
your system we'd be marking perfectly good sectors bad for no reason.

It sounds like you're frustrated with one particular drive that may be
giving you these errors. I'm sure if you analysed it with SMART tools,
you'll find that this drive may actually be failing and it's slow
because it's run out of spare sectors to replace. The best way to find
out is to post the SMART report right here on the newsgroup and let the
experts here take a look at it. A utility called Everest will save a
file with this info in it, which you can then copy'n'paste here.

Yousuf Khan

 

[Man-wai Chang to The Door (24000bps)]

There's always little errors happening with modern drives, e.g. line

errors. There could be 100's of these per minute. If it weren't for the
fact that the drive's own electronics understood this situation, under
your system we'd be marking perfectly good sectors bad for no reason.

Like I said: No mercy! The data is more valuable than a tiny tiny spot
on a disk platter.

It sounds like you're frustrated with one particular drive that may be
giving you these errors. I'm sure if you analysed it with SMART tools,
you'll find that this drive may actually be failing and it's slow
because it's run out of spare sectors to replace. The best way to find
out is to post the SMART report right here on the newsgroup and let the
experts here take a look at it. A utility called Everest will save a
file with this info in it, which you can then copy'n'paste here.

I understood. Same "no mercy" policy when it comes to the reliability of
the whole disk drive!

--
@~@ Might, Courage, Vision, SINCERITY.
/ v \ Simplicity is Beauty! May the Force and Farce be with you!
/( _ )\ (x86_64 Ubuntu 9.10) Linux 2.6.32.8
^ ^ 17:52:01 up 1 day 2:42 0 users load average: 1.19 1.12 1.06
ä¸å€Ÿè²¸! ä¸è©é¨™! ä¸æ´äº¤! ä¸æ‰“交! ä¸æ‰“劫! ä¸è‡ªæ®º! è«‹è€ƒæ…®ç¶œæ´ (CSSA):
http://www.swd.gov.hk/tc/index/site_pubsvc/page_socsecu/sub_addressesa

 

[Jonathan de Boyne Pollard]

... as long as one is prepared to use partition types that most
versions of MS/PC/DR-DOS won't be able to cope with. This, of
course, was M. Bryce's point. And even then that is presuming that
one's disc is below the 2TiB limit, beyond which one has to do things
like switch from the MBR partitioning scheme to the EFI partitioning
scheme, which no version of MS/PC/DR-DOS at all can cope with.

Modern versions of Win handle [partition types that most versions of
MS/PC/DR-DOS won't be able to cope with] fine.

"Modern versions of Win" are not DOS, of course.

Not it was not [M. Bryce's point].

I'm confident that if asked xe would state xyrself that it was. What xe
wrote wasn't particularly unclear.

Only fools run dinosaurs like that.

Whether it's foolish to run MS/PC/DR-DOS is besides the point. The
premise is that one is, and the consequence of the premise is that it's
difficult to handle today's disc sizes with MS/PC/DR-DOS, given their
comparatively small partition size limits and the 2TiB limit of the old
MBR partitioning scheme (the only partitioning scheme that they
understand) itself.

 

[Jonathan de Boyne Pollard]

Yes, but why can't customers do a low-level format again AFTER YEARS
of use?

Were you to put the phrase "low-level format" into your favourite WWW
search engine, you would find lots of information on that subject, too.

 

[Arno]

There's always little errors happening with modern drives, e.g. line
errors. There could be 100's of these per minute. If it weren't for the
fact that the drive's own electronics understood this situation, under
your system we'd be marking perfectly good sectors bad for no reason.

It sounds like you're frustrated with one particular drive that may be
giving you these errors. I'm sure if you analysed it with SMART tools,
you'll find that this drive may actually be failing and it's slow
because it's run out of spare sectors to replace. The best way to find
out is to post the SMART report right here on the newsgroup and let the
experts here take a look at it. A utility called Everest will save a
file with this info in it, which you can then copy'n'paste here.

I second that. In fact I used to do timed surface runs to find
drives with problems in a cluster environment before moving
to smart. This was pretty reliable. Defective drives take
significantly longer to read data than others (If they do not
fail at it directly).

Arno

 

[Yousuf Khan]

Like I said: No mercy! The data is more valuable than a tiny tiny spot
on a disk platter.

But that's just it, it's not errors on the recording medium, it could be
errors of timing and other things. The error only occurs because
something is too busy or some cluster on the disk didn't rotate around
fast enough for the head to read it the first time.

I understood. Same "no mercy" policy when it comes to the reliability of
the whole disk drive!

I'd say get the report of that drive posted here, and we can tell you
what's going on with it.

Yousuf Khan

 

[Arno]

But that's just it, it's not errors on the recording medium, it could be
errors of timing and other things. The error only occurs because
something is too busy or some cluster on the disk didn't rotate around
fast enough for the head to read it the first time.

Also on reading modern disks start before the headas have completely
settled. That frequnetly makes ECC use necessary after a seek and
occasionaly a second read. Errors from this do not indicate any
problem with the disk, this is just a performance optimization.

I'd say get the report of that drive posted here, and we can tell you
what's going on with it.

Indeed.

Arno

 

[Rod Speed]

Oh yes ... I remember DOS ... and floppy disks too!
Most of today's disks are way too big to do a DOS format without first setting up lots of logical partitions.
Wrong. You can format the entire drive with one partition if you want.
... as long as one is prepared to use partition types that most
versions of MS/PC/DR-DOS won't be able to cope with. This, of
course, was M. Bryce's point. And even then that is presuming that
one's disc is below the 2TiB limit, beyond which one has to do
things like switch from the MBR partitioning scheme to the EFI
partitioning scheme, which no version of MS/PC/DR-DOS at all can
cope with.

Modern versions of Win handle [partition types that most versions of MS/PC/DR-DOS won't be able to cope with] fine.

"Modern versions of Win" are not DOS, of course.

What was being disucssed was using DOS just for the format of the drive.

No it was not [M. Bryce's point].

I'm confident that if asked xe would state xyrself that it was.

Your confidence is completely irrelevant.

What xe wrote wasn't particularly unclear.

It wasnt unclear that what was being discussed was JUST using DOS for the format of the drive.

Whether it's foolish to run MS/PC/DR-DOS is besides the point.

No it is not.

The premise is that one is, and the consequence of the premise is that it's difficult to handle today's disc sizes
with MS/PC/DR-DOS, given their comparatively small partition size limits

No such limit exists with JUST using DOS to format the drive.

and the 2TiB limit of the old MBR partitioning scheme (the only partitioning scheme that they understand) itself.

That isnt relevant to the drive he wants to format.

 

[Rod Speed]

Were you to put the phrase "low-level format" into your favourite WWW
search engine, you would find lots of information on that subject, too.

But with an awful lot of shit to wade thru. Lot simpler to ask here.

 

[Robin Bignall]

SMART is actually quite useful these days, but it requires that you get
the modern disk monitoring software that can interpret it for you. A
good one is Hard Disk Sentinel, but its most advanced SMART disk testing
features require a registration fee. But its overall disk health report
is completely free, and that's all you really need.

Other good ones seem to be HDScan (completely free), and Seatools (free
from Seagate, but requires a Seagate or Maxtor drive to be installed on
your system, but it'll work with all other makes of drives, as long as
you have at least one Seagate in there).

SMART does its own internal disk testing that is quite thorough.

Another piece of software that will give you a SMART readout is
Speedfan, even if you use it occasionally only for that purpose, not
to control/monitor fans. It is free.

 

[Ed Light]

Whether it's foolish to run MS/PC/DR-DOS is besides the point. The

It looks like you may be answering Rod Speed. Most of us have him
filtered out, and it's quite clear sailing!
--
Ed Light

Better World News TV Channel:
http://realnews.com

Iraq Veterans Against the War and Related:
http://ivaw.org
http://couragetoresist.org
http://antiwar.com

Send spam to the FTC at
(e-mail address removed)
Thanks, robots.

 

[Yousuf Khan]

Another piece of software that will give you a SMART readout is
Speedfan, even if you use it occasionally only for that purpose, not
to control/monitor fans. It is free.

Yeah, it does display SMART values, but does not make any sort of
analysis of that data.

Yousuf Khan

 

[Sjouke Burry]

Yeah, it does display SMART values, but does not make any sort of
analysis of that data.

Yousuf Khan

Everest Home edition does.
Extract from report:

[ HDS722580VLAT20 (VNR21EC2T11N3M) ]

01 Raw Read Error Rate 60 100 100 0 OK: Value is normal
02 Throughput Performance 50 100 100 0 OK: Value is normal
03 Spin Up Time 24 113 113 11600047 OK: Value is normal
04 Start/Stop Count 0 91 91 37210 OK: Always passing
05 Reallocated Sector Count 5 100 100 0 OK: Value is normal
07 Seek Error Rate 67 100 100 0 OK: Value is normal
08 Seek Time Performance 20 100 100 0 OK: Value is normal
09 Power-On Time Count 0 98 98 18847 OK: Always passing
0A Spin Retry Count 60 100 100 0 OK: Value is normal
0C Power Cycle Count 0 100 100 1853 OK: Always passing
C0 Power-Off Retract Count 50 69 69 37833 OK: Value is normal
C1 Load/Unload Cycle Count 50 69 69 37833 OK: Value is normal
C2 Temperature 0 148 148 17, 37 OK: Always passing
C4 Reallocation Event Count 0 100 100 0 OK: Always passing
C5 Current Pending Sector Count 0 100 100 0 OK: Always passing
C6 Off-Line Uncorrectable Sector Count 0 100 100 0 OK: Always passing
C7 Ultra ATA CRC Error Rate 0 200 200 0 OK: Always passing

[ HDS722516VLAT80 (VNR43EC4GMYM6M) ]

01 Raw Read Error Rate 60 100 100 0 OK: Value is normal
02 Throughput Performance 50 100 100 0 OK: Value is normal
03 Spin Up Time 24 99 99 20185385 OK: Value is normal
04 Start/Stop Count 0 95 95 23580 OK: Always passing
05 Reallocated Sector Count 5 100 100 35 OK: Value is normal
07 Seek Error Rate 67 100 100 0 OK: Value is normal
08 Seek Time Performance 20 100 100 0 OK: Value is normal
09 Power-On Time Count 0 98 98 18811 OK: Always passing
0A Spin Retry Count 60 100 100 0 OK: Value is normal
0C Power Cycle Count 0 100 100 1838 OK: Always passing
C0 Power-Off Retract Count 50 81 81 23735 OK: Value is normal
C1 Load/Unload Cycle Count 50 81 81 23735 OK: Value is normal
C2 Temperature 0 177 177 18, 31 OK: Always passing
C4 Reallocation Event Count 0 100 100 40 OK: Always passing
C5 Current Pending Sector Count 0 100 100 0 OK: Always passing
C6 Off-Line Uncorrectable Sector Count 0 100 100 0 OK: Always passing
C7 Ultra ATA CRC Error Rate 0 200 200 0 OK: Always passing

 

[Rod Speed]

Sjouke Burry wrote

Yousuf Khan wrote

Everest Home edition does.

The purported 'analysis' is completely useless.

It show OK with even the most obvious problems.

A good example is the reallocated sector count
with your second drive, thats much too high.

Extract from report:

[ HDS722580VLAT20 (VNR21EC2T11N3M) ]
01 Raw Read Error Rate 60 100 100 0 OK: Value is normal
02 Throughput Performance 50 100 100 0 OK: Value is normal
03 Spin Up Time 24 113 113 11600047 OK: Value is normal
04 Start/Stop Count 0 91 91 37210 OK: Always passing
05 Reallocated Sector Count 5 100 100 0 OK: Value is normal
07 Seek Error Rate 67 100 100 0 OK: Value is normal
08 Seek Time Performance 20 100 100 0 OK: Value is normal
09 Power-On Time Count 0 98 98 18847 OK: Always passing
0A Spin Retry Count 60 100 100 0 OK: Value is normal
0C Power Cycle Count 0 100 100 1853 OK: Always passing
C0 Power-Off Retract Count 50 69 69 37833 OK: Value is normal
C1 Load/Unload Cycle Count 50 69 69 37833 OK: Value is normal
C2 Temperature 0 148 148 17, 37 OK: Always passing
C4 Reallocation Event Count 0 100 100 0 OK: Always passing
C5 Current Pending Sector Count 0 100 100 0 OK: Always passing
C6 Off-Line Uncorrectable Sector Count 0 100 100 0 OK: Always passing
C7 Ultra ATA CRC Error Rate 0 200 200 0 OK: Always passing
[ HDS722516VLAT80 (VNR43EC4GMYM6M) ]
01 Raw Read Error Rate 60 100 100 0 OK: Value is normal
02 Throughput Performance 50 100 100 0 OK: Value is normal
03 Spin Up Time 24 99 99 20185385 OK: Value is normal
04 Start/Stop Count 0 95 95 23580 OK: Always passing
05 Reallocated Sector Count 5 100 100 35 OK: Value is normal
07 Seek Error Rate 67 100 100 0 OK: Value is normal
08 Seek Time Performance 20 100 100 0 OK: Value is normal
09 Power-On Time Count 0 98 98 18811 OK: Always passing
0A Spin Retry Count 60 100 100 0 OK: Value is normal
0C Power Cycle Count 0 100 100 1838 OK: Always passing
C0 Power-Off Retract Count 50 81 81 23735 OK: Value is normal
C1 Load/Unload Cycle Count 50 81 81 23735 OK: Value is normal
C2 Temperature 0 177 177 18, 31 OK: Always passing
C4 Reallocation Event Count 0 100 100 40 OK: Always passing
C5 Current Pending Sector Count 0 100 100 0 OK: Always passing
C6 Off-Line Uncorrectable Sector Count 0 100 100 0 OK: Always passing
C7 Ultra ATA CRC Error Rate 0 200 200 0 OK: Always passing

 

[Yousuf Khan]

Yeah, it does display SMART values, but does not make any sort of
analysis of that data.

Yousuf Khan

Everest Home edition does.
Extract from report:

[ HDS722580VLAT20 (VNR21EC2T11N3M) ]

01 Raw Read Error Rate 60 100 100
0 OK: Value is normal
02 Throughput Performance 50 100 100
0 OK: Value is normal
03 Spin Up Time 24 113 113
11600047 OK: Value is normal
04 Start/Stop Count 0 91 91
37210 OK: Always passing
05 Reallocated Sector Count 5 100 100
0 OK: Value is normal
07 Seek Error Rate 67 100 100
0 OK: Value is normal
08 Seek Time Performance 20 100 100
0 OK: Value is normal
09 Power-On Time Count 0 98 98
18847 OK: Always passing
0A Spin Retry Count 60 100 100
0 OK: Value is normal
0C Power Cycle Count 0 100 100
1853 OK: Always passing
C0 Power-Off Retract Count 50 69 69
37833 OK: Value is normal
C1 Load/Unload Cycle Count 50 69 69
37833 OK: Value is normal
C2 Temperature 0 148 148 17,
37 OK: Always passing
C4 Reallocation Event Count 0 100 100
0 OK: Always passing
C5 Current Pending Sector Count 0 100 100
0 OK: Always passing
C6 Off-Line Uncorrectable Sector Count 0 100 100
0 OK: Always passing
C7 Ultra ATA CRC Error Rate 0 200 200
0 OK: Always passing

That's not the kind of analysis I'm talking about. It needs to be able
to see all of those numbers and come up with an overall health report.
Hard Disk Sentinel can do that.

Yousuf Khan