Lowest-Level Format?

[(PeteCresswell)]

IIRC from the SCSI days, there were different levels of
formatting - where the lowest level checked for bad sectors and
locked them out, effectively giving a 100%-good disk when done.

Fast-forwarding to today: one of my 1-TB SATA backup discs is
throwing errors on ChkDsk - *lots* of errors....

This is the second time for this disk. The first time, I did a
"Format", after which it made it through four incremental backups
with no problem (ChkDsk before each incremental...).

Before trashing it, I thought I'd see if there is anything "lower
level" in the way of formatting than:

- Bringing the disc up in Disk Management,

- Deleting it's only partition,

- Creating a new partition, and

- Formatting it with QuickFormat=False


Or should I just suck it up and spend fifty bucks on a new disc?

 

[Zaphod Beeblebrox]

IIRC from the SCSI days, there were different levels of
formatting - where the lowest level checked for bad sectors and
locked them out, effectively giving a 100%-good disk when done.

Fast-forwarding to today: one of my 1-TB SATA backup discs is
throwing errors on ChkDsk - *lots* of errors....

This is the second time for this disk. The first time, I did a
"Format", after which it made it through four incremental backups
with no problem (ChkDsk before each incremental...).

Before trashing it, I thought I'd see if there is anything "lower
level" in the way of formatting than:

- Bringing the disc up in Disk Management,

- Deleting it's only partition,

- Creating a new partition, and

- Formatting it with QuickFormat=False


Or should I just suck it up and spend fifty bucks on a new disc?

You could use the manufacturer's diagnostic software to check out the
disk to be sure, but it sounds like it is on its way out and you
should just "suck it up and spend fifty bucks on a new disc" as you
put it.

--
Zaphod

Pan-Galactic Gargle Blaster: A cocktail based on Janx Spirit.
The effect of one is like having your brain smashed out
by a slice of lemon wrapped round a large gold brick.

 

[John John MVP]

IIRC from the SCSI days, there were different levels of
formatting - where the lowest level checked for bad sectors and
locked them out, effectively giving a 100%-good disk when done.

Fast-forwarding to today: one of my 1-TB SATA backup discs is
throwing errors on ChkDsk - *lots* of errors....

This is the second time for this disk. The first time, I did a
"Format", after which it made it through four incremental backups
with no problem (ChkDsk before each incremental...).

Before trashing it, I thought I'd see if there is anything "lower
level" in the way of formatting than:

- Bringing the disc up in Disk Management,

- Deleting it's only partition,

- Creating a new partition, and

- Formatting it with QuickFormat=False


Or should I just suck it up and spend fifty bucks on a new disc?

In this day and age there is really no such thing as low level
formatting, some folks refer to the disk manufacturer's zero fill
utility as low level formatting but it really isn't.

Get a disk diagnostic utility from your disk manufacturer's web site and
see what it reports for your disk. You can run the manufacturer's zero
fill utility and it may re-evaluate the bad sector count. If chkdsk is
reporting bad sectors this indicates that the spare sector pool is
depleted and the drive is shot.

John

 

[Patok]

Get a disk diagnostic utility from your disk manufacturer's web site and
see what it reports for your disk. You can run the manufacturer's zero
fill utility and it may re-evaluate the bad sector count. If chkdsk is
reporting bad sectors this indicates that the spare sector pool is
depleted and the drive is shot.

That's the most common form of failure, but not the only one. I had
a drive where the electronics failed, not the drive. The effects were
weird - it would suddenly disconnect, entire sections - files and
folders - would disappear and then reappear. It was heat related - I
was able to retrieve the data from it by placing the drive in a
ziplock bag, and placing that in ice water, and kept adding ice when
it melted.

 

[Ken Blake, MVP]

Before trashing it, I thought I'd see if there is anything "lower
level" in the way of formatting than:

.....

Low-level formatting no longer exists.

Or should I just suck it up and spend fifty bucks on a new disc?

Yep!

 

[VanguardLH]

one of my 1-TB SATA backup discs is throwing errors on ChkDsk - *lots*
of errors....

Does the HDD support S.M.A.R.T.? If so, check the pending reallocation
count. If it's anything other than zero then the logic on the HDD has
already used up all the reserve sectors to mask out bad sectors and
there's no more room to mask out any more. Time for a new HDD.

http://en.wikipedia.org/wiki/S.M.A.R.T.#Known_ATA_S.M.A.R.T._attributes

Go look at the "Current Pending Sector Count" value. You'll need
something that can read the SMART data off your HDD.

 

[Paul]

Does the HDD support S.M.A.R.T.? If so, check the pending reallocation
count. If it's anything other than zero then the logic on the HDD has
already used up all the reserve sectors to mask out bad sectors and
there's no more room to mask out any more. Time for a new HDD.

http://en.wikipedia.org/wiki/S.M.A.R.T.#Known_ATA_S.M.A.R.T._attributes

Go look at the "Current Pending Sector Count" value. You'll need
something that can read the SMART data off your HDD.

Current Pending Sector Count

Count of "unstable" sectors (waiting to be remapped, because of read errors).

If an unstable sector is subsequently read successfully, this value is decreased
and the sector is not remapped.

Read errors on a sector will not remap the sector (since it might be readable
later); instead, the drive firmware remembers that the sector needs to be remapped,
and remaps it the next time it's written.

That makes the statistic, a measure of how many questionable sectors,
need to be "processed" on the next write attempt to them. Seeing
a high count (100's +) implies the health of the disk is going
downhill. Writing the disk, from end to end, would resolve the
count one way or another, and whatever happens would be recorded
in some other statistic.

But with normal ratios of reads to writes, and content change
on the disk, seeing a count grow there, implies the writes
aren't being delivered at a high enough rate, and to the
right places, to fix it all up again.

That is the variable I use, to determine when to replace drives
on my systems here. I like it as a "flakiness metric".

*******

A low level format doesn't exist on modern disks, because they
use embedded servo, a pattern written on the disk at the factory.
It can't be re-written by the user. The user can write zeros
to each sector data area if they want, but that is about it.

On very old drives, one surface was dedicated to servo, the other
surfaces were basically synchronized to that servo surface. A low level
format, writes the N-1 surfaces with both new sector headers and zero data.
So that's a kind of "soft format". And works as long as the
servo surface is healthy, and the alignment of the platters
isn't screwed up. Such a scheme stopped being practical, as
the data density went up.

Embedded servo, means the servo pattern is detectable locally,
and the mechanical tolerances in the spindle are less of an issue.
(And with FDB motors, the runout on the spindle is also a lot less.)

*******

In terms of erasure, zeroing the disk, fills all the sectors
a user can access. When a sector is spared out, the data in
the spared out sector is "trapped". But, the standards say,
that an attempt to use the built-in Secure Erase feature,
will try to zero both the user visible sectors, and the
sectors that are no longer being used. So if you thought an
important credit card number or password, were hiding on a
half dead disk, the Secure Erase is the closest thing to
removing it (short of a sledge hammer or shredder). If the disk
isn't functional enough to complete the Secure Erase, you
can always shred it later.

Paul

 

[Patok]

A low level format doesn't exist on modern disks, because they
use embedded servo, a pattern written on the disk at the factory.
It can't be re-written by the user. The user can write zeros
to each sector data area if they want, but that is about it.

On very old drives, one surface was dedicated to servo, the other
surfaces were basically synchronized to that servo surface. A low level
format, writes the N-1 surfaces with both new sector headers and zero data.
So that's a kind of "soft format". And works as long as the
servo surface is healthy, and the alignment of the platters
isn't screwed up. Such a scheme stopped being practical, as
the data density went up.

How old are you talking here? Was that scheme used on the MFM
drives? I remember doing low level format on such, but thought that
they were like floppies in that regard - all soft-formatted. I could
be wrong, though.
It was fun times - I had a "turbo" XT clone with a 20MB MFM drive,
and the CPU was a NEC V20. It could emulate 8080 and ran CP/M natively
under DOS. The FDD was a highly nonstandard 5" double-density
80-track, and I custom-formatted 800K floppies on it (10 sectors per
track instead of 9). I still have a box of them somewhere in storage.

 

[(PeteCresswell)]

Per VanguardLH:

Go look at the "Current Pending Sector Count" value. You'll need
something that can read the SMART data off your HDD.

I use Hard Disk Sentinel, I see this: http://tinyurl.com/3lqyp72

Although "Current Pending Sector Count" = 0, some of the other
figures sound a little ominous to somebody like me who knows
nothing.....

I'll dig up the manufacturer's utility and turn it loose next.

 

[VanguardLH]

Per VanguardLH:

I use Hard Disk Sentinel, I see this: http://tinyurl.com/3lqyp72

Although "Current Pending Sector Count" = 0, some of the other
figures sound a little ominous to somebody like me who knows
nothing.....

I'll dig up the manufacturer's utility and turn it loose next.

I could recommend SpinRite if you want to qualify the HDD but then the
cost of SpinRite is the same, or more, than getting a replacement HDD.
SpinRite is something you get to use on more than one HDD, like you
support the workstations at work or a computer shop, or you just like to
have a big toolbox of diagnostic software (i.e., you collect diag tools
like some gals collect shoes and like some guys collect gadgets).

 

[Bob Willard]

How old are you talking here? Was that scheme used on the MFM drives?

I'm still using a ST31230N 1 GB HD on one (Win98) desktop; 3 platters, 5
R/W heads, plus one head for the servo surface. The date of manufacture
on that HD seems to be 1995; the manual is dated 1994.

Since I don't normally let my HDs power down and since I normally run my
PCs 24x7, that 1GB HD has about 15 years of power-on, spinning, runtime
-- and it still works. Lots of other stuff in that desktop has been
replaced, but that HD just keeps on turning.

FWIW, that desktop has two other HDs, 2 FDs, and 2 CDs. The 1 GB HD
holds the pagefile. One HD for code, one for data, and one for the
pagefile plus some temps, turned out to give the best performance for
the workload on what was once my primary desktop.

 

[Patok]

I'm still using a ST31230N 1 GB HD on one (Win98) desktop; 3 platters, 5
R/W heads, plus one head for the servo surface. The date of manufacture
on that HD seems to be 1995; the manual is dated 1994.

That's too new. First of all it's 3.5 inch, not 5, and second, it's
SCSI. It seems that internally it uses one of the RLL recording
methods, and indeed appears to have a servo surface. I was curious
about the older 5 inch drives, where you had to have a specific
controller to attach it - they were not compatible. I think the one I
used had what was called MFM and the others - something RLL.

Since I don't normally let my HDs power down and since I normally run my
PCs 24x7, that 1GB HD has about 15 years of power-on, spinning, runtime
-- and it still works. Lots of other stuff in that desktop has been
replaced, but that HD just keeps on turning.

With a MTBF of 800,000 power-on hours, you can expect it to outlive
*you*.

 

[Paul]

Bob Willard wrote:

That's too new. First of all it's 3.5 inch, not 5, and second, it's
SCSI. It seems that internally it uses one of the RLL recording methods,
and indeed appears to have a servo surface. I was curious about the
older 5 inch drives, where you had to have a specific controller to
attach it - they were not compatible. I think the one I used had what
was called MFM and the others - something RLL.

Back when drives were a bit smaller in capacity, the interface was called SASI.
The disks we had on our computers, had two PCBs. One was a board that
bolted to the drive (that was the SASI part) and a board that plugged
into the host computer.

http://en.wikipedia.org/wiki/Shugart_Associates_System_Interface#History

We never got to see the pricing of these things, but I think somewhere
in the past, I saw a reference to a price over $1000 for this generation.

http://en.wikipedia.org/wiki/ST506

The drives were "full height", and a lot taller than a modern desktop drive.

http://media.bestofmicro.com/seagate-st412-harddrive-hdd,S-B-217451-13.jpg

Paul

 

[Patok]

Back when drives were a bit smaller in capacity, the interface was
called SASI.
The disks we had on our computers, had two PCBs. One was a board that
bolted to the drive (that was the SASI part) and a board that plugged
into the host computer.

http://en.wikipedia.org/wiki/Shugart_Associates_System_Interface#History

Aha! Thanks to your links, I remembered which drive I was using -
it was a ST-225, with an MFM controller card. Like here:

http://upload.wikimedia.org/wikipedia/commons/4/4f/ST_225_20MB_drive_and_WDC_controller.jpg

Looking at the specs, it seems it did *not* have a servo surface. It
had 2 platters and 4 heads, 615 cylinders equal to 2,460 tracks. With
17 sectors per track - 41,820 sectors and 20MB. So apparently all
soft-formatted, no servo surface, and I certainly remember formatting
it low level. How did I do that though - I seem to remember using
Debug and then calling an entry point in the Bios?

 

[Ken Blake, MVP]

Aha! Thanks to your links, I remembered which drive I was using -
it was a ST-225, with an MFM controller card.

I remember the ST225. It was what I had on my first personal computer,
in 1987. It was 20MB for about $200 US, or $10 per megabyte.

Today you can buy a 2TB drive for about $75. That's 8/10 of a penny
per megabyte, a *very* big improvement in price.

 

[Paul]

Aha! Thanks to your links, I remembered which drive I was using - it
was a ST-225, with an MFM controller card. Like here:

http://upload.wikimedia.org/wikipedia/commons/4/4f/ST_225_20MB_drive_and_WDC_controller.jpg


Looking at the specs, it seems it did *not* have a servo surface. It had
2 platters and 4 heads, 615 cylinders equal to 2,460 tracks. With 17
sectors per track - 41,820 sectors and 20MB. So apparently all
soft-formatted, no servo surface, and I certainly remember formatting it
low level. How did I do that though - I seem to remember using Debug and
then calling an entry point in the Bios?

The available info seems to suggest ST225 was more like a floppy. The
head movement was controlled by a stepper. And you could write your
own format on it (MFM or RLL). So the head position was open loop.
Considering the low density, the ST412 and ST506 could have moved
their heads with a stepper too. And you're right, when you format
something like that, you get to define every bit of the magnetic
content. For a reference, all you'd need is something to use as
an index mark.

The two servo schemes (separate surface or embedded servo) allow
a voice coil to be used to move the head. Servo gives you a closed
loop response (feedback about where you're going). Separate servo
surface works up to a certain track per inch density, and then
embedded takes over from there.

http://www.pcguide.com/ref/hdd/op/actActuator-c.html

So I guess I missed one. Possible combinations:

1) Stepper to move the head, completely soft format.
2) Voice coil, separate servo surface, tracks on all other surfaces
would align with the servo pattern. Soft format for
the data surfaces. Necessary once the tracks per inch
starts to climb.
3) Voice coil, embedded servo (interleaved, on every surface).
Factory formatted, with whatever constitutes a sector header
being permanent, and a "low level format" really doesn't exist
any more. Portions of the pattern are written once at the factory
and that's it. A format operation, only gets to zero the data portion
of each sector.

If you look at drives around 250GB or so, the density is still
low enough, they use a "servo writer" at the factory. There is
a silver sticker on the side of the drive, covering a hole
that the servo writer arm pokes through. At the factory, the
servo writer applies the servo pattern to the platters. And then
later, the drive's own heads will follow that embedded pattern.

On the newest drives, that hole is gone. I read somewhere, that
they now write the servo pattern, using the drive's own head.
And some factory equipment plays a part in controlling the process.
That suggests it is theoretically possible, for a 3TB drive,
to rewrite every part of the platters (embedded servo and all).
But whether it can do that, depends on whether the drive itself
has the ability to precisely position the head during the write
process. If there was no servo pattern to work with, then some
other means would be needed to measure head position. (Like maybe
a laser interferometer.)

Since the Hitachi technical web pages disappeared, the only
decent pages describing some of this stuff have disappeared.
And no other web sites, bother to explain some of these options.
For example, the Hitachi web site had a nice page describing the
lubricant they use on the platters now. It's only a couple
molecules thick.

Paul

 

[(PeteCresswell)]

Per Yousuf Khan:

Look at HD Sentinel's Overview tab, that should tell you everything you
need to know, it interprets the SMART data for you. From what I see
here, there's nothing wrong with that drive at the low level.

Since this is a backup disk, is it an external disk? If it is, then what
is your procedure for removing the disk?

Thanks.

I also ran the mfr's utility and HD Tune - after formatting - and
both said it was a-ok.

Yes, external in the sense that it gets slid into a SATA bay or
inserted into an external "NexStar" toaster-like device.

I have both drives set to not cache data.

On the external toaster-like device, I just turn it off and
remove the drive.

On the bay, I don't do anything except just pull the drive.

But I have five other drives being similarly abused and this is
the only one that's had problems.... so far.... but it's been
about a year of swapping drives every 2-3 days.

Each time I swap, the replacement drive gets a ChkDsk.

I think I'll just keep running the thing.

 

[Yousuf Khan]

Yes, external in the sense that it gets slid into a SATA bay or
inserted into an external "NexStar" toaster-like device.

I have both drives set to not cache data.

On the external toaster-like device, I just turn it off and
remove the drive.

On the bay, I don't do anything except just pull the drive.

But I have five other drives being similarly abused and this is
the only one that's had problems.... so far.... but it's been
about a year of swapping drives every 2-3 days.

Each time I swap, the replacement drive gets a ChkDsk.

I think I'll just keep running the thing.

What I was trying to get at here is whether or not you use the "Safely
Remove Hardware" wizard before removing these drives.

Yousuf Khan

 

[(PeteCresswell)]

Per Yousuf Khan:

What I was trying to get at here is whether or not you use the "Safely
Remove Hardware" wizard before removing these drives.

No, I do not normally do that.

Used to, but I think the internal sled/bay that I use most does
not even offer it. Both devices are eSata connections.

I take your implied point - but keep coming back to only one out
of six drives being affected. OTOH, that is "So far....." -)

 

[Yousuf Khan]

Per Yousuf Khan:

No, I do not normally do that.

Used to, but I think the internal sled/bay that I use most does
not even offer it. Both devices are eSata connections.

I take your implied point - but keep coming back to only one out
of six drives being affected. OTOH, that is "So far....." -)

If you converted your XP over to using the AHCI drivers rather than the
IDE emulation drivers, you should be able to mark these eSATA drives as
removable drives. Safely Remove Hardware wizard should then work on these.

Yousuf Khan