External FireWire/USB RAID?

[Alex Vilner]

Hello,

It is tough to find a single technical resource to answer all the
questions, so this group seems like the best place )

I have a P-4 machine (3Ghz, 1.5 Gig RAM), with 2 hard drives, 2 CD/DVD
drives (one reader, one writer), so my IDE connections are maxed out.
The 2 hard drives are about 120 Gig each. The primary bootable drive
has the OS, all apps and personal files. The second drive is split
into two partitions, one is for temp storage (swap file, and the
junk), and one for backup, onto which I regularly do DriveImage of the
primary drive. With digital photography and ever-increasing size of
image files, I am beginning to see the need to expand on the storage,
while maintaining a good backup of the data.

The machine has 6 USB 2.0 connections and a FireWire port. Most of
USBs are used (2 printers, PDA, camera, card reader, mouse, etc..) but
the firewire is available, and so are 2-3 PCI slots.

I am looking for an EXTERNAL solution that would allow the following:
1. RAID-1 (for recoverability. I guess, mirroring is the best thing I
can do, unless I went with RAID-5).
2. Fairly fast excess (by fairly fast I mean comparable to internal
drive access, since the data files will pretty much be migrated over).
3. Accessible in the PC-DOS mode (I have yet to figure out another way
of doing drive image of an active Windows XP drive).
4. Sharing among home networked computers.
5. Growth potential -- so, probably, at least 4 disks of 250-300Gig
each... This, with mirroring should provide 500-600Gig usable, which
is ok in my book

I have seen Firewire/USB drive enclosures, that go for $120-200,
without disks... So this may be a good option, other than building a
new desktop for just the storage...

Now, for questions:
Am I nuts to consider this? What other people have done to address
growing storage needs and the reliability of storage? Will Windows XP
Pro allow me to do dynamic disk and RAID-1 (or 5) on the
Firewire-connected drive?

Any suggestions/references would be greatly appreciated.

Thank you in advance!

--Alex

 

[DanielEKFA]

Hello,

It is tough to find a single technical resource to answer all the
questions, so this group seems like the best place )

I have a P-4 machine (3Ghz, 1.5 Gig RAM), with 2 hard drives, 2 CD/DVD
drives (one reader, one writer), so my IDE connections are maxed out.
The 2 hard drives are about 120 Gig each. The primary bootable drive
has the OS, all apps and personal files. The second drive is split
into two partitions, one is for temp storage (swap file, and the
junk), and one for backup, onto which I regularly do DriveImage of the
primary drive. With digital photography and ever-increasing size of
image files, I am beginning to see the need to expand on the storage,
while maintaining a good backup of the data.

The machine has 6 USB 2.0 connections and a FireWire port. Most of
USBs are used (2 printers, PDA, camera, card reader, mouse, etc..) but
the firewire is available, and so are 2-3 PCI slots.

I am looking for an EXTERNAL solution that would allow the following:
1. RAID-1 (for recoverability. I guess, mirroring is the best thing I
can do, unless I went with RAID-5).
2. Fairly fast excess (by fairly fast I mean comparable to internal
drive access, since the data files will pretty much be migrated over).
3. Accessible in the PC-DOS mode (I have yet to figure out another way
of doing drive image of an active Windows XP drive).
4. Sharing among home networked computers.
5. Growth potential -- so, probably, at least 4 disks of 250-300Gig
each... This, with mirroring should provide 500-600Gig usable, which
is ok in my book

I have seen Firewire/USB drive enclosures, that go for $120-200,
without disks... So this may be a good option, other than building a
new desktop for just the storage...

Now, for questions:
Am I nuts to consider this? What other people have done to address
growing storage needs and the reliability of storage? Will Windows XP
Pro allow me to do dynamic disk and RAID-1 (or 5) on the
Firewire-connected drive?

Any suggestions/references would be greatly appreciated.

Thank you in advance!

--Alex

With the kind of dough you'd put out on four external drives of that size of
which you'd only have storage space equivalent to two of them since you'd
be running RAID1, I'd say you'd be much better off (and save some money)
with an internal RAID5 capable card and four drives of which you'd have
space equivalent to three. Finding a hardware USB or Firewire solution that
can do RAID would be difficult if not impossible, so you'd be using
software RAID, which would strain your processor. Also, even if you have 6
USB plugs and one or more Firewire plugs you can be pretty certain that
they're running off of the same hub, so you'd have to buy an extra Firewire
card for each disk to prevent bottlenecks. I say Firewire, because for disk
use Firewire is preferable to USB even if the speed is 80 Mbps lower,
because Firewire is unmanaged whereas USB is managed (ie. USB uses your
processor a lot while transferring, Firewire does not). Also, the transfer
bandwidth is unlikely to be maxed out anyway.

But back to the point: If you can do with old school ATA instead of SATA, I
can heartily recommend the Promise Fastrack SX4000 or 8000 (four or eight
drives supported). It has hardware RAID5 (XOR processor on-board), up to
256 MB buffer, it's realiable and cheap. I own one, and I can confirm that
it does work in DOS albeit at reduced speed. I use DOS based Ghost to make
backups of a 6GB XP system partition and that takes over an hour, so you
can see speed isn't impressive in DOS, I think because the card runs in
some kind of compatibility mode. But anyway, if you're doing backup of a
system drive you'd probably like it to be as small as possible anyway, so
you're probably already installing programs to a different partition,
right?

I have to say that I specifically looked for an ATA RAID5 card when I bought
mine, because I already owned three identical ATA disks, and I wanted
something cheaper too. So looking for a S-ATA RAID5 drive is probably a
good idea, just make sure that a) There's an XOR processor on-board so it
doesn't use your CPU, and b) One channel per drive is preferable to two,
meaning that you'd like e.g. four master drives, not two masters and two
slaves. The Fastrack cards have four and eight channels respectively, so
there's a dedicated channel for each drive.

HTH,
Daniel

 

[DanielEKFA]

With the kind of dough you'd put out on four external drives of that size
of which you'd only have storage space equivalent to two of them since
you'd be running RAID1, I'd say you'd be much better off (and save some
money) with an internal RAID5 capable card and four drives of which you'd
have space equivalent to three. Finding a hardware USB or Firewire
solution that can do RAID would be difficult if not impossible, so you'd
be using software RAID, which would strain your processor. Also, even if
you have 6 USB plugs and one or more Firewire plugs you can be pretty
certain that they're running off of the same hub, so you'd have to buy an
extra Firewire card for each disk to prevent bottlenecks. I say Firewire,
because for disk use Firewire is preferable to USB even if the speed is 80
Mbps lower, because Firewire is unmanaged whereas USB is managed (ie. USB
uses your processor a lot while transferring, Firewire does not). Also,
the transfer bandwidth is unlikely to be maxed out anyway.

But back to the point: If you can do with old school ATA instead of SATA,
I can heartily recommend the Promise Fastrack SX4000 or 8000 (four or
eight drives supported). It has hardware RAID5 (XOR processor on-board),
up to 256 MB buffer, it's realiable and cheap. I own one, and I can
confirm that it does work in DOS albeit at reduced speed. I use DOS based
Ghost to make backups of a 6GB XP system partition and that takes over an
hour, so you can see speed isn't impressive in DOS, I think because the
card runs in some kind of compatibility mode. But anyway, if you're doing
backup of a system drive you'd probably like it to be as small as possible
anyway, so you're probably already installing programs to a different
partition, right?

I have to say that I specifically looked for an ATA RAID5 card when I
bought mine, because I already owned three identical ATA disks, and I
wanted something cheaper too. So looking for a S-ATA RAID5 drive is
probably a good idea, just make sure that a) There's an XOR processor
on-board so it doesn't use your CPU, and b) One channel per drive is
preferable to two, meaning that you'd like e.g. four master drives, not
two masters and two slaves. The Fastrack cards have four and eight
channels respectively, so there's a dedicated channel for each drive.

HTH,
Daniel

Oh, one more reason I bought ATA: Unless you have PCI-X, you won't get any
transfer speed increase from SATA, since the bottleneck would be the PCI
bus. If you went with 10000+ RPM SATAs though, you'd get better seek times,
but for 7200 RPM drives on a regular PCI bus, it wouldn't matter if you got
ATA or S-ATA. Someone please correct me if I'm wrong here.

 

[Folkert Rienstra]

FireWire connected External Raid may be available.

Over FireWire?

FireWire Hubs?

And what do you think your processor is doing during IO?

Also, the transfer bandwidth is unlikely to be maxed out anyway.

The bandwidth is lower than that of PCI and you ramble about maxing out PCI?

Or just plain DOS mode. Learn how to use DOS to get your speed back.

Card.

Clueless. S-ATA *is* one drive per channel.

There is no master slave in S-ATA.

Oh, one more reason I bought ATA: Unless you have PCI-X, you won't get any
transfer speed increase from SATA, since the bottleneck would be the PCI
bus.

Clueless. There is no difference between ATA and SATA re PCI.

If you went with 10000+ RPM SATAs though, you'd get better seek times,
but for 7200 RPM drives on a regular PCI bus, it wouldn't matter if you got
ATA or S-ATA.

Someone please correct me if I'm wrong here.

Done. You even corrected yourself.

 

[Alex Vilner]

Daniel,

Thank you for the detailed response. Unfortunately, in my case the
internal shelves are already used up (2 hard drives, 2 CD/DVD drives).
These contain the data, the OS, etc. -- so I cannot resort to using
the internal RAID-5 solution... The only other option is to build a
separate machine with a large case, slow CPU, little memory, fast
network card, to serve as just the storage machine. But the cost of
hard drives will still remain an issue...

I am not specifially set on SATA -- the understanding is that this is
a backup/storage growth solution for files that will be primarily
read, and infrequently at that. I can tolerate some performance
decreases...

Dough-wise, the cheaper, the better, obviously. If I can get a
firewire enclosure for 4 disks for <$200, and go with 2 disks to start
with (250Gb @ $150 each) this makes it into a $500 solution, with
future growth potential (another two disks at a later time, also
mirrored, when storage becomes cheaper, when I outgrow the first
pair).

I have to go external, be it a machine or a storage box, so for me the
overhead of building a machine used for just storage is not
justifyable (plus I will most likely want to upgrade the network and
the router to a gigabit)..

Something like this:
http://www.cooldrives.com/fi80qu5fitoi.html

Then the question is, will Windows XP Pro be able to do RAID-5 on this
device??

Thanks again.

--Alex

 

[Al Dykes]

Daniel,

Thank you for the detailed response. Unfortunately, in my case the
internal shelves are already used up (2 hard drives, 2 CD/DVD drives).
These contain the data, the OS, etc. -- so I cannot resort to using
the internal RAID-5 solution... The only other option is to build a
separate machine with a large case, slow CPU, little memory, fast
network card, to serve as just the storage machine. But the cost of
hard drives will still remain an issue...

If you have an available PCI slot you can but an IDE card for a few
bucks that will give you 4 or more disk channels.

IME, USB is not usable as a real disk drive. performace will suck.
Using a collection of external USB/Fw disks as a raid array sounds
like a real good way to lose data as soon as a power or data cable
comes unplugged. A raid rebuild is slow enough on an real disk. It
would be really slow on an IDE channel. It wouldn't suprise me that
software RAID tools won't let you use a FW/USB device as a raid
member. never tried it.

Go internal SATA.

You can get one of these to put several disk in your chassis;

http://makeashorterlink.com/?R457324C9

http://www.adaptec.com/worldwide/product/proddetail.html?sess=no&language=English
+US&prodkey=AAR-2410SA_ENCL&cat=%2fTechnology%2fRAID%2fSerial+ATA+RAID

Other companies make similar products.

 

[Eric Gisin]

You can certainly get Firewire RAID 2-4 drive enclosures, but they are pricey.
DOS drivers are available for OHCI. Performance will be comparable to a 5400
rpm drive.

Only backup to externals drives. A PSU failure may kill your internals.

Another alternative is build a server with an old Pentium II, using Win 200X
server or Linux for RAID 5. This is cheaper than RAID 5 controllers or
enclosures.

 

[DanielEKFA]

FireWire connected External Raid may be available.

Which is what I said.

Over FireWire?

Over any protocol. If you don't have hardware RAID, the other option is
software RAID. See how it works?

FireWire Hubs?

Yeah, or "channel" or whatever the word is. You're not exactly helping here,
if you have the right word, write it!

And what do you think your processor is doing during IO?

Less with Firewire than it is with USB. Firewire features a P2P-like
topology that doesn't require management like USB does with its star
topology where nothing can happen without the star. One Firewire peer can
communicate completely un-managed (ie. without a star, or if you like,
computer) with another peer, whereas USB cannot. Everything goes through
the star. I was told that this would significantly decrease CPU usage,
although studying up on it, the difference seems to be only about 10-15% to
Firewire's advantage, so it may not be the decisive factor. But either way,
Firewire is still much faster than USB2.0 as you can see for instance in
this test where Firewire 400 outperforms USB2.0 by as much as 70%:

http://www.g4techtv.com/freshgear/features/39129/USB_20_Versus_FireWire_pg3.html

The bandwidth is lower than that of PCI and you ramble about maxing out
PCI?

You really are a very bad reader, aren't you? Where exactly do you see me
saying PCI? I've been talking about Firewire the whole paragraph, and you
think I'm talking about PCI bandwidth? Now what kind of feeble brain would
draw that conclusion? But allow me to help you understand: Even if the Mbps
figures used in the Firewire and USB specs were more than theoretical
(read: accurate), then whether you had 480 Mbps or 400 Mbps wouldn't make
much of a difference since the drives used in external solutions won't be
reading or writing data at either speed anyway, except for short bursts
(and still Firewire is faster). Get it now or should I draw you a picture?

Or just plain DOS mode. Learn how to use DOS to get your speed back.

I've used DOS since 1989, and I think I know DOS pretty well. So unless you
know this particular card and have a trick, your comment isn't exactly
useful. But just for kicks, let me know what it is you think would help in
DOS to get full performance with such a card.

Clueless. S-ATA *is* one drive per channel.


There is no master slave in S-ATA.

Well that's good, then that won't be an issue.

Clueless. There is no difference between ATA and SATA re PCI.

Huh? How retarded are you trying to show yourself off as? PCI offers 132
MBps, PCI-X 1064 MBps. You wouldn't call that a speed difference?
Especially taking into account that all PCI cards (on-board and in slots)
are sharing that one bus speed? Have you even considered the fact that a
decent RAID5 card will sport a large buffer, like the SX's 256 MB buffer
capability? Or do you need more clues?

Done. You even corrected yourself.

I'm not sure who - if any - taught you communicative skills, but you should
definitely go back to school and learn to actually express the corrections
rather than calling people "clueless" and generally appearing like a
smart-assed ****. Or perhaps you really *are* a smart-assed ****? Because
for sure you haven't been very kind nor very helpful. Out of the handful of
comments you've made to my post, only one has been informative, the rest
has just been bashing (and/or incorrect). You don't belong in a forum, you
belong in a bar.

 

[DanielEKFA]

Daniel,

Thank you for the detailed response. Unfortunately, in my case the
internal shelves are already used up (2 hard drives, 2 CD/DVD drives).
These contain the data, the OS, etc. -- so I cannot resort to using
the internal RAID-5 solution... The only other option is to build a
separate machine with a large case, slow CPU, little memory, fast
network card, to serve as just the storage machine. But the cost of
hard drives will still remain an issue...

Or, buy a new chassis? I totally respect and understand if you go with the
external solution, I just think the internal one is cheaper, more
compatible (good for when you've had enough of Windows and start
socializing with a certain penguin , and faster.

I am not specifially set on SATA -- the understanding is that this is
a backup/storage growth solution for files that will be primarily
read, and infrequently at that. I can tolerate some performance
decreases...

I have the same scenario here, the speed is not my first priority, I just
needed 1.2 TB of space for data that must to be always available and
secure. So software RAID rather than hardware RAID could very well be the
logical choice for you. Although it *is* pretty cool when you copy a 700 MB
AVI file from and to the same internal RAID-5 array in 6 seconds

Dough-wise, the cheaper, the better, obviously. If I can get a
firewire enclosure for 4 disks for <$200, and go with 2 disks to start
with (250Gb @ $150 each) this makes it into a $500 solution, with
future growth potential (another two disks at a later time, also
mirrored, when storage becomes cheaper, when I outgrow the first
pair).

I have to go external, be it a machine or a storage box, so for me the
overhead of building a machine used for just storage is not
justifyable (plus I will most likely want to upgrade the network and
the router to a gigabit)..

Something like this:
http://www.cooldrives.com/fi80qu5fitoi.html

You know, I hadn't even considered this kind of setup, I just had an
assumption that we were talking a bunch of external drives like Maxtor or
LaCie make them (silly me), but this solution could be *very* cool indeed.
Especially with Firewire 800.

Then the question is, will Windows XP Pro be able to do RAID-5 on this
device??

I'm not much into software RAID, but AFAIK Windows can't do it natively
anyway, so you're gonna need a third-party app to do it. So basically, any
drive that can be recognized by Windows would be a candidate for software
RAID. Just make sure that your enclosure doesn't do anything JBOD'ish (like
some LaCie Firewire drives, where more than one disk is used to simulate
one larger disk). If it did, you wouldn't be able to use it for RAID.

 

[DanielEKFA]

Actually, something like this might be of interest. It's neither USB nor
Firewire, but S-ATA in a breakout box (watch line wrapping on the link):

http://www.adaptec.com/worldwide/pr...glish+US&prodkey=AAR-2410SA&source=prodfinder

Basically internal RAID but with the drives in a box outside the case.

 

[Folkert Rienstra]

Mission accomplished.

[snip]

 

[Sayso Takewashi]

Mission accomplished.

[snip]

By adding one or two words to his comments,you dont make an asset to
usenet!

Since he replied with clearfull statements to your comments,i think,he
is not the younger age boy,you used to think off...



You loose!

 

[Folkert Rienstra]

Mission accomplished.

[snip]

By adding one or two words to his comments,you dont make an asset to
usenet!

Since he replied with clearfull statements to your comments,

Why don't you check out what he said and discover that most of what
he said was deliberate misinterpretation of my comments, 2 plain lies,
a complete misunderstanding of point-to-point (vs multidrop?) and it's
supposed relationship to Processor usage, total misunderstanding of what
Serial ATA is and a completely false perception of it's relation to PCI.

i think,

Don't think, check.

he is not the younger age boy, you used to think off...

Of course he is when calling people "retards" and "cunts" where
I only said that he had 'less' than a 'clue'.
It took only a few comments to drill through his "I'm such a nice and
helpful person" -posture and reveal the Troll that he really is. By
answering him I would have given him credit that he doesn't deserve
and that is why I didn't and refrained to "Mission Accomplished".

 

[DanielEKFA]

Mission accomplished.

[snip]

By adding one or two words to his comments,you dont make an asset to
usenet!

Since he replied with clearfull statements to your comments,

Why don't you check out what he said and discover that most of what
he said was deliberate misinterpretation of my comments, 2 plain lies,
a complete misunderstanding of point-to-point (vs multidrop?) and it's
supposed relationship to Processor usage, total misunderstanding of what
Serial ATA is and a completely false perception of it's relation to PCI.

i think,

Don't think, check.

he is not the younger age boy, you used to think off...

Of course he is when calling people "retards" and "cunts" where
I only said that he had 'less' than a 'clue'.
It took only a few comments to drill through his "I'm such a nice and
helpful person" -posture and reveal the Troll that he really is. By
answering him I would have given him credit that he doesn't deserve
and that is why I didn't and refrained to "Mission Accomplished".

You loose!

If I were a troll, I'd be laughing now, since you'd be doing my dance. But
the truth is, I see that now, that you are a troll.

*PLONK*

 

[Alex Vilner]

Ok, so, let's say I bite the bullet and go with the external solution,
not the firewire/USB, but rather an old pentium machine, with a
Gigabit network connection to the one I have in place.

Then, using hardware, I can get the RAID levels I need. How will I be
able to create disk images from my main box, though? Again, I figure
any drivers I need to load are in Windows. If the image is created in
PC-DOS mode, how will that work? Has anybody done this before?

Thanks!

 

[Tortuga]

Daniel,

Thank you for the detailed response. Unfortunately, in my case the
internal shelves are already used up (2 hard drives, 2 CD/DVD drives).
These contain the data, the OS, etc. -- so I cannot resort to using
the internal RAID-5 solution... The only other option is to build a
separate machine with a large case, slow CPU, little memory, fast
network card, to serve as just the storage machine. But the cost of
hard drives will still remain an issue...

I am not specifially set on SATA -- the understanding is that this is
a backup/storage growth solution for files that will be primarily
read, and infrequently at that. I can tolerate some performance
decreases...

Dough-wise, the cheaper, the better, obviously. If I can get a
firewire enclosure for 4 disks for <$200, and go with 2 disks to start
with (250Gb @ $150 each) this makes it into a $500 solution, with
future growth potential (another two disks at a later time, also
mirrored, when storage becomes cheaper, when I outgrow the first
pair).

I have to go external, be it a machine or a storage box, so for me the
overhead of building a machine used for just storage is not
justifyable (plus I will most likely want to upgrade the network and
the router to a gigabit)..

Something like this:
http://www.cooldrives.com/fi80qu5fitoi.html

Then the question is, will Windows XP Pro be able to do RAID-5 on this
device??

Strangely enough, I'm researching nearly the same thing. I am presently
using an external 2 drive firewire enclosure connected to an XP-Pro box as a
small (100GB) storage unit. I want to create a large (around 1 TB) music
storage system (mostly read-only), using an additional firewire enclosure
with the capability of some "RAID"ish style redundancy.

Here is what I've found so far:
1) There is a registry tweak in WinXP-Pro to allow for dynamic disks using
1394 firewire connections
http://support.microsoft.com/default.aspx?scid=kb;en-us;299598&sd=tech
2) WinXP-Pro can be setup to use dynamic disks in RAID-5 mode
http://www4.tomshardware.com/storage/20041119/index.html
3) Cooldrives.com has a good selection of external firewire 400/800
enclosures. The one I am looking at is:
http://www.cooldrives.com/fi1340mu4bay.html
4) I believe the latest version of Ghost does support dynamic drives, so
drive imaging should not be a problem.

I plan to put in four 300 GB disks total (3 data + 1 parity under RAID-5).
Since speed is not my main concern, this seems the lowest cost solution
(i.e. 1 enclosure + 4 drives).

Hope this helps.