SCSI vs SATA Hih-Perf

[chrisv]

You need to catch up with the times. You are correct about the original
Itaniums being dogs, but I'm talking about the new Itanium2 processors,
which are also 64-bit. As for Intel being expensive, you get what you pay
for. The new Itanium2 sytems are SWEEEEEEET!

Ignore the troll, too stupid to understand what Arno meant by "dead
technology".

 

[lmanna]

Arno Wagner wrote:

In comp.sys.ibm.pc.hardware.storage J. Clarke
Arno Wagner wrote:

In comp.sys.ibm.pc.hardware.storage (e-mail address removed) wrote:
Hello all,
[...]
One thing you can be relatively sure of is that the SCSI controller
will work well with the mainboard. Also Linux has a long history of
supporting SCSI, while SATA support is new and still being worked on.

If he's using 3ware host adapters then "SATA support" is not an
issue--that's handled by the processor on the host adapter and all that
the Linux driver does is give commands to that processor.

Do you have any evidence to present that suggests that 3ware RAID
controllers have problems with any known mainboard?

No. I was mostly thinking of SMART support, which is not there
for SATA on Linux (unless you use the old IDE driver). Normal disk
access works fine in my experience.

Actually, that would be a function of the 3ware drivers. With a 3ware
host adapter you do not use the SATA drivers, you use drivers specific to
3ware, and the 3ware drivers _do_ support SMART under Linux.

And, does that work reliably and with the usual Linux tools,
i.e. smartctl? Would kind of surprise me, since libata does
not have smart support at all at the moment, since the ATA
passthru opcodes have only very recently be defined by the
SCSI T10 committee.

Yes, it does. That is _specifically_ addressed by the 3ware drivers.

You seem hung up on the idea that the 3ware host adapters use the generic
ATA drivers. They do not, they have their own unique open source drivers.

work.


At the moment there is no SATA command queuing under Linux, as you
can quickly discover by looking at the Serial ATA (SATA) Linux
software status report page here:

http://linux.yyz.us/sata/software-status.html

I was not saying that SATA queuing is worse. I was saying (or intended to)
that SCSI has command queuing under Linux while SATA does not

currently.

This is true if you're using some 20 buck dumb-chip host adapter, not if
you're using a 3ware, which has its own built in support for command
queuing that is transparent to the OS.

[...]

I am suggesting that WDs strategy is suspicious.

Why? They see SATA as the coming thing. Are you suggesting that Western
Digital is incapable of producing a SCSI drive?

I am suggesting that WD is trying to create a market beween ATA
and SCSI by claiming to be as good as SCSI with SATA prices. If
it sounds to good to be true, it probably is.

Haven't priced Raptors have you. Pricing is a lot closer to 10,000 RPM SCSI
drives of the same capacity than to IDE drives.

It is a driver and software questtion with newer interfaces as well.
I had numerous problems with SATA under Linux.

Using an intelligent RAID controller that has its own drivers or using a 20
buck chip?

[...]

Raptors have a jumper that selects startup in full power mode or startup
in standby, intended specifically to address this issue.

Good. And does the 3ware controllers support staggered starts?
Yes.

while
in SCSI that is usually a jumper-option. Typical is auto-start,
auto-start with a selectable delay and no auto-start. On SATA
you would have to to staggered power or the like to get the same
effect.

Just tell the drive to come out of standby whenever you are ready.

That should be sometheing the controller and the drive do. Id
the OS does it, it can fail in numerous interessting ways.

So let's see, it's badness if it's not there and it's badness if it's there?
Sorry, but you can't have it both ways. The capability is there. If your
pet OS doesn't support it and you need it then get a real OS.

Of course you are free to do that. But I have 4TB or RAIDed storage
under Linux, about half of which is SATA. And I did run in the problems
I describe here.

Using Raptors and intelligent RAID controllers, or using consumer products?

You are talking about the LL drivers. There is an SCSI abstraction
layer in the kernel as well as an SATA abstraction layer. The former
is stable, proven and full-featured. The latter is pretty basic at
the moment.

So use the SCSI abstraction layer. The OS sees the RAID, it does not see
individual drives in the RAID.

To quote the maintainer:

Basic Serial ATA support

The "ATA host state machine", the core of the entire driver, is
considered production-stable.

The error handling is very simple, but at this stage that is an
advantage. Error handling code anywhere is inevitably both complex and
sorely under-tested. libata error handling is intentionally
simple. Positives: Easy to review and verify correctness. Never data
corruption. Negatives: if an error occurs, libata will simply send the
error back the block layer. There are limited retries by the block
layer, depending on the type of error, but there is never a bus

reset.

We're not talking about "basic serial ATA support", we're talking about a
third-generation intelligent RAID controller that has its own drivers that
are unrelated to and do not use the SATA support built into the linux
kernel.

See above.

I saw above.

Also if specific drivers are needed for specific
hardware, they tend to be less reliable because the user-base is
smaller.

I see. So you would, given the choice between a purpose-made IBM server
with an IBM RAID controller and a consumer-crap machine made with generic
components, choose the consumer-crap because the consumer-crap drivers are
going to be "less reliable because the user-base is smaller"? Is that what
you are saying? Because you can get an intelligent SATA RAID controller
from the company that used to be IBM's RAID controller division.

I'm sorry, but the argument you are presenting would preclude the use of
_any_ intelligent RAID controller, SATA, SCSI, or otherwise.

Not in the set-up of the OP. You did read that, did you?

If he's proposing to soft-RAID using a 3ware host adapter then he's a damned
fool.

Well my colleague is His argument is that by using a dual opteron
fully dedicated to File Serving is more High-perf than a hardware raid,
even though we spend $ 1400 on the 3ware cards ( 2x12 oprts SATA ). I
think we can achieve the same perf and more reliability with just 12
SCSI drives using the onboard SCSI with the Linux SCSI abstraction
layer and software RAID. What do you guys think ?

 

[Arno Wagner]

Arno Wagner wrote:

In comp.sys.ibm.pc.hardware.storage J. Clarke
Arno Wagner wrote:

In comp.sys.ibm.pc.hardware.storage (e-mail address removed) wrote:
Hello all,
[...]
One thing you can be relatively sure of is that the SCSI controller
will work well with the mainboard. Also Linux has a long history of
supporting SCSI, while SATA support is new and still being worked on.

If he's using 3ware host adapters then "SATA support" is not an
issue--that's handled by the processor on the host adapter and all that
the Linux driver does is give commands to that processor.

Do you have any evidence to present that suggests that 3ware RAID
controllers have problems with any known mainboard?

No. I was mostly thinking of SMART support, which is not there
for SATA on Linux (unless you use the old IDE driver). Normal disk
access works fine in my experience.

Actually, that would be a function of the 3ware drivers. With a 3ware
host adapter you do not use the SATA drivers, you use drivers specific to
3ware, and the 3ware drivers _do_ support SMART under Linux.

And, does that work reliably and with the usual Linux tools,
i.e. smartctl? Would kind of surprise me, since libata does
not have smart support at all at the moment, since the ATA
passthru opcodes have only very recently be defined by the
SCSI T10 committee.

Yes, it does. That is _specifically_ addressed by the 3ware drivers.

You seem hung up on the idea that the 3ware host adapters use the generic
ATA drivers. They do not, they have their own unique open source drivers.

For you access scenario, SCSI will also be superior, since SCSI
has supported command queuing for a long time.

I'm sorry, but it doesn't follow that because SCSI has supported
command queuing for a long time that the performance will be superior.

Actually for small reads command queuing helps massively. The
"has been available for a long time" just means that it will work.

So where is the evidence that SCSI command queuing works better for small
reads than does SATA command queuing?

At the moment there is no SATA command queuing under Linux, as you
can quickly discover by looking at the Serial ATA (SATA) Linux
software status report page here:

http://linux.yyz.us/sata/software-status.html

I was not saying that SATA queuing is worse. I was saying (or intended to)
that SCSI has command queuing under Linux while SATA does not

currently.

This is true if you're using some 20 buck dumb-chip host adapter, not if
you're using a 3ware, which has its own built in support for command
queuing that is transparent to the OS.

[...]
I am suggesting that WDs strategy is suspicious.

Why? They see SATA as the coming thing. Are you suggesting that Western
Digital is incapable of producing a SCSI drive?

I am suggesting that WD is trying to create a market beween ATA
and SCSI by claiming to be as good as SCSI with SATA prices. If
it sounds to good to be true, it probably is.

Haven't priced Raptors have you. Pricing is a lot closer to 10,000 RPM SCSI
drives of the same capacity than to IDE drives.

It may be up
to SCSI standards, but I have doubts. SATA is far to new to compete
with SCSI on reliability

Reliability in a disk is primarily a function of the mechanical
components, not the interface.

It is a driver and software questtion with newer interfaces as well.
I had numerous problems with SATA under Linux.

Using an intelligent RAID controller that has its own drivers or using a 20
buck chip?

[...]
Raptors have a jumper that selects startup in full power mode or startup
in standby, intended specifically to address this issue.

Good. And does the 3ware controllers support staggered starts?
Yes.

while
in SCSI that is usually a jumper-option. Typical is auto-start,
auto-start with a selectable delay and no auto-start. On SATA
you would have to to staggered power or the like to get the same
effect.

Just tell the drive to come out of standby whenever you are ready.

That should be sometheing the controller and the drive do. Id
the OS does it, it can fail in numerous interessting ways.

So let's see, it's badness if it's not there and it's badness if it's there?
Sorry, but you can't have it both ways. The capability is there. If your
pet OS doesn't support it and you need it then get a real OS.

Not saying that SCSI is not the superior solution but the reasons
given seem to be ignoring the fact that a "smart" SATA RAID
controller is being compared with a "dumb" SCSI setup.

Not really. It is more a relatively new, supposedly smart technology
against a proven, older, reliable, knowen to be smart technology.
SCSI targets are really quite smart, while SATA targets are not too
bright. The 3ware controllers may help some, but I doubt they
can do that much.

You have made enough statements about SATA that are simply not true that
I wonder at the validity of your assessment.

Of course you are free to do that. But I have 4TB or RAIDed storage
under Linux, about half of which is SATA. And I did run in the problems
I describe here.

Using Raptors and intelligent RAID controllers, or using consumer products?

In addition the kernel knows how to talk to SCSI targets, while SATA is
still in flux. Data transfer on SATA works, but everything else is
still being worked on, like SMART support.

So let's see, you'd favor the use of a brand new LSI Logic SCSI RAID
controller over a brand new LSI Logic SATA RAID controller because "the
kernel knows how to talk to SCSI targets" despite the fact that both
devices use brand new drivers?

You are talking about the LL drivers. There is an SCSI abstraction
layer in the kernel as well as an SATA abstraction layer. The former
is stable, proven and full-featured. The latter is pretty basic at
the moment.

So use the SCSI abstraction layer. The OS sees the RAID, it does not see
individual drives in the RAID.

To quote the maintainer:

Basic Serial ATA support

The "ATA host state machine", the core of the entire driver, is
considered production-stable.

The error handling is very simple, but at this stage that is an
advantage. Error handling code anywhere is inevitably both complex and
sorely under-tested. libata error handling is intentionally
simple. Positives: Easy to review and verify correctness. Never data
corruption. Negatives: if an error occurs, libata will simply send the
error back the block layer. There are limited retries by the block
layer, depending on the type of error, but there is never a bus

reset.

We're not talking about "basic serial ATA support", we're talking about a
third-generation intelligent RAID controller that has its own drivers that
are unrelated to and do not use the SATA support built into the linux
kernel.

You're assuming that all contact with drives is via the SCSI or SATA
kernel drivers and not through a dedicated controller with drivers
specific to that controller.

See above.

I saw above.

Also if specific drivers are needed for specific
hardware, they tend to be less reliable because the user-base is
smaller.

I see. So you would, given the choice between a purpose-made IBM server
with an IBM RAID controller and a consumer-crap machine made with generic
components, choose the consumer-crap because the consumer-crap drivers are
going to be "less reliable because the user-base is smaller"? Is that what
you are saying? Because you can get an intelligent SATA RAID controller
from the company that used to be IBM's RAID controller division.

I'm sorry, but the argument you are presenting would preclude the use of
_any_ intelligent RAID controller, SATA, SCSI, or otherwise.

The RAID logic is pretty smart in both cases, since done by the
kernel, but when having this many disks you _will_ want to
poll defective lists/counts. drive temperature and the like
periodically to get early warnings.

With the 3ware host adapter, the RAID logic is ON THE BOARD, _not_ in the
kernel.

Not in the set-up of the OP. You did read that, did you?

If he's proposing to soft-RAID using a 3ware host adapter then he's a damned

fool.

Well my colleague is His argument is that by using a dual opteron
fully dedicated to File Serving is more High-perf than a hardware raid,
even though we spend $ 1400 on the 3ware cards ( 2x12 oprts SATA ). I
think we can achieve the same perf and more reliability with just 12
SCSI drives using the onboard SCSI with the Linux SCSI abstraction
layer and software RAID. What do you guys think ?

Well, I only have experience with an Adaptec 8 drive SATA
RAID card. It was significantly slower than connecting the 8
drives to two cheap Promise tx4 controllers and using software
RAID on a dual Athlon 2800+. It took forever when resyncing after
a drive had peen replaced. Also the software and the drivers
were bad. I hear that is not a problem with 3ware under Linux.

The nice thing about softweare RAID is that it gives you a lot of
flexibility, like partiton-based RAID and using leftover space
on some drives for other things. I don't know whether you need that.

As I said before I would go with the 12 SCSI drives and software
RAID. Also becaue I think that you will not get a PSU that
can support a dual Opteron and 24 drives reliably. Or at least
that such a PSU will be very expensive. My personal rule of thumb
is is 20W per drive plut the mainboard. That would give you somehing
like 480W (disks) + 250 W (Mainboard), i.e. a > 750W PSU. I am not
sure you how expensive that would be. On the other hand the 12 disk
set-up should be o.k. with a 550W or so PSU and you can get those
for reasonable money.

I have said dual Athlon with 16 disks run reliably with a 550W
Enermax PSU. Before it was on a 450W Fortron which died after
a power outage.

Arno

 

[J. Clarke]

Arno Wagner wrote:

In comp.sys.ibm.pc.hardware.storage J. Clarke
Arno Wagner wrote:

In comp.sys.ibm.pc.hardware.storage (e-mail address removed) wrote:
Hello all,
[...]
One thing you can be relatively sure of is that the SCSI controller
will work well with the mainboard. Also Linux has a long history of
supporting SCSI, while SATA support is new and still being worked on.

If he's using 3ware host adapters then "SATA support" is not an
issue--that's handled by the processor on the host adapter and all that
the Linux driver does is give commands to that processor.

Do you have any evidence to present that suggests that 3ware RAID
controllers have problems with any known mainboard?

No. I was mostly thinking of SMART support, which is not there
for SATA on Linux (unless you use the old IDE driver). Normal disk
access works fine in my experience.

Actually, that would be a function of the 3ware drivers. With a 3ware
host adapter you do not use the SATA drivers, you use drivers specific to
3ware, and the 3ware drivers _do_ support SMART under Linux.

And, does that work reliably and with the usual Linux tools,
i.e. smartctl? Would kind of surprise me, since libata does
not have smart support at all at the moment, since the ATA
passthru opcodes have only very recently be defined by the
SCSI T10 committee.

Yes, it does. That is _specifically_ addressed by the 3ware drivers.

You seem hung up on the idea that the 3ware host adapters use the generic
ATA drivers. They do not, they have their own unique open source drivers.

For you access scenario, SCSI will also be superior, since SCSI
has supported command queuing for a long time.

I'm sorry, but it doesn't follow that because SCSI has supported
command queuing for a long time that the performance will be superior.

Actually for small reads command queuing helps massively. The
"has been available for a long time" just means that it will work.

So where is the evidence that SCSI command queuing works better for small
reads than does SATA command queuing?

At the moment there is no SATA command queuing under Linux, as you
can quickly discover by looking at the Serial ATA (SATA) Linux
software status report page here:

http://linux.yyz.us/sata/software-status.html

I was not saying that SATA queuing is worse. I was saying (or intended to)
that SCSI has command queuing under Linux while SATA does not

currently.

This is true if you're using some 20 buck dumb-chip host adapter, not if
you're using a 3ware, which has its own built in support for command
queuing that is transparent to the OS.

[...]
I am suggesting that WDs strategy is suspicious.

Why? They see SATA as the coming thing. Are you suggesting that Western
Digital is incapable of producing a SCSI drive?

I am suggesting that WD is trying to create a market beween ATA
and SCSI by claiming to be as good as SCSI with SATA prices. If
it sounds to good to be true, it probably is.

Haven't priced Raptors have you. Pricing is a lot closer to 10,000 RPM SCSI
drives of the same capacity than to IDE drives.

It may be up
to SCSI standards, but I have doubts. SATA is far to new to compete
with SCSI on reliability

Reliability in a disk is primarily a function of the mechanical
components, not the interface.

It is a driver and software questtion with newer interfaces as well.
I had numerous problems with SATA under Linux.

Using an intelligent RAID controller that has its own drivers or using a 20
buck chip?

[...]
Raptors have a jumper that selects startup in full power mode or startup
in standby, intended specifically to address this issue.

Good. And does the 3ware controllers support staggered starts?
Yes.

while
in SCSI that is usually a jumper-option. Typical is auto-start,
auto-start with a selectable delay and no auto-start. On SATA
you would have to to staggered power or the like to get the same
effect.

Just tell the drive to come out of standby whenever you are ready.

That should be sometheing the controller and the drive do. Id
the OS does it, it can fail in numerous interessting ways.

So let's see, it's badness if it's not there and it's badness if it's there?
Sorry, but you can't have it both ways. The capability is there. If your
pet OS doesn't support it and you need it then get a real OS.

Not saying that SCSI is not the superior solution but the reasons
given seem to be ignoring the fact that a "smart" SATA RAID
controller is being compared with a "dumb" SCSI setup.

Not really. It is more a relatively new, supposedly smart technology
against a proven, older, reliable, knowen to be smart technology.
SCSI targets are really quite smart, while SATA targets are not too
bright. The 3ware controllers may help some, but I doubt they
can do that much.

You have made enough statements about SATA that are simply not true that
I wonder at the validity of your assessment.

Of course you are free to do that. But I have 4TB or RAIDed storage
under Linux, about half of which is SATA. And I did run in the problems
I describe here.

Using Raptors and intelligent RAID controllers, or using consumer products?

In addition the kernel knows how to talk to SCSI targets, while SATA is
still in flux. Data transfer on SATA works, but everything else is
still being worked on, like SMART support.

So let's see, you'd favor the use of a brand new LSI Logic SCSI RAID
controller over a brand new LSI Logic SATA RAID controller because "the
kernel knows how to talk to SCSI targets" despite the fact that both
devices use brand new drivers?

You are talking about the LL drivers. There is an SCSI abstraction
layer in the kernel as well as an SATA abstraction layer. The former
is stable, proven and full-featured. The latter is pretty basic at
the moment.

So use the SCSI abstraction layer. The OS sees the RAID, it does not see
individual drives in the RAID.

To quote the maintainer:

Basic Serial ATA support

The "ATA host state machine", the core of the entire driver, is
considered production-stable.

The error handling is very simple, but at this stage that is an
advantage. Error handling code anywhere is inevitably both complex and
sorely under-tested. libata error handling is intentionally
simple. Positives: Easy to review and verify correctness. Never data
corruption. Negatives: if an error occurs, libata will simply send the
error back the block layer. There are limited retries by the block
layer, depending on the type of error, but there is never a bus

reset.

We're not talking about "basic serial ATA support", we're talking about a
third-generation intelligent RAID controller that has its own drivers that
are unrelated to and do not use the SATA support built into the linux
kernel.

You're assuming that all contact with drives is via the SCSI or SATA
kernel drivers and not through a dedicated controller with drivers
specific to that controller.

See above.

I saw above.

Also if specific drivers are needed for specific
hardware, they tend to be less reliable because the user-base is
smaller.

I see. So you would, given the choice between a purpose-made IBM server
with an IBM RAID controller and a consumer-crap machine made with generic
components, choose the consumer-crap because the consumer-crap drivers are
going to be "less reliable because the user-base is smaller"? Is that what
you are saying? Because you can get an intelligent SATA RAID controller
from the company that used to be IBM's RAID controller division.

I'm sorry, but the argument you are presenting would preclude the use of
_any_ intelligent RAID controller, SATA, SCSI, or otherwise.

The RAID logic is pretty smart in both cases, since done by the
kernel, but when having this many disks you _will_ want to
poll defective lists/counts. drive temperature and the like
periodically to get early warnings.

With the 3ware host adapter, the RAID logic is ON THE BOARD, _not_ in the
kernel.

Not in the set-up of the OP. You did read that, did you?

If he's proposing to soft-RAID using a 3ware host adapter then he's a damned
fool.

Well my colleague is His argument is that by using a dual opteron
fully dedicated to File Serving is more High-perf than a hardware raid,
even though we spend $ 1400 on the 3ware cards ( 2x12 oprts SATA ).

He's fighting the system that way, not working with it. If hardware RAID
was so lousy then IBM wouldn't be putting in their midrange and high end
servers.

I
think we can achieve the same perf and more reliability with just 12
SCSI drives using the onboard SCSI with the Linux SCSI abstraction
layer and software RAID. What do you guys think ?

I think that if he's determined to do soft RAID the outcome is going to be a
good deal more satisfactory if he's not fighting a device that wasn't
designed to support that mode of operation.

But he should be asked to show in detail, with test results and
caclulations, why his proposed solution is superior to using the 3ware
board in the manner in which it was designed to be used.

If it's a given that software RAID will be used then I'd go for the SCSI
solution simply because there's not a "dumb" SATA RAID controller with
enough ports to do what you want to do.

 

[Arno Wagner]

(e-mail address removed) wrote: [...]

Well my colleague is His argument is that by using a dual
opteron fully dedicated to File Serving is more High-perf than a
hardware raid, even though we spend $ 1400 on the 3ware cards (
2x12 oprts SATA ).

He's fighting the system that way, not working with it. If hardware RAID
was so lousy then IBM wouldn't be putting in their midrange and high end
servers.

I think we can achieve the same perf and more reliability with just
12 SCSI drives using the onboard SCSI with the Linux SCSI
abstraction layer and software RAID. What do you guys think ?

I think that if he's determined to do soft RAID the outcome is going to be a
good deal more satisfactory if he's not fighting a device that wasn't
designed to support that mode of operation.

I fully agree to that.

[...]

If it's a given that software RAID will be used then I'd go for the SCSI
solution simply because there's not a "dumb" SATA RAID controller with
enough ports to do what you want to do.

That could be problematic, yes. I think the largest you get is
8 ports (Promise), but that requires PCI-X. If you use 4 port PCI
controllers you would have to use 6 for 24 disks. I would not
trust such a configuration, if it is possible at all.

Arno

 

[J. Clarke]

(e-mail address removed) wrote:

J. Clarke wrote:
Arno Wagner wrote:

[...]

If he's proposing to soft-RAID using a 3ware host adapter then
he's a damned fool.
Well my colleague is His argument is that by using a dual
opteron fully dedicated to File Serving is more High-perf than a
hardware raid, even though we spend $ 1400 on the 3ware cards (
2x12 oprts SATA ).

He's fighting the system that way, not working with it. If hardware RAID
was so lousy then IBM wouldn't be putting in their midrange and high end
servers.

I think we can achieve the same perf and more reliability with just
12 SCSI drives using the onboard SCSI with the Linux SCSI
abstraction layer and software RAID. What do you guys think ?

I think that if he's determined to do soft RAID the outcome is going to
be a good deal more satisfactory if he's not fighting a device that
wasn't designed to support that mode of operation.

I fully agree to that.

[...]

If it's a given that software RAID will be used then I'd go for the SCSI
solution simply because there's not a "dumb" SATA RAID controller with
enough ports to do what you want to do.

That could be problematic, yes. I think the largest you get is
8 ports (Promise), but that requires PCI-X.

Interesting. I had not seen that one. But is it truly dumb? If we're
thinking of the same model it belongs to the SX series which normally has
at least some onboard intelligence.

As for PCI-X, that's actually another good point. With 12 drives there may
be times that the PCI bus becomes a bottleneck.

 

[Arno Wagner]

Arno Wagner wrote:

(e-mail address removed) wrote:

J. Clarke wrote:
Arno Wagner wrote: [...]

If he's proposing to soft-RAID using a 3ware host adapter then
he's a damned fool.
Well my colleague is His argument is that by using a dual
opteron fully dedicated to File Serving is more High-perf than a
hardware raid, even though we spend $ 1400 on the 3ware cards (
2x12 oprts SATA ).
He's fighting the system that way, not working with it. If hardware RAID
was so lousy then IBM wouldn't be putting in their midrange and high end
servers.

I think we can achieve the same perf and more reliability with just
12 SCSI drives using the onboard SCSI with the Linux SCSI
abstraction layer and software RAID. What do you guys think ?

I think that if he's determined to do soft RAID the outcome is going to
be a good deal more satisfactory if he's not fighting a device that
wasn't designed to support that mode of operation.

I fully agree to that.

[...]

If it's a given that software RAID will be used then I'd go for the SCSI
solution simply because there's not a "dumb" SATA RAID controller with
enough ports to do what you want to do.

That could be problematic, yes. I think the largest you get is
8 ports (Promise), but that requires PCI-X.

Interesting. I had not seen that one. But is it truly dumb? If we're
thinking of the same model it belongs to the SX series which normally has
at least some onboard intelligence.

Yes, I am thinking of the SX8. It is one of these RAID-accelerators
where there is an XOR engine and the like in the card, but it cannot
do the full job without assistance from the CPU. It is somewhere halfway
between a dumb controller and true hardware RAID, also in price.

As for PCI-X, that's actually another good point. With 12 drives there may
be times that the PCI bus becomes a bottleneck.

Definitely.

Arno

 

[Folkert Rienstra]

Arno Wagner wrote:

(e-mail address removed) wrote:


J. Clarke wrote:
Arno Wagner wrote:
[...]

If he's proposing to soft-RAID using a 3ware host adapter then
he's a damned fool.
Well my colleague is His argument is that by using a dual
opteron fully dedicated to File Serving is more High-perf than a
hardware raid, even though we spend $ 1400 on the 3ware cards (
2x12 oprts SATA ).
He's fighting the system that way, not working with it. If hardware RAID
was so lousy then IBM wouldn't be putting in their midrange and high end
servers.

I think we can achieve the same perf and more reliability with just
12 SCSI drives using the onboard SCSI with the Linux SCSI
abstraction layer and software RAID. What do you guys think ?

I think that if he's determined to do soft RAID the outcome is going to
be a good deal more satisfactory if he's not fighting a device that
wasn't designed to support that mode of operation.

I fully agree to that.

[...]
If it's a given that software RAID will be used then I'd go for the SCSI
solution simply because there's not a "dumb" SATA RAID controller with
enough ports to do what you want to do.

That could be problematic, yes. I think the largest you get is
8 ports (Promise), but that requires PCI-X.

Interesting. I had not seen that one. But is it truly dumb? If we're
thinking of the same model it belongs to the SX series which normally has
at least some onboard intelligence.

Yes, I am thinking of the SX8. It is one of these RAID-accelerators
where there is an XOR engine and the like in the card, but it cannot
do the full job without assistance from the CPU. It is somewhere halfway
between a dumb controller and true hardware RAID, also in price.

As for PCI-X, that's actually another good point.

With 12 drives there may be times that the PCI bus becomes a bottleneck.

If it's not the SCSI bus first.
And he will be using server boards so PCI is not a problem.

Definitely.

With 64 kB files read randomly, spread over 6 strips or even 12?
Yeah, right.

Maybe the queue mechanism can combine a few into sequential reads but
even then you are still looking at very small IO with huge latency overhead.
(64 kB transfers in 1 ms vs a 6 ms access time and that's for a single drive)
So you are using 1/7th the available bandwidth of the resultant drive.

I think even standard PCI will cope.
And 120 MB/s is still stinking fast for the odd big file.

 

[Folkert Rienstra]

Arno Wagner wrote:
Arno Wagner wrote:
Arno Wagner wrote:

In comp.sys.ibm.pc.hardware.storage (e-mail address removed) wrote:
Hello all,
[...]

[onehelluvabigsnip]

Well my colleague is His argument is that by using a dual opteron
fully dedicated to File Serving is more High-perf than a hardware raid,
even though we spend $ 1400 on the 3ware cards ( 2x12 oprts SATA ).

He's fighting the system that way, not working with it. If hardware RAID
was so lousy then IBM wouldn't be putting in their midrange and high end
servers.

I think we can achieve the same perf and more reliability with just 12
SCSI drives using the onboard SCSI with the Linux SCSI abstraction
layer and software RAID. What do you guys think ?

I think that if he's determined to do soft RAID the outcome is going to
be a good deal more satisfactory if he's not fighting a device that wasn't
designed to support that mode of operation.

But he should be asked to show in detail, with test results and
caclulations, why his proposed solution is superior to using the
3ware board in the manner in which it was designed to be used.

If it's a given that software RAID will be used then I'd go for the SCSI
solution simply because there's not a "dumb" SATA RAID controller with
enough ports to do what you want to do.

That also applies to SCSI.
In RAID you are not supposed to be using more than four drives per channel
so with 12 drives you are looking at 3 channels.

But since this is small record IO you may be able to sqeeuze them onto a
2-channel card though.