Commoditization of 4-way

[Yousuf Khan]

I know there is a little bit of skepticism (well actually a lot of it) about
x86's potential in the 4-way market. The high-end server market has feasted
on the high prices due to the lack of competition from any commodity systems
at this level. There's been talk for years that Intel is finally going to
make something real for the 4-way systems. It's hasn't happened yet. But it
may be finally starting now. Opterons are the real deal now.

The following article expects that AMD will take the price of 4-way systems
down from an average of $10,000 to $5000 by next year. And customers seem to
be responding. Some of the smaller OEMs are seeing a demand shift towards
AMD systems, ranging from 40% to 100% AMD at some server makers. And they're
all getting ready to pound Dell, which won't be able to match these systems.
But really the more long term effect will not be the one against Dell, but
the one against the big-iron boys who will see one of their most profitable
segments disappear into the sea of commoditization.

http://www.crn.com/sections/coverstory/coverstory.asp?ArticleID=50198

Yousuf Khan

 

[Thomas Womack]

The following article expects that AMD will take the price of 4-way systems
down from an average of $10,000 to $5000 by next year.

The price of a 4-way system is not much over $5000 *now*, isn't it?
$1700 for the Tyan board, $2800 for four Opteron 844's, $250 for a chassis,
and, well, you get one SATA disc and only one of the Opterons gets any memory
with the change from $5000.

If I had anything resembling a business plan, I'd be tempted to get something
like the system above, but if I'm spending UKP3000 I think a second-hand
Smartcar would be more fun.

I wonder if AMD will drop the Opteron 840 price enormously at some stage;
4 x 1400MHz with separate memory to each processor isn't bad for, say, a
shell-account machine.

Tom

 

[Stephen Sprunk]

I know there is a little bit of skepticism (well actually a lot of it) about
x86's potential in the 4-way market. The high-end server market has feasted
on the high prices due to the lack of competition from any commodity systems
at this level. There's been talk for years that Intel is finally going to
make something real for the 4-way systems. It's hasn't happened yet. But it
may be finally starting now. Opterons are the real deal now.

Intel's antiquated FSB design means 4-way (and up) systems require
specialized chipsets to avoid bottlenecks, and the low volume of these
chipsets means higher prices which leads back to low volume.

Opteron's scalability using glueless HT for SMP is turning the 4-way market
on its head, and probably the 8-way market as well once people figure out
how to cram that many CPUs and memory slots into a starndard-sized system.

S

 

[Never anonymous Bud]

Fresh from an Iraqi prisoner interrogation Thomas Womack

The price of a 4-way system is not much over $5000 *now*, isn't it?
$1700 for the Tyan board,

Where did you find a price? I can't locate ANYONE that has that MB available.

$2800 for four Opteron 844's,

Not even listed on Pricewatch now, 842s and 846s are though.

and, well, you get one SATA disc and only one of the Opterons gets any memory
with the change from $5000.

Hmmm, less than $200 to cover memory and 'other things'.
I think you're going over $6K by a decent margin for working system.






To reply by email, remove the XYZ.

Lumber Cartel (tinlc) #2063. Spam this account at your own risk.

This sig censored by the Office of Home and Land Insecurity....

 

[nobody]

The price of a 4-way system is not much over $5000 *now*, isn't it?
$1700 for the Tyan board, $2800 for four Opteron 844's, $250 for a chassis,
and, well, you get one SATA disc and only one of the Opterons gets any memory
with the change from $5000.

If I had anything resembling a business plan, I'd be tempted to get something
like the system above, but if I'm spending UKP3000 I think a second-hand
Smartcar would be more fun.

I wonder if AMD will drop the Opteron 840 price enormously at some stage;
4 x 1400MHz with separate memory to each processor isn't bad for, say, a
shell-account machine.

Tom

IMHO, it borders a crime not to outfit all 4 Opterons with the memory
in such a highest end system. And then, I have not seen a quad box
that didn't have a nice SCSI RAID (at least 3 disks in RAID 5 config).
Only the RAM (4 GB in 8 516 MB sticks of PC400 DDR ECC Registered) is
near $1000 ($122 for each - the lowest on PW). As for the price for
SCSI RAID solution, the sky is the limit.
Yet, the price point may be reached if we believe Mr. Ruiz on dual
core chips getting ready by next year. In this case, every dual board
will be capable to effectively host a quad system (and it will be true
SMP, not just hyperthreading!). Dual boards are already close to
$200, and probably will only get cheaper by next year.

 

[Rob Stow]

Fresh from an Iraqi prisoner interrogation Thomas Womack



Where did you find a price? I can't locate ANYONE that has that MB available.

$1545 at Lynn Computer, up from $1495 a couple of weeks ago.
They are taking orders only - no stock yet.
http://www.lynncomp.com

Not even listed on Pricewatch now, 842s and 846s are though.

$750 at Lynn, down from $9xx.

Hmmm, less than $200 to cover memory and 'other things'.
I think you're going over $6K by a decent margin for working system.

Even a $6K, that is half the price of a basic 4-way Opty 840
system from the vendors that actually have something to sell.
Now if only I could get that percentage saving when building
my own desktop ...

 

[Never anonymous Bud]

Even a $6K, that is half the price of a basic 4-way Opty 840
system from the vendors that actually have something to sell.

Well, as you already said, the MB you mentioned isn't even
available to end-users.

Add in that I said MORE than $6K, plus the time and effort it
takes to put such a beast together and get it running, and
$10K isn't that much of a leap.

When I can get a 4-way Opty MB for about $500, and the
CPUs for about $350 apiece, I'll consider the price almost
in my range.





To reply by email, remove the XYZ.

Lumber Cartel (tinlc) #2063. Spam this account at your own risk.

This sig censored by the Office of Home and Land Insecurity....

 

[Thomas Womack]

IMHO, it borders a crime not to outfit all 4 Opterons with the memory
in such a highest end system.

Well, of course; a more sensible system won't leave much change from
$7000, since you probably do want 4GB of memory and 15krpm SCSI
RAID. It's still cheaper than manufacturer-guaranteed second-hand
cars, and deeply cheaper than any new car in the UK. It's probably
too *loud* to have as a desktop, even if I could think of desktop
tasks for which such a behemoth would be sane.

I'm not sure I'd call a system with two dual-core CPUs a quad system,
though I'm not quite sure where that prejudice comes from; I suppose
that part of the issue of a quad system is the enormous motherboard
required physically to fit four sockets, four cooling systems, four
sets of memory ... on memory-intensive tasks I think I'd rather have
more memory subsystems than more cores, dual-core Opterons will be no
less memory-starved than 800MHz FSB Noconas.

Tom

 

[myren, lord]

the big steps towards 4 way will be the incoming motherboard
technologies. the processors have only a marignal role here.

fb-ram and pci-express, together, represent the greatest enabler of 4
way or better technologies. fb-ram will enable far less pins with far
less demanding specifications for those huge banks of ram hanging off
your processors: and advantage only compounded by a direct to cpu
connection. pci-express by its serial nature allows for further
simplification of a overgrowing pci bus, as well as, iirc, longer signal
paths.

although opteron has eliminated the massively expensive switching
responsibility of the motherboard, it poses new challenges of feeding
four processors ram and i/o. the current technologies use gi-normous
amounts of channels to accomplish this. when fb-ram and pci-express
come out, the number of paths and the tolerances for these paths should
be improved by an order of magntiude. it is only when everyone and
their mother can cook up a new motherboard design from cheap parts that
we will see the rediuculous 4-way margins slowly be eatten away. you
can bet the current big boys will be fighting against this
commoditization as hard as they can.

opteron is amazing in that it has its switching technology built in via
ht. when it comes out natively supporting fb-ram in another year and a
half, then we will see a dawn of the new four way systems.

 

[Yousuf Khan]

fb-ram and pci-express, together, represent the greatest enabler of 4
way or better technologies. fb-ram will enable far less pins with far
less demanding specifications for those huge banks of ram hanging off
your processors: and advantage only compounded by a direct to cpu
connection. pci-express by its serial nature allows for further
simplification of a overgrowing pci bus, as well as, iirc, longer
signal paths.

FBRAM has yet to prove itself (or even come out).

Hypertransport allows one to do something even more performance oriented for
the overgrowing PCI bus than PCI-E. It allows you to hang multiple PCI buses
directly off of separate processors, load balancing the load and
partitioning the data off in a way.

Yousuf Khan

 

[=?ISO-8859-1?Q?Jan_Vorbr=FCggen?=]

opteron is amazing in that it has its switching technology built in via ht.

Yawn - the 21364 had that how many years ago? Oh yes, and all the transputers
had a programmeable memory interface on-chip. Glue logic? We need no stinkin'
glue logic!

Jan

 

[Stephen Sprunk]

ht.

Yawn - the 21364 had that how many years ago? Oh yes, and all the transputers
had a programmeable memory interface on-chip. Glue logic? We need no stinkin'
glue logic!

True, but AFAIK the K8 was the first chip with a non-negligible market share
to
have such features.

Nothing is new under the sun, not even when the Alpha did whatever is
currently being discussed. Making innovation _profitable_ is arguably more
important than bringing it to market first and then going under, because the
former leads to permanent change and the latter is just fodder for
comp.arch.

S

 

[myren, lord]

FBRAM has yet to prove itself (or even come out).

This is true. Whether its FBRAM or the next technology is in question.
What is certain is that RAM presents a significant design challenge
for motherboard builders in terms of signal integrity and simple volume
of routing.

Hypertransport allows one to do something even more performance oriented for
the overgrowing PCI bus than PCI-E. It allows you to hang multiple PCI buses
directly off of separate processors, load balancing the load and
partitioning the data off in a way.

I have a sneaking suspicion this is going to be how we'll start seeing
multiple x16 pci-e links for multi-display systems. I'm scared we
havent seen anyone advertising multiple graphics links.

Yes, the I/O bandwidth to feed these processors is essential, but a
fibre array does the job well enough across pci 66/64. the ability to
pipe network data every which way is extremely fun, but non essential.

Although this is certainly useful and advantageous, the fact of the
matter is that having multiple pci busses will not /initially/ be a
major factor in making 4 way systems cheaper, in commoditizing the
market. The main limitation as it stands is $3000 motherboards. When
someone kicks out an order of magnitude cheaper because they can and
because its simple, then we'll come back to the virtues of HT and the
multiple pci busses it can provide.

The problem is that until this event happens ($300 quad mobo), system
builders can make boards as cheap as they please or utilizing all the
excess amounts of HT glory they want, but they'll always charge $2000+
because sales would not differ signficantly from $1000. The market is
defined as high end servers, so presumably the customer is going to be
paying through the nose for 15k scsi raid and all those other goodies,
$1000 on a mobo here or there is nothing. When $300 quad comes out, the
tune changes a lot. Quad is suddenly considerable for web clusters.

I'd wager to define commoditization as the threshold where mainstream
enthusiasts (a contradiction in terms if i ever heard one) consider
building 4 way systems.

Myren

 

[Mitch Alsup]

Yawn - the 21364 had that how many years ago?
Jan

And how many did they sell?

 

[Mitch Alsup]

fb-ram and pci-express, together, represent the greatest enabler of 4
way or better technologies. fb-ram will enable far less pins with far
less demanding specifications for those huge banks of ram hanging off
your processors: and advantage only compounded by a direct to cpu
connection. pci-express by its serial nature allows for further
simplification of a overgrowing pci bus, as well as, iirc, longer signal
paths.

FB-RAM pins operate in the 2.5GHz to 6GHz range. Anybody who thinks
a simple protocal makes operating at these pin speed easy should pass
the pipe.

Second, the push for FBDIMM is to enable even larger memory systems
not to save a few pins. With FBDIMM, an opteron can go from 8 DDR-II
DIMMs to 32 FBDIMMs using a similar number of pins. This allows even
larger memories to be built and capture ever more of the footprint
of the database applications.

PCI-express brings little to the party that HT does not already
bring except that Intel is pushing it rather than AMD.

 

[Del Cecchi]

snip

PCI-express brings little to the party that HT does not already
bring except that Intel is pushing it rather than AMD.

PCI-Express doesn't crash and burn on an error.

del cecchi

 

[Stephen Fuld]

snip

PCI-express brings little to the party that HT does not already
bring except that Intel is pushing it rather than AMD.

And the fact that it preserves the investment in software of lots and lots
of PCI drivers that exist today. Preserving that investment was one of the
reasons that the Intel desktop people decided they wanted to do PCI Express
rather than sign on to IB.

 

[myren, lord]

i thought intel was one of the infiniband people too. infiniband and
pci-e arent mutually exlcusive technologies by any means.

parents said amd wasnt a fan of pci-e. are they designing any competition?

 

[Andi Kleen]

And the fact that it preserves the investment in software of lots and lots
of PCI drivers that exist today. Preserving that investment was one of the
reasons that the Intel desktop people decided they wanted to do PCI Express
rather than sign on to IB.

Seen from the software (not firmware) side HT is basically completely PCI
compatible. I don't know of any visible differences. The north
bridge on a Opteron system is implemented in the CPU and e.g. all the
PCI config accesses originate from the north bridge. But the request
has to travel over an HT link before it can actually talk to an real
PCI bridge. This works completely transparent.

For the OS it looks like any other x86 compatible PCI PC. Of course
on basically every other modern x86 "PCI" chipset there is some kind
of non PCI link involved in such an operation; an HT system is not very
different e.g. from a typical Intel based system.

The only software that really knows about HT is the firmware that has
to set up the routing. But even on an "pure PCI" system this is
completely chipset dependent black magic and not standardized at all.

As far as I can see the main difference in practice right now is that
there is a PCI-E connector and there isn't one for HT. And the error
handling issues Del noted (hopefully be fixed with HT 2.x)
Ok and HT hotplug would be nice too.

-Andi

 

[Greg Lindahl]

As far as I can see the main difference in practice right now is that
there is a PCI-E connector and there isn't one for HT.

But there is a candidate for one -- if anyone is interested, I can put
you in touch with the people involved.

-- greg