Another AMD supercomputer, 13,000 quad-core

[.Some.Guy.]

The new computer, to be built by Sun Microsystems Inc. using 13,000
microprocessors made by Advanced Micro Devices Inc., will be more
powerful than any supercomputer currently in operation, UT says.

It will have a peak computing power of more than 400 trillion
operations per second, which makes it more than 40 percent faster than
the current supercomputing speed champ, Blue Gene/L, developed by IBM
Corp. for the Lawrence Livermore National Laboratory in California.

http://www.statesman.com/business/content/business/stories/technology/09/29/29computer.html

 

[nobody]

The new computer, to be built by Sun Microsystems Inc. using 13,000
microprocessors made by Advanced Micro Devices Inc., will be more
powerful than any supercomputer currently in operation, UT says.

It will have a peak computing power of more than 400 trillion
operations per second, which makes it more than 40 percent faster than
the current supercomputing speed champ, Blue Gene/L, developed by IBM
Corp. for the Lawrence Livermore National Laboratory in California.

http://www.statesman.com/business/content/business/stories/technology/09/29/29computer.html

....which will result in a nice uptick in overall Opteron sales. Hmm,
what kind of frame rate this thing will reach in [your favorite game
here]? ;-)

NNN

 

[Yousuf Khan]

The new computer, to be built by Sun Microsystems Inc. using 13,000
microprocessors made by Advanced Micro Devices Inc., will be more
powerful than any supercomputer currently in operation, UT says.

It will have a peak computing power of more than 400 trillion
operations per second, which makes it more than 40 percent faster than
the current supercomputing speed champ, Blue Gene/L, developed by IBM
Corp. for the Lawrence Livermore National Laboratory in California.

http://www.statesman.com/business/content/business/stories/technology/09/29/29computer.html

It seems so simple to put together a supercomputer these days. Just put
tons and tons of processors on. 10 000 here, 12 000 there, 13 000 here,
15 000 there, etc.

Yousuf Khan

 

[Ed]

http://www.statesman.com/business/content/business/stories/technology/09/29/29computer.html

Sources tell me that the system will be using Opteron powered Sun blades
using Thumper disk systems. The blades are quad socket and will start
off dual core, but can migrate to quad core.

amdzone.com

 

[efffemm]

It seems so simple to put together a supercomputer these days. Just put
tons and tons of processors on. 10 000 here, 12 000 there, 13 000 here,
15 000 there, etc.

Yousuf Khan

once you get over about 1000 CPUs,
you also keep adding air conditioners to keep the build cool.....

 

[GIRunit]

ditto. the cooling on that joker fails and it would go up like the 4th
of july!

 

[Robert Myers]

It seems so simple to put together a supercomputer these days. Just put
tons and tons of processors on. 10 000 here, 12 000 there, 13 000 here,
15 000 there, etc.

Just so, and it has been so for a long time. "Supercomputers" are now
more about real estate than about technology or cleverness. Buy
yourself a big warehouse, lots of cable, and some switches. You will
probaby also need to speak to your local electrical utility about your
needs.

The action isn't in the processors anymore. It's partly in the
interconnect, which is why Opteron is so attractive for building
supercomputers that can actually do something. It's also in power
consumption, which is why x86 clones (nor Itanium) may not be the
future of supercomputers.

RM

 

[George Macdonald]

Just so, and it has been so for a long time. "Supercomputers" are now
more about real estate than about technology or cleverness. Buy
yourself a big warehouse, lots of cable, and some switches. You will
probaby also need to speak to your local electrical utility about your
needs.

The action isn't in the processors anymore. It's partly in the
interconnect, which is why Opteron is so attractive for building
supercomputers that can actually do something. It's also in power
consumption, which is why x86 clones (nor Itanium) may not be the
future of supercomputers.

Hey Robert - haven't heard from you here in a while.

Trouble is, nobody, not even the U.S. taxpayer, can afford a new custom
architecture and CPU design... which is going to be obsolete the day it
first starts "crunching". I believe that even the Japanese have finally
come to the same conclusion. Maybe the Chinese, with their uncanny ability
to hide epxense, will bite??

 

[Robert Myers]

Hey Robert - haven't heard from you here in a while.

My surplus U2 didn't come equipped with an internet connection.

Trouble is, nobody, not even the U.S. taxpayer, can afford a new custom
architecture and CPU design... which is going to be obsolete the day it
first starts "crunching". I believe that even the Japanese have finally
come to the same conclusion. Maybe the Chinese, with their uncanny ability
to hide epxense, will bite??

But specialized coprocessors are not out of reach. Folding at Home is
doing great things with GPU's.

Of course, saying "oh, we'll use coprocessors" is just another version
of the interconnect problem, but it may be the only practical solution
to the power consumption problem and the only solution at all to the
class of problems that appears to concern you: problems with a long
critical path.

Robert.

 

[Del Cecchi]

George Macdonald wrote:
snip

Trouble is, nobody, not even the U.S. taxpayer, can afford a new custom
architecture and CPU design... which is going to be obsolete the day it
first starts "crunching". I believe that even the Japanese have finally
come to the same conclusion. Maybe the Chinese, with their uncanny ability
to hide epxense, will bite??

Sure, all kinds of people can afford such a design. What's it cost?
Couple of Billion, or half of a fab? The Government (US Government that
is) takes in about 2500 Gigabucks per year and spends somewhat more.
The feds could afford an new state of the art architecture and set of
chips and boxes with the chips without breaking a sweat. It would be
cheap compared to developing an manufacturing a new fighter plane, or a
new destroyer.

On the other hand, even the government would have to have a semi valid
reason for doing so, and it isn't clear that there is one. Although
Cray and their vector processors are an interesting data point.

 

[Robert Myers]

On the other hand, even the government would have to have a semi valid
reason for doing so, and it isn't clear that there is one. Although
Cray and their vector processors are an interesting data point.

When the Cray-1 first came out, people talked overenthusiastically
about numerical wind tunnels and stunning computer animations and
graphics. The company I worked for closed a lab without a second
thought on the not entirely-incorrect theory that fluid mechanical
experiments were mostly a thing of the past. Funny thing is, most of
those predictions have come true, just not in the way or on the time
scale anyone would have expected at the time.

A similar development in biotechnology would have all the floor space
around MIT filled with computational scientists instead of wet
chemists. We're no further away from that than we were from a
numerical wind tunnel when people spoke glibly of such a thing, but
people talk much more cautiously now than they did then.

It's about vision and pizazz, not transistors, and the annual
supercomputer linpack ho-hum hasn't helped.

Robert.

 

[Del Cecchi]

When the Cray-1 first came out, people talked overenthusiastically
about numerical wind tunnels and stunning computer animations and
graphics. The company I worked for closed a lab without a second
thought on the not entirely-incorrect theory that fluid mechanical
experiments were mostly a thing of the past. Funny thing is, most of
those predictions have come true, just not in the way or on the time
scale anyone would have expected at the time.

A similar development in biotechnology would have all the floor space
around MIT filled with computational scientists instead of wet
chemists. We're no further away from that than we were from a
numerical wind tunnel when people spoke glibly of such a thing, but
people talk much more cautiously now than they did then.

It's about vision and pizazz, not transistors, and the annual
supercomputer linpack ho-hum hasn't helped.

Robert.

Actually I was referring to the thought that Cray, especially the vector
line, seems to be supported by the NSA. Do they sell those vector
machines to anyone besides the Government?

 

[Robert Myers]

Actually I was referring to the thought that Cray, especially the vector
line, seems to be supported by the NSA. Do they sell those vector
machines to anyone besides the Government?

I believe that Boeing, at least, is still a customer.

Robert.

 

[David Kanter]

I believe that Boeing, at least, is still a customer.

I suspect that there is a very small number of customers for Cray, my
understanding is that they are largely on the government dole and have
no chance of being a commercial viable endeavor.

DK

 

[George Macdonald]

I suspect that there is a very small number of customers for Cray, my
understanding is that they are largely on the government dole and have
no chance of being a commercial viable endeavor.

First, is this because you suspect that Cray as a company is incompetent or
misguided, technically?... or that HPC is just an uneconomic market? It's
certainly a market where it is difficult to decide a street price for
product.

Second, which "Cray" are you talking about? The high-capacity COTS cluster
Cray or the high-capabilty (real supercomputer?) one?

Quite honestly, I don't have a good grasp of the answers here and even the
"experts" in the HPC filed seem confused and at odds. It seems, though,
that one important issue involved is: was the anti-dumping embargo on
NEC/HNSX correct or just protectionism to cover up mistakes by the US govt
in procurement policy and/or by Cray in technical direction? That's
somewhat history now but the dilemma remains relevant I think.

I don't see much evidence of the large corps clamoring for high-end
high-capability computers, though there are niches where they might bite,
if the price is not astronomical. Seems like this pretty much leaves it in
the hands of NSF and other govt. sponsors to initiate the financial backing
- IOW enterprise needs will bask in the the err, fall-out.... maybe!

 

[David Kanter]

First, is this because you suspect that Cray as a company is incompetent or
misguided, technically?

I haven't seen much evidence to support incompetence. However, it is
rather apparent that Cray will not be successful financially b/c of a
combination of economic and technical reasons. OTOH, some of those
technical reasons keep Cray alive, albeit in a dimished and marginal
state.

... or that HPC is just an uneconomic market? It's
certainly a market where it is difficult to decide a street price for
product.

HPC is a difficult, difficult market with a mercurial customer base and
low volumes. In other words, it's a commercial disaster.

Second, which "Cray" are you talking about? The high-capacity COTS cluster
Cray or the high-capabilty (real supercomputer?) one?

I'm talking about the company Cray. They are not an economically
viable enterprise, through the combination of their products, or any
individual product line.

Their COTS cluster is not very interesting, since it's a very tough
market with competition from Dell (and that competition will look even
more attractive if they ever release 2S opteron systems) and HP that is
hard to beat. Cray has the same problem as SGI, except that their
value added (the real vector machines) are vastly less popular than the
Altix. I also don't think Cray does storage, but I may be wrong.

Quite honestly, I don't have a good grasp of the answers here and even the
"experts" in the HPC filed seem confused and at odds. It seems, though,
that one important issue involved is: was the anti-dumping embargo on
NEC/HNSX correct or just protectionism to cover up mistakes by the US govt
in procurement policy and/or by Cray in technical direction? That's
somewhat history now but the dilemma remains relevant I think.

When was this embargo?

I don't see much evidence of the large corps clamoring for high-end
high-capability computers,

I actually think the demand for such machines has increased over time.
The issue is whether demand increases faster than Moore's law increases
integration. As we all know, it only takes 4 sockets to build a 8P
machine, and in a short period of time that will be 16S. The larger
machines are being eaten from below, and the one factor that they can
stave them off with is bandwidth to I/O, memory, etc.

though there are niches where they might bite,
if the price is not astronomical. Seems like this pretty much leaves it in
the hands of NSF and other govt. sponsors to initiate the financial backing
- IOW enterprise needs will bask in the the err, fall-out.... maybe!

HPC is a crappy market. Sellers beware : )

DK

 

[Del Cecchi]

First, is this because you suspect that Cray as a company is incompetent or
misguided, technically?... or that HPC is just an uneconomic market? It's
certainly a market where it is difficult to decide a street price for
product.

I do not think Cray is an incompetent company. It seems like that
vector processing has difficulty competing with various forms of
clusters in many HPC applications, yet Cray (formerly Terra) continues
to pursue it. In fact they bought the line from SGI for some reason.
The original Terra MTA seems to have disappeared from view.

Second, which "Cray" are you talking about? The high-capacity COTS cluster
Cray or the high-capabilty (real supercomputer?) one?

Real Supercomputer? What is the definition of "Real Supercomputer"? I
was referring to the traditional Cray Vector line.

Quite honestly, I don't have a good grasp of the answers here and even the
"experts" in the HPC filed seem confused and at odds. It seems, though,
that one important issue involved is: was the anti-dumping embargo on
NEC/HNSX correct or just protectionism to cover up mistakes by the US govt
in procurement policy and/or by Cray in technical direction? That's
somewhat history now but the dilemma remains relevant I think.

The dumping case was presumably due to the fact that a court found that
NEC was indeed "dumping" under the definiton in US law. But wasn't that
a long time ago, technologically speaking? And the HPC market has
changed.

I don't see much evidence of the large corps clamoring for high-end
high-capability computers, though there are niches where they might bite,
if the price is not astronomical. Seems like this pretty much leaves it in
the hands of NSF and other govt. sponsors to initiate the financial backing
- IOW enterprise needs will bask in the the err, fall-out.... maybe!

IBM seems to be selling some Blue Genes, and there seem to be a fair
number of large NUMA and Cluster machines around. As was shown by some
University with a bunch of Macs a pile of boxes and some interconnect
will go a long way, without paying HPC markups.

 

[Del Cecchi]

I haven't seen much evidence to support incompetence. However, it is
rather apparent that Cray will not be successful financially b/c of a
combination of economic and technical reasons. OTOH, some of those
technical reasons keep Cray alive, albeit in a dimished and marginal
state.


HPC is a difficult, difficult market with a mercurial customer base and
low volumes. In other words, it's a commercial disaster.


I'm talking about the company Cray. They are not an economically
viable enterprise, through the combination of their products, or any
individual product line.

Their COTS cluster is not very interesting, since it's a very tough
market with competition from Dell (and that competition will look even
more attractive if they ever release 2S opteron systems) and HP that is
hard to beat. Cray has the same problem as SGI, except that their
value added (the real vector machines) are vastly less popular than the
Altix. I also don't think Cray does storage, but I may be wrong.


When was this embargo?


I actually think the demand for such machines has increased over time.
The issue is whether demand increases faster than Moore's law increases
integration. As we all know, it only takes 4 sockets to build a 8P
machine, and in a short period of time that will be 16S. The larger
machines are being eaten from below, and the one factor that they can
stave them off with is bandwidth to I/O, memory, etc.


HPC is a crappy market. Sellers beware : )

DK

It appears to have been in 97
http://www.ucar.edu/communications/quarterly/fall97/supercomputer.html

"UCAR's acquisition of a NEC SX-4 supercomputer was officially stopped in
late August as a result of two federal decisions. The U.S. Department of
Commerce assigned a dumping margin of 454% for NEC supercomputers. On the
same day, the U.S. Court of International Trade rejected NEC's claim that
the Commerce Department had prejudged the case. In light of these
decisions and in accordance with the regulations of the U.S. Office of
Management and Budget that all procurements be conducted to provide, to
the maximum extent possible, open and free competition, NSF informed UCAR
that it cannot approve the award for the NEC SX-4...."

If that is the case that was referred to. They ended up buying a couple
of Crays.

From Electronic News, "Washington, D.C.--When Cray Research, the
supercomputer subsidiary of Silicon Graphics, Inc., and NEC, along with
its supercomputing subsidiary HNSX Supercomputers, Inc., failed to settle
amicably a Cray complaint over supercomputer dumping, the International
Trade Commission (ITC) was forced to issue a ruling on Friday. The ITC
upheld the U.S. Commerce Department's contention made in August (EN, Aug.
25) that NEC, Fujitsu and other Japanese supercomputer companies were
guilty of dumping and would have to pay the U.S. government duties on all
future imports of supercomputers. Cray will receive no direct
compensation from NEC or any other Japanese firm, the company said."

 

[George Macdonald]

I haven't seen much evidence to support incompetence. However, it is
rather apparent that Cray will not be successful financially b/c of a
combination of economic and technical reasons. OTOH, some of those
technical reasons keep Cray alive, albeit in a dimished and marginal
state.

As usual, "apparent" to you... and you are entitled to that opinion.

HPC is a difficult, difficult market with a mercurial customer base and
low volumes. In other words, it's a commercial disaster.

I thought it was clear that was what I was getting at. The market is
somewhat cyclical though as the pendulum swings between distributed systems
and centralized computing. In fact I sense a swing back to centralized in
the business computing market now after the dominance of distributed for
the past few years.

I'm talking about the company Cray. They are not an economically
viable enterprise, through the combination of their products, or any
individual product line.

Missing the point as I see it.

Their COTS cluster is not very interesting, since it's a very tough
market with competition from Dell (and that competition will look even
more attractive if they ever release 2S opteron systems) and HP that is
hard to beat. Cray has the same problem as SGI, except that their
value added (the real vector machines) are vastly less popular than the
Altix. I also don't think Cray does storage, but I may be wrong.

Dell compared with Cray? What are you smoking?

Now an Itanium cluster is compared with a HPC high-capability system? I
must get some of that stuff you have! If Cray is in trouble, SGI is in
(extended) death rattle.

When was this embargo?

A few years ago, though it was really an anti-dumping order, which was
effectively an embargo... before Cray took over HNSX.

I actually think the demand for such machines has increased over time.
The issue is whether demand increases faster than Moore's law increases
integration. As we all know, it only takes 4 sockets to build a 8P
machine, and in a short period of time that will be 16S. The larger
machines are being eaten from below, and the one factor that they can
stave them off with is bandwidth to I/O, memory, etc.

I said "high-capabilty" systems. While they can be, and usually have,
multiple processors they are not built from COTS. There are simply
problems to be solved, in govt sponsored projects and business, where
nothing else will do.

HPC is a crappy market. Sellers beware : )

There *is* a demand; for commercial/business use, the price is the problem.
For the technical buyer, selling it to management is a *big* hurdle...
which is further confused by management's understanding of the real costs
of distributed computing.

 

[George Macdonald]

I do not think Cray is an incompetent company. It seems like that
vector processing has difficulty competing with various forms of
clusters in many HPC applications, yet Cray (formerly Terra) continues
to pursue it. In fact they bought the line from SGI for some reason.
The original Terra MTA seems to have disappeared from view.

Yeah, it's a difficult market - there are, however, many scientists out
there dreaming of solving the next generation of problems with yet to be
produced high-capability hardware. In the business realm though, it'd be
kinda hard for the likes of Ford to justify the expense, given their
current financial condition.

Real Supercomputer? What is the definition of "Real Supercomputer"? I
was referring to the traditional Cray Vector line.

I was going to say "vector" but I gather that seems insufficient now with
enhancements added to vector processors for scalar, short vector, parallel
etc.

The dumping case was presumably due to the fact that a court found that
NEC was indeed "dumping" under the definiton in US law. But wasn't that
a long time ago, technologically speaking? And the HPC market has
changed.

Aren't the Japanese still building huge supercomputers?... or at least till
very recently?

IBM seems to be selling some Blue Genes, and there seem to be a fair
number of large NUMA and Cluster machines around. As was shown by some
University with a bunch of Macs a pile of boxes and some interconnect
will go a long way, without paying HPC markups.

Sure but that won't tackle the scientific "dreamer" problems.