Y
Yousuf Khan
Dell driven out of a market by low-cost competition.
http://biz.yahoo.com/rc/040816/tech_china_dell_1.html?printer=1
Yousuf Khan
http://biz.yahoo.com/rc/040816/tech_china_dell_1.html?printer=1
Yousuf Khan
A
Adam Warner
Hi Yousuf Khan,
How can Dell compete upon price-performance in these markets when they
don't sell CPUs that provide better price-performance and features?
Dell only sells PCs equipped with Intel CPUs, an arrangement not
expected to change in the near term, Amelio said. Lenovo, Hewlett
Packard and China's No. 2 PC seller, Founder Group, have all recently
introduced models in China powered by AMD chips.
Thankfully Intel's got an astonishing marketing machine in Western
countries. Check out these objective truths:
<http://www.infoworld.com/article/04/08/13/33TCworkstation_1.html>
Intel's Xeon-based workstations are much faster than workstations based
on AMD's Opteron when it comes to heavy multitasking
<http://www.infoworld.com/infoworld/article/04/08/13/33TCworkstation-sb_1.html>
Despite a great deal of hype, AMD's 2.2GHz Opteron 248 CPU -- as
embodied in the IBM IntelliStation A Pro workstation -- doesn't fare
well under heavy workloads.
...
In fact, across the range of tests, the Opteron system took an average
of 15 percent longer to complete the tasks than the Xeon.
The Opterons are "in fact CPU-bound and running out of processor
bandwidth." They can't even keep up with last generation Xeons. "The story
gets worse for AMD when you factor in the newest Xeon processors from
Intel."
Infoworld's bottom line:
"... with heavy processing, the 2.4GHz Opterons show their limitations and
the A Pro starts to crawl." They're no match for 3.2GHz Xeons which are
"the performance king."
The benchmark methodology and paucity of information appears to preclude
anyone reproducing the results.
Regards,
Adam
How can Dell compete upon price-performance in these markets when they
don't sell CPUs that provide better price-performance and features?
Dell only sells PCs equipped with Intel CPUs, an arrangement not
expected to change in the near term, Amelio said. Lenovo, Hewlett
Packard and China's No. 2 PC seller, Founder Group, have all recently
introduced models in China powered by AMD chips.
Thankfully Intel's got an astonishing marketing machine in Western
countries. Check out these objective truths:
<http://www.infoworld.com/article/04/08/13/33TCworkstation_1.html>
Intel's Xeon-based workstations are much faster than workstations based
on AMD's Opteron when it comes to heavy multitasking
<http://www.infoworld.com/infoworld/article/04/08/13/33TCworkstation-sb_1.html>
Despite a great deal of hype, AMD's 2.2GHz Opteron 248 CPU -- as
embodied in the IBM IntelliStation A Pro workstation -- doesn't fare
well under heavy workloads.
...
In fact, across the range of tests, the Opteron system took an average
of 15 percent longer to complete the tasks than the Xeon.
The Opterons are "in fact CPU-bound and running out of processor
bandwidth." They can't even keep up with last generation Xeons. "The story
gets worse for AMD when you factor in the newest Xeon processors from
Intel."
Infoworld's bottom line:
"... with heavy processing, the 2.4GHz Opterons show their limitations and
the A Pro starts to crawl." They're no match for 3.2GHz Xeons which are
"the performance king."
The benchmark methodology and paucity of information appears to preclude
anyone reproducing the results.
Regards,
Adam
R
Robert Myers
Adam said:
Do you have evidence you'd like to present that supports your
implication that the InfoWorld conclusions are wrong? Or should we just
stick with your judgment that everybody who buys Intel hardware is a
sucker for Intel's propaganda machine (which is, indeed, very impressive)?
If marketing muscle is a clue to long term survivability (and it is),
then marketing muscle is a legitimate consideration in making buying
decisions.
Leaving that question aside, just how well Intel processors with long
pipelines and SMT stack up against AMD processors with a shorter
pipelines and smaller memory latency but no SMT juggle realistic heavy
workstation workloads is an interesting question, but the net effect of
your post is to leave the weight of evidence with Intel on that
particular question.
RM
E
Evgenij Barsukov
Robert said:
How about this results, which are reasonably easy to reproduce:
http://www.anandtech.com/linux/showdoc.aspx?i=2163&p=1
Opteron shows to outperform (significanly more expensive) Xeon
in database applications.
Regards,
Evgenij
--
__________________________________________________
*science&fiction*free programs*fine art*phylosophy:
http://sudy_zhenja.tripod.com
----------remove hate_spam to answer--------------
Y
Yousuf Khan
Adam Warner said:
It seems Intel doesn't have enough money to market to the entire Chinese
market properly like it does in the Western world. Thus it's processors are
at a disadvantage, simply based on price.
Yousuf Khan
R
Robert Myers
Evgenij said:
I don't think there is any question but that Opteron is an impressive
server chip. I'm not a database guy, so I can't judge the relevance of
the particular benchmark that is cited.
_Published_ results from tpc.org put systems with Opteron in the hunt
for top $/tpmC:
http://www.tpc.org/tpcc/results/tpcc_results.asp?print=false&orderby=priceperf&sortby=asc
but not at the top of the list.
In any case, the point of the InfoWorld article was that the Xeon
workstations excelled on mixed workloads...the kind an actual
workstation user _might_ experience...different for different kinds of
users to be sure, but a better measure of workstation performance than a
database benchmark.
Intel hypes hyperthreading every chance it gets because it's something
Intel's got that AMD doesn't. There's been much online discussion among
people who could be expected to be knowledgeable, and the best
conclusion I can draw about SMT is that, as a design strategy, it's a
wash...if you consider performance per watt or performance per
transistor. That leaves open the question of responsiveness. Anybody
who uses a workstation and does CPU-intensive work has had the
experience of having the system become annoyingly slow. Does
hyperthreading help with _that_? The InfoWorld article suggests that it
does, and a database benchmark doesn't seem particularly relevant.
RM
T
Tony Hill
The CPU has almost nothing to do with the price. The key phrase from
the article is right here:
"Sellers have cut prices to as little as 3,000 yuan ($362) per unit by
offering models without Microsoft's Windows operating system"
That is where the price difference is coming from. Windows is the
ONLY expensive component in a modern low-end computer. The cost of a
WinXP Home Edition license roughly $100. The cost of service and
support is another $100+. The cost of ALL the hardware comes up to
under $200 for a low-end system, and most of that is tied up in the
hard drive and motherboard.
When Dell buys Intel Celeron chips they are paying damn near nothing
for them. Maybe $35 or $40. AMD might be able to sell their chips
for $30 or $35, shaving a few percent off the top, but even in China
and other developing markets that isn't going to make a huge
difference. But cutting $100 off the top by dropping WinXP from the
price definitely will make a huge difference.
A
Adam Warner
Hi Tony Hill,
You make a great point, thanks Tony. But why would a savvy consumer choose
an Intel _Celeron_ over most AMD CPU choices? Doesn't Dell need to hope
that Intel's marketing is so strong in China that consumers will choose
the Intel brand even if computers are priced the same? If Dell cannot rely
upon this perception it cannot compete. Period. Even if it starts selling
"naked PCs". What happens if 64-bit computing becomes a checklist point?
Or gamers find out that an AMD Athlon64 3000+ beat a P4 3.2GHz _Extreme
Edition_ running Doom 3?
Intel has to provide Dell with suitable price
erformance options so it
can compete effectively. Whether this is already hurting Dell is debatable.
Regards,
Adam
You make a great point, thanks Tony. But why would a savvy consumer choose
an Intel _Celeron_ over most AMD CPU choices? Doesn't Dell need to hope
that Intel's marketing is so strong in China that consumers will choose
the Intel brand even if computers are priced the same? If Dell cannot rely
upon this perception it cannot compete. Period. Even if it starts selling
"naked PCs". What happens if 64-bit computing becomes a checklist point?
Or gamers find out that an AMD Athlon64 3000+ beat a P4 3.2GHz _Extreme
Edition_ running Doom 3?
Intel has to provide Dell with suitable price
erformance options so itcan compete effectively. Whether this is already hurting Dell is debatable.
Regards,
Adam
A
Adam Warner
Hi Robert Myers,
Without ascribing a point or motivation to the Infoworld article I will
simply state what it does: It plays on the fears of IT buyers that the
Opteron may not be able to hack it when the going gets tough. It
establishes an amorphous criterion and scary results so the fear can
propagate without ever being disproved or reputations being at stake.
The results are not presented in a way that supports verification. Even
the hypothesis ("mixed workloads...the kind an actual workstation user
_might_ experience") is subjective. The article is powerful benchmarketing.
When Kristopher Kubicki of Anandtech produced his first article on the
Intel Xeon 3.6 he was eviscerated because people could demonstrate how
individual results were so screwed up. At a time which tests one's mettle
he came through admirably.
We should be discussing verifiable benchmarks. Benchmarketing is
fascinating and it's always important to know what Dilbert's Pointy Haired
Boss is going to believe next. But if I'm required to disprove Infoworld's
article then I've already lost.
I asked no-one to believe me. This forum's informed readership is able to
reach their own conclusions about the usefulness of the Infoworld article.
If you have some benchmarks that show that Xeon workstations are much
faster than Opteron workstations at a defined mixed workload then we will
have some concrete figures to discuss and put into context. When results
are verifiable people will be able to comment, for example, "you used the
wrong Linux scheduler for this kind of workstation load. You have
maximised throughput at the expense of interactive responsiveness." The
article asks us to believe these three truths, simultaneously:
(a) The Opteron workstation is faster when running a few tasks.
(b) The Xeon workstation is more responsive when running many tasks.
(c) The Xeon workstation is faster when running many tasks: "In fact,
across the range of tests, the Opteron system took an average of 15
percent longer to complete the tasks than the Xeon."
"The Opteron machine outperformed the Xeons when lightly loaded with
minimal multitasking, but once the real work started, the Opteron stopped.
It was effectively shut down by the same multitasking load that the two
Xeons performed with ease. In the clean environment, it still performed at
less than half the speed of the older and allegedly less-capable Xeons."
I suspect there is a fundamental misconfiguration or inappropriate
software choice that many IT professionals would have been able to
resolve. But is the reputation of Infoworld at stake in the same way that
Anandtech's was? If the answer is no then you need to question why you
believe things based upon authority alone, especially when other
authoritative sources are available which not only say "trust us" but also
provide information to verify that trust.
Regards,
Adam
Without ascribing a point or motivation to the Infoworld article I will
simply state what it does: It plays on the fears of IT buyers that the
Opteron may not be able to hack it when the going gets tough. It
establishes an amorphous criterion and scary results so the fear can
propagate without ever being disproved or reputations being at stake.
The results are not presented in a way that supports verification. Even
the hypothesis ("mixed workloads...the kind an actual workstation user
_might_ experience") is subjective. The article is powerful benchmarketing.
When Kristopher Kubicki of Anandtech produced his first article on the
Intel Xeon 3.6 he was eviscerated because people could demonstrate how
individual results were so screwed up. At a time which tests one's mettle
he came through admirably.
We should be discussing verifiable benchmarks. Benchmarketing is
fascinating and it's always important to know what Dilbert's Pointy Haired
Boss is going to believe next. But if I'm required to disprove Infoworld's
article then I've already lost.
I asked no-one to believe me. This forum's informed readership is able to
reach their own conclusions about the usefulness of the Infoworld article.
If you have some benchmarks that show that Xeon workstations are much
faster than Opteron workstations at a defined mixed workload then we will
have some concrete figures to discuss and put into context. When results
are verifiable people will be able to comment, for example, "you used the
wrong Linux scheduler for this kind of workstation load. You have
maximised throughput at the expense of interactive responsiveness." The
article asks us to believe these three truths, simultaneously:
(a) The Opteron workstation is faster when running a few tasks.
(b) The Xeon workstation is more responsive when running many tasks.
(c) The Xeon workstation is faster when running many tasks: "In fact,
across the range of tests, the Opteron system took an average of 15
percent longer to complete the tasks than the Xeon."
"The Opteron machine outperformed the Xeons when lightly loaded with
minimal multitasking, but once the real work started, the Opteron stopped.
It was effectively shut down by the same multitasking load that the two
Xeons performed with ease. In the clean environment, it still performed at
less than half the speed of the older and allegedly less-capable Xeons."
I suspect there is a fundamental misconfiguration or inappropriate
software choice that many IT professionals would have been able to
resolve. But is the reputation of Infoworld at stake in the same way that
Anandtech's was? If the answer is no then you need to question why you
believe things based upon authority alone, especially when other
authoritative sources are available which not only say "trust us" but also
provide information to verify that trust.
Regards,
Adam
Y
Yousuf Khan
Robert said:
Actually the problem with the Infoworld article is that it's not even really
a true test of multitasking performance. If you read the article, and then
do some checking up on the tools used, it's very shady. First of all, the
benchmarking application is described on the company's website here:
http://analyzer.csaresearch.com/
It's actually called *HTP* Analyzer (i.e. Hyperthreading Analyzer). So it's
a benchmark specifically designed for and geared towards Hyperthreading.
Therefore it's aware of how to detect it, and how to make full use of it. If
you read through the description of this benchmarker a little bit, you'll
find there are two major components of this benchmark suite. First
component, it states that it can test real-world applications through a
test-script functionality; and second, it tests the system's multitasking
efficiency by running simultaneous background workloads. So you think that
since it runs real-world apps in a test-script, therefore it must be one of
those good applications benchmarks and not one of those bad synthetic
benchmarks. However, then you read about what it uses to load down the
background tasks with. According to its webpage, it creates "simulations" of
real-world workloads such as Database, Workflow, and Multimedia. Now these
aren't real database, workflow or multimedia applications, just simulations
of them -- so they are synthetic workloads. He's not running multiple
simultaneous real-world applications; he's running only one real-world app
thread, but several synthetic app threads to load it down. It's a synthetic
benchmark cleverly masquerading as an applications benchmark.
Now, how could this benefit a Hyperthreading processor over a non-HT one?
Well, in an HT CPU, the benchmark can configure it such that it runs the
applications test-script in the primary HT logical processor, while all of
the synthetic load-generating simulations are executed in the secondary
logical processor. Windows would multitask the applications test script in
the run queue of one logical processor where nothing else would be running,
while the synthetics would contend amongst themselves for attention in the
secondary logical processor. In a non-HT CPU, all of the tasks (whether real
or synthetic) would contend for timeslices within the same Windows' run
queue.
So given three simulated workloads and one real application load, when you
put the real application in its own logical processor, what you've
effectively done is given the application test-script a 3:1 priority
advantage over the synthetic workload simulations. In a non-HT CPU, all of
the threads go into the same Windows run queue, and they all get equal
priority according to the default task scheduling behaviour. Only the
real-world app test-script's elapsed time is ever recorded; the results of
the
simulated workloads are never measured and discarded, since they are only
there to add a simulated workload and therefore they are disposable.
Now, is this a good measure of a multitasking workload? Only if you consider
a proper use of multitasking to be running one real-world app in the
foreground while disposable workload simulators bog it down in the
background.
Okay those were just the technical faults about this benchmark. There's also
some conspiracy theory stuff here. One of the co-authors of this article,
Randall C. Kennedy, happens to be the designer of this benchmark:
http://www.csaresearch.com/about.asp
Mr. Kennedy was once an employee of Intel, according to the above biography:
"Later, as a contract testing and development engineer for Intel
Corporation, he led the effort to create tools and resources to articulate
the company's performance initiatives surround high-end desktops (Constant
Computing) and Gigabit Ethernet networking."
Which sounds like he worked in the benchmarketing department.
Furthermore, this guy is some sort of long-time crusader for Hyperthreading.
He's written articles favouring Hyperthreading for a long time now, this one
from about two years ago:
http://www.networkcomputing.com/1324/1324buzz2.html
Nothing wrong with being a crusader for the technology and showing to world
an example of an application that really benefits from Hyperthreading, just
so long as you don't try to pass that off as a benchmark.
Yousuf Khan
Y
Yousuf Khan
Tony Hill said:
Dell sells some systems in the US for around $399, so why is $362 such an
unreachable price point in China? Those systems in the US most likely have
Windows installed on them too. Microsoft gives OEMs such as Dell a break on
prices for prepackaged systems.
It's likely that AMD is able to offer those low prices for the highest
performance Sempron 2800+ or higher, whereas Intel can only offer those
prices on Celeron 2.2Ghz or lower. Mhz marketing then misfires for Intel.
The Celerons that would match up against those Semprons would cost much more
to make for Intel, since Intel would actually have to increase the real-life
clock frequency, whereas AMD only has to dick around with the clock
frequency slightly and assign a huge new Quantispeed number.
Yousuf Khan
Y
Yousuf Khan
Here's another article that basically puts the blame for Intel's (and
therefore Dell's) uncompetiveness squarely on the shoulders of Intel, from
the following article:
http://www.chinadaily.com.cn/english/doc/2004-08/17/content_366242.htm
<quote>
Lenovo, earlier this month, launched a much-cheaper consumer PC series,
using CPUs (central processing units) made by AMD.
Analysts widely believe the low-price strategy, aimed at tapping the
township and rural markets, will help Lenovo increase its market share.
Insiders said Lenovo had asked Intel, without success, to provide low-price
CPUs for its new PC series.
Tapping China's township and rural markets is a natural choice, as the
penetration of PCs in big cities has reached 60-70 per cent, Yang said.
"If our partner cannot give us support, we will surely choose another," Yang
said.
</quote>
Both Lenovo (largest) and Founder (2nd largest) are doing business with AMD,
after years of being Intel loyalists. It looks like the price war in China
is serious stuff and it cannot be influenced by advertising anymore, just
price.
Yousuf Khan
therefore Dell's) uncompetiveness squarely on the shoulders of Intel, from
the following article:
http://www.chinadaily.com.cn/english/doc/2004-08/17/content_366242.htm
<quote>
Lenovo, earlier this month, launched a much-cheaper consumer PC series,
using CPUs (central processing units) made by AMD.
Analysts widely believe the low-price strategy, aimed at tapping the
township and rural markets, will help Lenovo increase its market share.
Insiders said Lenovo had asked Intel, without success, to provide low-price
CPUs for its new PC series.
Tapping China's township and rural markets is a natural choice, as the
penetration of PCs in big cities has reached 60-70 per cent, Yang said.
"If our partner cannot give us support, we will surely choose another," Yang
said.
</quote>
Both Lenovo (largest) and Founder (2nd largest) are doing business with AMD,
after years of being Intel loyalists. It looks like the price war in China
is serious stuff and it cannot be influenced by advertising anymore, just
price.
Yousuf Khan
R
Robert Myers
Adam said:
I wonder who the readers of Anandtech really are.
Who are the readers of Infoworld? Whether the data reflect reality or
not, I'm sure they've got data to show that their readers are serious
prospective enterprise buyers.
In any case, what does a serious enterprise buyer point to when trying
to defend a purchasing decision? To an open source benchmark on
Anandtech? Probably not. The most quoted number I see is $/tpmC, and
that's the comparison I'd want to have handy, not some random benchmark
from Anandtech, and not the kind of ragged commentary it has attracted.
The actual costs and the actual benefits of owning a particular piece of
hardware are, of course, significantly more complicated that $/tpmC,
but, if the Anandtech review is what you want to present in support of
the broad assertion stated within your rhetorical question:
your methodology is at least as flawed as the methodology of the
Infoworld article.
PHB's sign checks and approve PO's. It's important to understand how
they think.
You don't have to prove or disprove anything, but if you're going to
take a broad swipe at the intelligence and judgment of a large group of
people (everyone who buys Intel hardware), you should probably be ready
for some pushback.
I'm sympathetic to your point of view: given the choice, I prefer
benchmarks that I can perform with my own grubby little hands at the
keyboard...at least in theory.
Accurate benchmarking is difficult, there are tons of gotchas, and, in
practice, I tend to refer to standard published benchmarks that are
relevant to my needs, even though the source code for the benchmarks is
not published.
Which gets us to PHB's and to the putative actual readership of a
publication like Infoworld: busy people for whom a purchasing decision
is not a religious experience. They don't need to know that their
purchasing decision is perfect, only that it is defensible. Defensible
reasoning for that kind of purchase is: what do others mostly buy
(Intel, but AMD is making sufficient inroads to have made a significant
leap in acceptability) and what criteria do they look at (published
industry-standard benchmarks).
My reaction to 15 percent longer is said:
Now the article has my attention. I don't care _that_ much about the
details of what a workstation does (certainly not fifteen percent), but
I don't want my workstation to be "shut down" just because I've got a
few things going.
As to evidence, I don't have any I'd like to offer, but then I haven't
made any assertions about the relative capability of Xeon workstations.
You may well be right, but who has time to configure a workload when one
is trying to think and to get things done?
The evidence that the Infoworld article provides is weak and anecdotal.
What does hypethreading really do for a user in such a situation? I
still really don't know. For theoretical reasons, I expect SMT to be
helpful in precisely the situation that is described: the CPU burdened
with multiple, unrelated tasks. The anecdotal evidence supports the
theoretical expectation, that's all.
It does interfere with your apparent need to believe that Opteron is a
slam dunk in terms of value and that only marketing keeps AMD from
driving Intel out of business. I don't know what to do about that.
You are, I assure you, accusing the wrong person of believing things
based upon authority. If the readers of Infoworld find that it provides
them with misleading or useless information, they'll stop reading it and
advertisers will stop providing revenue because Infoworld won't be able
to show readership.
We all have our preferences, but I find the kind of review that
Anandtech and Tom's Hardware (for example) produce to be nearly useless.
Not completely useless, but nearly so. Commentary in Infoworld that
doesn't provide enough details to be verified or refuted? Who knows, if
the specific point is underscored thoroughly enough, maybe someone will
explore the assertion carefully. Until then, it's almost just idle chatter.
RM
A
Adam Warner
Yousuf, many thanks for the analysis. Since the Xeon workstations were
admitted slower to begin with it was extraordinary that they ended up
having faster throughput while remaining highly responsive (this is
usually a tradeoff). If the total work done is never recorded the paradox
is easily resolved ("Only the real-world app test-script's elapsed time is
ever recorded; the results of the simulated workloads are never measured
and discarded, since they are only there to add a simulated workload and
therefore they are disposable.")
Since you raised the link between the reviewer and the benchmarks suite
I've come across this Anandtech forum thread:
`First "real" Nocona vs. Opteron review?'
<http://forums.anandtech.com/messageview.cfm?catid=28&threadid=1348215>
Randall C. Kennedy starts by simply claiming: "Opteron is really good at
doing a few things at once. Saturate the CPU, however, and it tanks." In a
subsequent message he writes: "I meant vs. Xeon. Under complex workloads,
Xeon - especially the new Nocona-based model - stomps all over Opteron."
He makes a strong recommendation:
07/31/2004 11:10 AM
Typical. Your reaction to a poor showing by your CPU of preference is
to dismiss the test as being irrelevant. A bit pathological, don't you
think?
Unfortunately, in my position I don't have the luxury of becoming
emotionally attached to products. My customers - who are primarily in
the financial services sector - have zero tolerance for delays. Time
is literally money for these people, and my workloads model their
runtime environment (which is a huge target market for workstation
vendors).
Bottom Line: I'm strongly recommending that my customers avoid
Opteron-based workstations for demanding, multi-process, multi-tasking
workloads, and I'm echoing these sentiments in my InfoWorld Test Center
contributions on the subject.
RCK
-------------------------
Director, CSA Research
http://www.csaresearch.com
Regards,
Adam
admitted slower to begin with it was extraordinary that they ended up
having faster throughput while remaining highly responsive (this is
usually a tradeoff). If the total work done is never recorded the paradox
is easily resolved ("Only the real-world app test-script's elapsed time is
ever recorded; the results of the simulated workloads are never measured
and discarded, since they are only there to add a simulated workload and
therefore they are disposable.")
Since you raised the link between the reviewer and the benchmarks suite
I've come across this Anandtech forum thread:
`First "real" Nocona vs. Opteron review?'
<http://forums.anandtech.com/messageview.cfm?catid=28&threadid=1348215>
Randall C. Kennedy starts by simply claiming: "Opteron is really good at
doing a few things at once. Saturate the CPU, however, and it tanks." In a
subsequent message he writes: "I meant vs. Xeon. Under complex workloads,
Xeon - especially the new Nocona-based model - stomps all over Opteron."
He makes a strong recommendation:
07/31/2004 11:10 AM
Typical. Your reaction to a poor showing by your CPU of preference is
to dismiss the test as being irrelevant. A bit pathological, don't you
think?
Unfortunately, in my position I don't have the luxury of becoming
emotionally attached to products. My customers - who are primarily in
the financial services sector - have zero tolerance for delays. Time
is literally money for these people, and my workloads model their
runtime environment (which is a huge target market for workstation
vendors).
Bottom Line: I'm strongly recommending that my customers avoid
Opteron-based workstations for demanding, multi-process, multi-tasking
workloads, and I'm echoing these sentiments in my InfoWorld Test Center
contributions on the subject.
RCK
-------------------------
Director, CSA Research
http://www.csaresearch.com
Regards,
Adam
A
Adam Warner
http://analyzer.csaresearch.com/
Error: Incompatible Browser Detected
We're Sorry! This Performance Portal site requires Microsoft Internet
Explorer 5.0 or later (IE 5.5 or later recommended). You can obtain the
latest version of IE from the Microsoft Internet Explorer web site.
Note: For a complete list of system requirements, please see our
Performance Portal product information page.
Error: Incompatible Browser Detected
We're Sorry! This Performance Portal site requires Microsoft Internet
Explorer 5.0 or later (IE 5.5 or later recommended). You can obtain the
latest version of IE from the Microsoft Internet Explorer web site.
Note: For a complete list of system requirements, please see our
Performance Portal product information page.
A
Adam Warner
Hi Yousuf Khan,
If the amount of work done in the background is never taken into account
then the technique is grossly misleading. Here's how the testing technique
could be improved:
1. Measure the amount of work completed by the Xeon workstation in the
simulated workloads.
2. On the Opteron workstation reduce the priority on the simulated
workloads until the Opteron only completes as much work in the simulated
workloads as the Xeon.
3. Compare the responsiveness and throughput of the foreground real-world
application while each workstation is approximately completing _the same
amount of background work_.
Everyone who multitasks cares about how much work is being done in the
background.
Regards,
Adam
If the amount of work done in the background is never taken into account
then the technique is grossly misleading. Here's how the testing technique
could be improved:
1. Measure the amount of work completed by the Xeon workstation in the
simulated workloads.
2. On the Opteron workstation reduce the priority on the simulated
workloads until the Opteron only completes as much work in the simulated
workloads as the Xeon.
3. Compare the responsiveness and throughput of the foreground real-world
application while each workstation is approximately completing _the same
amount of background work_.
Everyone who multitasks cares about how much work is being done in the
background.
Regards,
Adam
Y
Yousuf Khan
Adam Warner said:
Yeah, I know about those remarks of his in Anandtech too. He posted them
almost a month or two before this Infoworld article came out.
It was enough for me to join Anandtech's forums and post a message asking
him to explain his benchmark methodologies. So far I haven't received any
response from him. But likely he might not be following the thread right
now.
Yousuf Khan
Y
Yousuf Khan
Adam Warner said:
This is quite possible to do in the Opteron using just Windows task
switching mechanisms. Raise the priority of the foreground process, while
reducing the priorities of the disposable workloads.
Yes, exactly, if you're multitasking in the background, then chances are
that the programs running in the background are just as important to you as
those in the foreground. In both cases, you're trying to get some useful
work done, otherwise you wouldn't be running the secondary processes.
If this guy had only made the benchmark have the ability to run a second
test-script with a real-world application running in there too, then measure
its completion time, it would be a real worthwhile application. It would be
useful to know how fast it could run all tasks it is running, not just one
task.
Yousuf Khan
Y
Yousuf Khan
Robert Myers said:
Apparently, many of them are writers for Infoworld.
The aforementioned Randall C. Kennedy, the co-author of the Hyperthreading
benchmark in Infoworld can be found wondering around the forums at
Anandtech.
Yousuf Khan
R
Robert Myers
Yousuf said:
Your key claim (I believe) is that the benchmark software is a
subterfuge by way of giving scheduling attention to the jobs on the
hyperthreaded system but not on the Opteron system. That's an
interesting theory, and it may well be correct, but your analysis rests
on assumptions about the actual benchmark and about scheduling behavior
that I don't know how to check.
One can always, at least in theory, arrange job priorities so that
background jobs interfere minimally with foreground jobs. Without any
constraint on how the background jobs are hog-tied, you could probably
get any result you wanted...if indeed you are fiddling with scheduling
priorities.
"Benchmark" is a pretty broad term. The manufacturer benchmarks that
are published in places like specbench.org, tpc.org, and
http://www.cs.virginia.edu/stream site aren't perfect, but at least they
put hardware on a common footing and the rules are spelled out in detail
for all to see and to complain about. Manufacturers are free to do
whatever they want, so long as they don't break the rules. That leaves
alot of room for creativity, and people get pretty creative.
As to everything else, a benchmark tests the hardware, the software, the
compiler, and the care, insight, skill, and impartiality of whoever is
performing the benchmark. That's alot of unknowns, no matter what you
call the result.
csaresearch.com has a skewed view of things resulting from a desire to
sell advertising? The "Seeing double?" stuff right on the web page you
linked to is probably a better clue than Randall Kennedy's c.v.
Someone is influenced by his "strong recommendations" despite an
apparent conflict of interest? Caveat emptor.
RM