Statistics and Excel 2007

B

bdmccullough

The June issue of the journal Computational Statistics and Data
Anaylsis has published a collection of five articles on Microsoft
Excel 2007. In the past, the journal has published articles critical
of the accuracy of prior versions of Excel. Immediately below
is a quote From the introduction to the special section, followed by
the titles, authors and abstracts of the five articles.


Over the years, this journal has published many articles describing
the errors in Microsoft Excel, errors that raise concerns
about Microsoft's quality assurance procedures, at least insofar as
Excel is concerned. Hence it is perhaps worth comparing
quality assurance in Microsoft's Excel to quality assurance in its
game division. A recent issue of Wired magazine (Thompson,
2007) describes the way that "Bungie", a Microsoft gaming subsidiary,
tests one of its products, a game called Halo3:

"Because it [Bungie] is owned by Microsoft, which launches dozens XBox
and PC games every year, Bungie has access
to one of the most advanced game-testing facilities ever built.
[Bungie has] now analyzed more than 3000 hours of
Halo3 played by some 600 everyday gamers, tracking everything from
favored weapons to how and where - down to
the square foot - players most frequently get killed.

"Bungie doesn't just test its own games this way. It also buys copies
of rival titles and studies those, too, to see how
Halo3 matches up."

"It is difficult not to think that if Microsoft tested business
software the way it tested game software, then the statistical
functions in Excel would be as accurate as those found in any other
major software package. If that were the case, then none
of the articles in this special section would have been written."




(1) "On the accuracy of statistical procedures in Microsoft Excel
2007"
B.D. McCullough, David A. Heiser
Computational Statistics and Data Analysis 52 (10), 4570-4578

abstract
Excel 2007, like its predecessors, fails a standard set of
intermediate-level accuracy tests in three areas: statistical
distributions, random number generation, and estimation. Additional
errors in specific Excel procedures are discussed.
Microsoft's continuing inability to correctly fix errors is discussed.
No statistical procedure in Excel should be used
until Microsoft documents that the procedure is correct; it is not
safe to assume that Microsoft Excel's statistical procedures
give the correct answer. Persons who wish to conduct statistical
analyses should use some other package.

(2) "The accuracy of statistical distributions in microsoft excel
2007"
A. Talha Yalta
Computational Statistics and Data Analysis 52 (10), 4579-4586

abstract
We provide an assessment of the statistical distributions in
Microsoft® Excel versions 97 through 2007 along with two competing
spreadsheet programs, namely Gnumeric 1.7.11 and OpenOffice.org Calc
2.3.0. We find that the accuracy of various statistical functions
in Excel 2007 range from unacceptably bad to acceptable but
significantly inferior in comparison to alternative implementations.
In particular, for the binomial, Poisson, inverse standard normal,
inverse beta, inverse student’s t, and inverse F distributions,
it is possible to obtain results with zero accurate digits as shown
with numerical examples.


(3) "Microsoft Excel's `Not The Wichmann-Hill' random number
generators"
B.D. McCullough
Computational Statistics and Data Analysis 52 (10), 4587-4593

abstract
Microsoft attempted to implement the Wichmann-Hill RNG in Excel 2003
and failed; it did not just produce numbers between zero and
unity, it would also produce negative numbers. Microsoft issued a
patch that allegedly fixed the problem so that the patched Excel 2003
and Excel 2007 now implement the Wichmann-Hill RNG, as least according
to Microsoft. We show that whatever RNG it is that Microsoft
has implemented in these versions of Excel, it is not the Wichmann-
Hill RNG. Microsoft has now failed twice to implement the dozen
lines of code that define the Wichmann-Hill RNG.

(4) "It's easy to produce chartjunk using microsoft excel 2007 but
hard to make good graphs"
Yu-Sung Su
Computational Statistics and Data Analysis 52 (10), 4594-4601

abstract
The purpose of default settings in a graphic tool is to make it easy
to produce good graphics that accord with the principles of
statistical graphics. If the defaults do not embody these principles,
then the only way to produce good graphics is to be sufficiently
familiar with the principles of statistical graphics. This paper shows
that Excel graphics defaults do not embody the appropriate
principles.
Users who want to use Excel are advised to know the principles of good
graphics well enough so that they can choose the appropriate options
to override the defaults. Microsoft® should overhaul the Excel
graphics engine so that its defaults embody the principles of
statistical
graphics and make it easy for non-experts to produce good graphs.

(5) "Teaching statistics with excel 2007 and other spreadsheets."
John Nash
Computational Statistics and Data Analysis 52 (10), 4602-4606

abstract
This article considers which activities in teaching statistics may be
suitable candidates for the application of spreadsheets, and
whether spreadsheets in general and Excel 2007 in particular are
suitable for these tasks.
 
P

perry jones

Dear Bruce,

thanks for pointing this out and thanks for this important work. While
working in the area of credit risk modeling I was often asked if I could
implement the model in Excel. I just presented some of your papers ...

In your paper you document the flaws of Excel's rng. Then you ask in
your conclusion if Excel users will have to wait until 2010 for a new
attempt by Microsoft to supply a working rng.
I know that there is an extension (add-in?) available for excel, which
implements the mersenne twister. I am not an Excel user, but may I ask
if this extension

(http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/VERSIONS/EXCEL/excel.html)

has been checked in some paper?

Thanks,
perry
(3) "Microsoft Excel's `Not The Wichmann-Hill' random number
generators"
B.D. McCullough
Computational Statistics and Data Analysis 52 (10), 4587-4593




The June issue of the journal Computational Statistics and Data
Anaylsis has published a collection of five articles on Microsoft
Excel 2007. In the past, the journal has published articles critical
of the accuracy of prior versions of Excel. Immediately below
is a quote From the introduction to the special section, followed by
the titles, authors and abstracts of the five articles.


Over the years, this journal has published many articles describing
the errors in Microsoft Excel, errors that raise concerns
about Microsoft's quality assurance procedures, at least insofar as
Excel is concerned. Hence it is perhaps worth comparing
quality assurance in Microsoft's Excel to quality assurance in its
game division. A recent issue of Wired magazine (Thompson,
2007) describes the way that "Bungie", a Microsoft gaming subsidiary,
tests one of its products, a game called Halo3:

"Because it [Bungie] is owned by Microsoft, which launches dozens XBox
and PC games every year, Bungie has access
to one of the most advanced game-testing facilities ever built.
[Bungie has] now analyzed more than 3000 hours of
Halo3 played by some 600 everyday gamers, tracking everything from
favored weapons to how and where - down to
the square foot - players most frequently get killed.

"Bungie doesn't just test its own games this way. It also buys copies
of rival titles and studies those, too, to see how
Halo3 matches up."

"It is difficult not to think that if Microsoft tested business
software the way it tested game software, then the statistical
functions in Excel would be as accurate as those found in any other
major software package. If that were the case, then none
of the articles in this special section would have been written."




(1) "On the accuracy of statistical procedures in Microsoft Excel
2007"
B.D. McCullough, David A. Heiser
Computational Statistics and Data Analysis 52 (10), 4570-4578

abstract
Excel 2007, like its predecessors, fails a standard set of
intermediate-level accuracy tests in three areas: statistical
distributions, random number generation, and estimation. Additional
errors in specific Excel procedures are discussed.
Microsoft's continuing inability to correctly fix errors is discussed.
No statistical procedure in Excel should be used
until Microsoft documents that the procedure is correct; it is not
safe to assume that Microsoft Excel's statistical procedures
give the correct answer. Persons who wish to conduct statistical
analyses should use some other package.

(2) "The accuracy of statistical distributions in microsoft excel
2007"
A. Talha Yalta
Computational Statistics and Data Analysis 52 (10), 4579-4586

abstract
We provide an assessment of the statistical distributions in
Microsoft® Excel versions 97 through 2007 along with two competing
spreadsheet programs, namely Gnumeric 1.7.11 and OpenOffice.org Calc
2.3.0. We find that the accuracy of various statistical functions
in Excel 2007 range from unacceptably bad to acceptable but
significantly inferior in comparison to alternative implementations.
In particular, for the binomial, Poisson, inverse standard normal,
inverse beta, inverse student’s t, and inverse F distributions,
it is possible to obtain results with zero accurate digits as shown
with numerical examples.


(3) "Microsoft Excel's `Not The Wichmann-Hill' random number
generators"
B.D. McCullough
Computational Statistics and Data Analysis 52 (10), 4587-4593

abstract
Microsoft attempted to implement the Wichmann-Hill RNG in Excel 2003
and failed; it did not just produce numbers between zero and
unity, it would also produce negative numbers. Microsoft issued a
patch that allegedly fixed the problem so that the patched Excel 2003
and Excel 2007 now implement the Wichmann-Hill RNG, as least according
to Microsoft. We show that whatever RNG it is that Microsoft
has implemented in these versions of Excel, it is not the Wichmann-
Hill RNG. Microsoft has now failed twice to implement the dozen
lines of code that define the Wichmann-Hill RNG.

(4) "It's easy to produce chartjunk using microsoft excel 2007 but
hard to make good graphs"
Yu-Sung Su
Computational Statistics and Data Analysis 52 (10), 4594-4601

abstract
The purpose of default settings in a graphic tool is to make it easy
to produce good graphics that accord with the principles of
statistical graphics. If the defaults do not embody these principles,
then the only way to produce good graphics is to be sufficiently
familiar with the principles of statistical graphics. This paper shows
that Excel graphics defaults do not embody the appropriate
principles.
Users who want to use Excel are advised to know the principles of good
graphics well enough so that they can choose the appropriate options
to override the defaults. Microsoft® should overhaul the Excel
graphics engine so that its defaults embody the principles of
statistical
graphics and make it easy for non-experts to produce good graphs.

(5) "Teaching statistics with excel 2007 and other spreadsheets."
John Nash
Computational Statistics and Data Analysis 52 (10), 4602-4606

abstract
This article considers which activities in teaching statistics may be
suitable candidates for the application of spreadsheets, and
whether spreadsheets in general and Excel 2007 in particular are
suitable for these tasks.
 
B

bdmccullough

Perry,

I am glad someone finds my work useful. :)

The Mersenne Twister is generally accepted as a good RNG for
simulation.

I am unaware of any paper the checks the reliability of the add-in you
mention.

The important thing is that the Mersenne Twister be implemented
correctly, and the company should offer proof of this. We know that
even large programming companies like Microsoft can botch up simple
RNGs like the Wichmann-Hill! :)

Regards,

Bruce
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top