Excel Calculation is Incorrect

[Guest]

Put this into a cell and change the format of the cell to number with 16
decimal places...

=1131.97-1123

The answer is 8.9700000000000300

Notice the 3 at the 14th place after the decimal.

Haha what a joke.

I have tried on two Excel 2003 boxes - one with SP1 and one without. Both
calculated it incorrectly

 

[Thief_]

I've seen this b4. It's not an actual bug in XL, from memory, it's really
got to do with the the way the CPU stores the numbers.

Check these:

http://www.cpearson.com/excel/rounding.htm
http://www.bugnet.com/analysis/excel_calc.html
http://support.microsoft.com/default.aspx?scid=kb;en-us;78113&Product=xlw
http://support.microsoft.com/default.aspx?scid=kb;en-us;214118&Product=xlw

 

[JE McGimpsey]

See

http://www.mcgimpsey.com/excel/pennyoff.html

 

[Don S]

Put this into a cell and change the format of the cell to number with 16
decimal places...

=1131.97-1123

The answer is 8.9700000000000300

Notice the 3 at the 14th place after the decimal.

Haha what a joke.

I have tried on two Excel 2003 boxes - one with SP1 and one without. Both
calculated it incorrectly

I'm not a math wiz, but I think it has something to do with binary
(computer) calculation of decimal (base 10) numbers. Use
=round(toberounded) & you'll have fewer "errors".

Don S