Calculation accuracy

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

When i tried calculating a simple math, i found a rounding issue in excel:

I made the following computation by having each value in a cell and inputed
formula to compute the answer:

A1 = 19978.39
B1 = 21456.06
C1 = 1477.67
D1 = A1 - B1 + C1 = -0.00000000001819 (instead of ZERO).

I dont understand the reason for this type of rounding eventhough the
amounts involved are relatively round.

May i have the esteemed views of this forum??

Thanks
 
This has to do with the way computer (not just Excel) store decimal numbers
(numbers to base 10) in binary format (numbers to base 2). For details read:
INFO: Visual Basic and Arithmetic Precision
http://support.microsoft.com/defaul...port/kb/articles/Q279/7/55.ASP&NoWebContent=1

(Complete) Tutorial to Understand IEEE Floating-Point Errors
http://support.microsoft.com/defaul...pport/kb/articles/Q42/9/80.ASP&NoWebContent=1

What Every Computer Scientist Should Know About Floating Point
http://docs.sun.com/source/806-3568/ncg_goldberg.html
Go to source>>
http://www.cpearson.com/excel/rounding.htm

best wishes
 
Back
Top