Subtracting two 2-decimal place numbers gives result 13-decimal places?

  • Thread starter Thread starter tsdutcher
  • Start date Start date
T

tsdutcher

first cell, the value: 61179.17
second cell, the value: 67083.97
third cell, formula subtracting first cell value from the second.
instead of 95.20, the result is actually 95.19999999999710
Whats up?
 
Or it may just be the fact that those decimal numbers cannot be represented
exactly by fixed point binary numbers.
As an exercise, try to work out the binary representation of 61179.17, and
of 67083.97.
 
Probably either the first or the second cell is the result of a formula
giving the multiple decimal places and formatted to only show the
two.........

Vaya con Dios,
Chuck, CABGx3






- Show quoted text -

Thanks, but I tried that . . . so I re-entered the numbers by hand
with only the two decimal places, and still get the extra significant
figures when I format the result to show them.
 
I assume that you mean 61083.97, not 67083.97, and that you mean
"subtracting second cell value from the first", not "subtracting first cell
value from the second"?

See my answer elsewhere in the thread for the explanation.
 
Probably either the first or the second cell is the result of a formula
giving the multiple decimal places and formatted to only show the
two.........

Vaya con Dios,
Chuck, CABGx3
 
67083.97 - 61179.17 = 5904.80, not 95.20.

As David Biddulph noted, Excel represents numbers in binary, and in binary
most terminating decimal fractions can only be approximated. As documented
in Help, the level of approximation may be accurate to no more than 15
decimal digits. If whatever calculation you actually performed still seems
wrong recognizing that you may not know what is in the 16th figure of your
input numbers, then post back with correct information.

Jerry
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Back
Top