Excel is calculating strange since a long time ... (0.05+(-0.07)+0.02+0) =?

  • Thread starter Thread starter weggler.willi
  • Start date Start date
W

weggler.willi

Type in a column

0.05
-0.07
0.02
0

and sum up all.

What's the result ? -3.46945E-18

Common, where did they go to school ?
 
Excel and almost every other computer program maintain decimal (base 10)
numbers in binary (base 2) floating point. Binary (base 2) has two numbers,
0 and 1, which is what computers understand. If a base 10 number is an
integer, it can be represented exactly in base 2. However if it has a
decimal fraction, many times it can only be approximated in base 2. For
example, 0.1 and 0.4 can be represented only approximately in base 2 whereas
0.5 can be represented exactly. To see how a base 10 number is represented
in base 2, 32 bit and 64 bit floating point, see:
http://babbage.cs.qc.edu/IEEE-754/Decimal.html This allows you to enter a
decimal number, see its base 2 representation and see the resultant decimal
number. For example 77.1 decimal after conversion to base 2, 32 bit floating
point, and back to base 10 is 77.099998.
 
Back
Top