Calculation error?

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

I have a number of formulas that are showing incorrect results.
One of these works out to be (with each value coming from another cell or
lookup table):

=.69-(ROUND(.03*22,8,2))
or
=.69-ROUND(.684,2)
or
=.69-.68
or
=.01

However, I get 0.0099999999999999
Other calculations yield similar results. Sometimes ending with a 95 or 98
instead of 99.

Now, I know I can work around this by inserting additional ROUND statments
in my formula, but I'd like to understand how this error is getting
introduced.

If I just type in =.69-ROUND(.648,2), I STILL get a multi-digit answer.
Even =.69-.68 doesn't come out to .01.

I don't recall having this kind of error in previous versions of Excel.
What's up here?
 
silver said:
I have a number of formulas that are showing incorrect results.
One of these works out to be (with each value coming from another cell or
lookup table):

=.69-(ROUND(.03*22,8,2))
or
=.69-ROUND(.684,2)
or
=.69-.68
or
=.01

However, I get 0.0099999999999999
Other calculations yield similar results. Sometimes ending with a 95 or
98
instead of 99.

Now, I know I can work around this by inserting additional ROUND statments
in my formula, but I'd like to understand how this error is getting
introduced.

If I just type in =.69-ROUND(.648,2), I STILL get a multi-digit answer.
Even =.69-.68 doesn't come out to .01.

I don't recall having this kind of error in previous versions of Excel.
What's up here?

Nothing is wrong! Excel, like all computer programs, can only calculate to
finite precision. (Your memory about previous versions of Excel being
different is mistaken.) Rounding errors are inherent - you just have to
learn to live with them and deal with your results accordingly, using ROUND
as you suggest or whatever is appropriate. Just don't expect exact answers!
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Back
Top