rounding error?

  • Thread starter Thread starter Steve
  • Start date Start date
S

Steve

A1 = 5.0568725
B1 = 5.0569
C1 = Round(A1,4)-B1

Why doesn't C1 give me a zero value?
 
Regardless of how the cell has been formatted, cell A1 still contains the
value of 5.0568725. To get a zero you would have to round the number in cell
A1 to 4 decimal positions"

=ROUND(5.0568725,4)
 
Go to format cell | number | category | number | decimal places 0 | ok
 
In my C1 formula why doesn't "round(a1,4)" calculate a value of 5.0569 so
when I subtract 5.0569 I get zero?
 
Initially I overlooked your formula when I responded the first time but I've
not been able to replicate your answer. I'm using Excel 2003 and regardless
of where I do the rounding I'm getting 0
 
My guess is that B1 is the result of a calculation an that =(B1-5.0569) is
not zero.

Computers do finite precision math, so rounding errors can accumulate.
Consequently, with floating point values you should test whether
ABS(x-y)<epsilon, instead of whether x=y.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Formula 4
include rounding in a formula 2
round up the number 7
Average rounded to nearest 0.50 4
Subtracting Rounded Numbers 6
If Function and rounding numbers 3
Formula problem 5
If Functions Time 13

Back
Top