1.9489 - System.Math.Truncate(1.9489) = 0.94890000000000008

  • Thread starter Thread starter Aamir Mahmood
  • Start date Start date
Try it in immediate window
1.9489 - System.Math.Truncate(1.9489)

Result is
0.94890000000000008

Am I doing something wrong?
 
Aamir Mahmood said:
Try it in immediate window
1.9489 - System.Math.Truncate(1.9489)

Result is
0.94890000000000008

Am I doing something wrong?

You're expecting decimal numbers to be exactly represented as doubles.

The literal 1.9489 ends up with a double of this exact value:
1.9489000000000000767386154620908200740814208984375

The immediate window happens to print out lots of digits - a simple
call to Console.WriteLine doesn't. Which is the most appropriate action
really depends on what you're doing.

See http://pobox.com/~skeet/csharp/floatingpoint.html for more
information.
 
Aamir said:

What is the question?

If you google for floating point errors, you should find tons of
information about this problem.

In short, a floating point value stored in a N-bit register is always
going to be an approximation of the real value. You will experience both
the case you see, that the value is slightly higher than what you expect
it to be, and the opposite, that it is slightly lower.

This is normal, can't be fixed, except for using a type with higher
precision and you must deal with it in your code.

As a quick-fix, try using the Decimal type.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads


Back
Top