Floating point 'near miss'

D

David Veeneman

I don't work with floating point numbers very often, but I have to on a
current project. I'm using doubles, which are causing me a real headache. I
have two variables, both of which should have a value of 0.3. Here are the
values I'm getting:

0.299999999999998
0.300000000000002

There are 15 decimals because I've tried using Math.Round at 15 places to
force these near misses to the correct value. I've learned that works on
some values, but not others. I could try 14 places, but then I'd end up with
other values that would have the problem at 14 places, and so on.

Is there an easy way to force numbers like this to their correct values? And
can anyone recommend a good article on handling floating point numbers in
C#? I Googled it this afternoon with very little luck.

Thanks in advance for your help!
 
J

Jon Skeet [C# MVP]

David Veeneman said:
I don't work with floating point numbers very often, but I have to on a
current project. I'm using doubles, which are causing me a real headache. I
have two variables, both of which should have a value of 0.3.

That's impossible. A double *can't* have a value of 0.3. The closest it
can have is 0.299999999999999988897769753748434595763683319091796875.

See http://www.pobox.com/~skeet/csharp/floatingpoint.html for more
information.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top