D
David Veeneman
I don't work with floating point numbers very often, but I have to on a
current project. I'm using doubles, which are causing me a real headache. I
have two variables, both of which should have a value of 0.3. Here are the
values I'm getting:
0.299999999999998
0.300000000000002
There are 15 decimals because I've tried using Math.Round at 15 places to
force these near misses to the correct value. I've learned that works on
some values, but not others. I could try 14 places, but then I'd end up with
other values that would have the problem at 14 places, and so on.
Is there an easy way to force numbers like this to their correct values? And
can anyone recommend a good article on handling floating point numbers in
C#? I Googled it this afternoon with very little luck.
Thanks in advance for your help!
current project. I'm using doubles, which are causing me a real headache. I
have two variables, both of which should have a value of 0.3. Here are the
values I'm getting:
0.299999999999998
0.300000000000002
There are 15 decimals because I've tried using Math.Round at 15 places to
force these near misses to the correct value. I've learned that works on
some values, but not others. I could try 14 places, but then I'd end up with
other values that would have the problem at 14 places, and so on.
Is there an easy way to force numbers like this to their correct values? And
can anyone recommend a good article on handling floating point numbers in
C#? I Googled it this afternoon with very little luck.
Thanks in advance for your help!