remainders of large floating point numbers

  • Thread starter Thread starter garazy
  • Start date Start date
G

garazy

Hi Lazyweb,

This is probably something I should have remembered from college.


On 'calc.exe'

( ( 10^15 - 0.1 ) % 2) = 1.9


In C#

double result = 1E+15;
result -= 0.1;
result % 2.0


The result is 1.875



Why does this occur?




Secondly,

Why do numbers bigger than 10^15 when 0.1 is subtracted then modulus 2
always return zero? The same when using IEEERemainder ?



Thanks,

Gary
 
Hi Lazyweb,

This is probably something I should have remembered from college.

On 'calc.exe'

( ( 10^15 - 0.1 ) % 2) = 1.9

In C#

double result = 1E+15;
result -= 0.1;
result % 2.0

The result is 1.875

Why does this occur?

Secondly,

Why do numbers bigger than 10^15 when 0.1 is subtracted then modulus 2
always return zero? The same when using IEEERemainder ?

Thanks,

Gary


It appears I've already half answered it

http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Back
Top