Comparing different floats fails

  • Thread starter Thread starter O.B.
  • Start date Start date
O

O.B.

In the following piece of code, the == operator is saying that the two
different float values are the same. What's going on here?


float testValueA = float.MaxValue;
float testValueB = float.MaxValue - (float)1.0;
if (testValueA == testValueB)
{
Console.WriteLine("Test Failed");
}
 
O.B. said:
In the following piece of code, the == operator is saying that the two
different float values are the same. What's going on here?

float testValueA = float.MaxValue;
float testValueB = float.MaxValue - (float)1.0;
if (testValueA == testValueB)
{
Console.WriteLine("Test Failed");
}

That is how floating point works.

The C# data type float has only about 7 decimal digits
precision.

float.MaxValue is in the 10^38 magnitude.

Adding 1 to that in 7 decimal digits precision gives
the same.

Arne
 
O.B. said:
In the following piece of code, the == operator is saying that the two
different float values are the same. What's going on here?

There aren't two different float values there. When you subtract one
from the largest possible float value, it's still closer to the largest
possible float value than to any other value, so testValueB is the same
as testValueA.

For a more concrete understanding of this - suppose you are writing
down numbers to three significant decimal digits. Now work out the
result of this calculation:

123000 - 0.0000987

The "perfct" result is 122999.9999013. However, we've got to represent
it to three significant figures - which is 123000 again.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Back
Top