Meaning of Double.Epsilon

  • Thread starter Thread starter Daniel
  • Start date Start date
cody said:
Yes in memory, but you do not know which CPU registers will be used.


Maybe not precision but it will not be exactly the same value.
The following program outputs "true true false false" on my machine:

double d=0.0;
float f=0.0f;
double d1=1.0;
float f1=1.0f;
double d2=1.3;
float f2=1.3f;
float f3=float.Parse("0.2");
float f4=0.2f;
MessageBox.Show(string.Format("{0} {1} {2} {3}", d==f, d1==f1, d2==f2,
f3==f4));


Thats clear. It it only guaranteed that a variable size 32 bit will be
stored in a register with *at least* 32 bit or more.

So, in the case of constants the '==' operator should always work
correctly.

It won't see above.


If you are lucky it will work on some machines with some values.


I'd vote against it. Testing equality is always bad idea anyway. Even if,
there must be a possibility to specify the tolerance value because the
fuzzyness really depends on the application. The method wouldn't do anything
else but Math.Abs(a,b)<Tolerance so there is very little point for including
such a method in the framework.

--
cody

Freeware Tools, Games and Humour
http://www.deutronium.de.vu || http://www.deutronium.tk

Equality comparison between integers (within the range and precision of your
chosen data type) and rational numbers of the form x/(2^y) ought to always
work. Other than that, you get into the ugliness of trying to losslessly
convert non-integral values between bases. (We all know how fun it is to
write out 1/3 as an "exact" decimal.)
 
Maybe not precision but it will not be exactly the same value.
The point is that
float a=1.3f;
double b=1.3f;
Console.WriteLine(a==b);
Will always return true, because converting "1.3f" (or any other 32-bit
float-value) to double will not change it; Of course this value can actually
be different from "1.3d".

But the comparison returned FALSE! Did you read my posting?
Your original example was:
float a = 1f;
float b = 1f;
if (a==b) { }

As shown above, "1f" (or any other float constant) will not be changed when
converted to a double (or 80-bit fp value), so that kind of comparison *is*
safe.

This could cause the same problem as my first example. I can remember that
soem time ago someone gave me an example of values which really were
difference although there were both float and originally the same constant
was assigned to them.
 
cody said:
...
But the comparison returned FALSE! Did you read my posting?

Cody, please! Correct your code to:

double d=0.0f;
float f=0.0f;
double d1=1.0f;
float f1=1.0f;
double d2=1.3f;
float f2=1.3f;
float f3=float.Parse("0.2");
float f4=0.2f;

Because that's what we've been talking about: converting a 32-bit float to a
64-bit float won't change it's value.
So, it's always safe to compare a 32-bit float with another 32-bit float,
even if the JITer chooses to put it in a 80-bit FPU register.
This could cause the same problem as my first example. I can remember that
soem time ago someone gave me an example of values which really were
difference although there were both float and originally the same constant
was assigned to them.

So, why don't you show that example?

Look, a 32-bit FP number has 1 sign bit, 8 bits exponent and 23 mantissa
bits.
A 64-bit FP number has 1 sign bit, 11 bits for the exponent, and 52 bits for
the mantissa part.
If you convert a 32-bit fp to a 64-bit fp number, all you have to do is
"append zeros" to the mantissa part, which doesn't change it's value, and
convert the exponent from 8 to 11 bits (which doesn't change it either). So,
how do you think something like this
float f1 = SomeValue;
Console.Writeline(f1 == (double)f1);
could ever return false (unless SomeValue was NaN, of course)?

Niki
 
But the comparison returned FALSE! Did you read my posting?
Cody, please! Correct your code to:
[..]
double d2=1.3f;
float f2=1.3f;
[..]


Damn you are right.. A very subtle difference. Now I understand. 1.3 is not
equal to 1.3f, but 1.3f is always 1.3f, no matter if it is put into a float
or double variable or stored in a 64 or 80 bit register.
Look, a 32-bit FP number has 1 sign bit, 8 bits exponent and 23 mantissa
bits.
A 64-bit FP number has 1 sign bit, 11 bits for the exponent, and 52 bits for
the mantissa part.
If you convert a 32-bit fp to a 64-bit fp number, all you have to do is
"append zeros" to the mantissa part, which doesn't change it's value, and
convert the exponent from 8 to 11 bits (which doesn't change it either).

Very interesting information.

But I could swear I read somewhere that the register will affect the
comparison :)
 
cody said:
Very interesting information.

But I could swear I read somewhere that the register will affect the
comparison :)

It can if the conversion goes the other way - if a register holds an 80
bit number and that's converted to a 64 bit number.

Here's an example which actually didn't show up the problem on your box
last time I posted it, but it does show it on mine:

using System;

class Test
{
static float member;

static void Main()
{
member = Calc();
float local = Calc();
Console.WriteLine(local==member);
}

static float Calc()
{
float d1 = 2.82323f;
float d2 = 2.3f;
return d1*d2;
}
}

The member variable is strictly 32 bits, but the local variable can be
enregistered - so it ends up being more accurate than the member
variable, and the comparison fails (on my box).
 
If you convert a 32-bit fp to a 64-bit fp number, all you have to do
is
It can if the conversion goes the other way - if a register holds an 80
bit number and that's converted to a 64 bit number.

Here's an example which actually didn't show up the problem on your box
last time I posted it, but it does show it on mine:

using System;

class Test
{
static float member;

static void Main()
{
member = Calc();
float local = Calc();
Console.WriteLine(local==member);
}

static float Calc()
{
float d1 = 2.82323f;
float d2 = 2.3f;
return d1*d2;
}
}

The member variable is strictly 32 bits, but the local variable can be
enregistered - so it ends up being more accurate than the member
variable, and the comparison fails (on my box).


Now I remember it was you with the example. But I still do not get it.
The explanation given by Niki seemed logical.
So if we never know which variable is put in which register so we can never
rely on that a comparison of two floating point variables will succeed?
So it stands as it is: Equality testing with floating point variables is
never a good idea because the result is undefined.
 
cody said:
Now I remember it was you with the example. But I still do not get it.
The explanation given by Niki seemed logical.
So if we never know which variable is put in which register so we can never
rely on that a comparison of two floating point variables will succeed?
So it stands as it is: Equality testing with floating point variables is
never a good idea because the result is undefined.

If the result is from a calculation, that's right. I believe that if
it's from a constant, or if it's from a calculation which has an answer
which can be exactly represented in the chosen data size, it's fine.

I suspect it's very, very rarely a good idea to use == for comparisons
with non-constants in the first place though.
 
Jon Skeet said:
If the result is from a calculation, that's right. I believe that if
it's from a constant, or if it's from a calculation which has an answer
which can be exactly represented in the chosen data size, it's fine.

Constant are ok, but calculation results usually aren't safe:
Console.WriteLine(Math.Sqrt(2)*Math.Sqrt(2) == 2);
prints out false on my machine.
I suspect it's very, very rarely a good idea to use == for comparisons
with non-constants in the first place though.

I'd definitely agree to that.

Niki
 
Niki Estner said:
Constant are ok, but calculation results usually aren't safe:
Console.WriteLine(Math.Sqrt(2)*Math.Sqrt(2) == 2);
prints out false on my machine.

Well, I'd say that's a slightly different problem again - that's just
the general inaccuracies of floating point arithmetic, rather than
being due to conversions from an 80-bit value to a 64-bit value. I
think it's unreasonable to expect the above to print True necessarily,
but take:

double b = Math.Sqrt(2);
Console.WriteLine (b==Math.Sqrt(2));

I think it's reasonable to expect that to print True, but it may not.
(I'm not saying it's a bad thing for it not to, other than being
counter-intuitive. I understand the reasons for why things work the way
they do.)
 
The member variable is strictly 32 bits, but the local variable can
be
But in Jons example the calculation can be left can certainly out without
altering the meaning the program because it compares the results of two same
calculations using constants only.
Constant are ok, but calculation results usually aren't safe:
Console.WriteLine(Math.Sqrt(2)*Math.Sqrt(2) == 2);
prints out false on my machine.

I wouldn't have expected any other result than a random value from 'true'
and 'false' :)

I always have been thinking of floating point values as "fuzzy things" where
the exact representation of the result of a calculation is not foreseeable.
Maybe there are IEEE standards but who knows where some machines calculates
floating point in their own way? I do not care as long as the result comes
very close to the "real" exact result.
I'd definitely agree to that.

Who wouldn't agree with that (except the microsoft guy who decided that
operator== for fp do not need a compiler warning).
 
cody said:
But in Jons example the calculation can be left can certainly out without
altering the meaning the program because it compares the results of two same
calculations using constants only.

The calculation *uses* constants, but the result of it isn't
expressable exactly in 64 bits. That's the difference - any constant is
basically defined to be "the 64-bit (or 32-bit in case of float) value
closest to what is literally specified". It would be an error (I
believe) for an enregistered variable to take a more accurate value.
I wouldn't have expected any other result than a random value from 'true'
and 'false' :)

I always have been thinking of floating point values as "fuzzy things" where
the exact representation of the result of a calculation is not foreseeable.
Maybe there are IEEE standards but who knows where some machines calculates
floating point in their own way? I do not care as long as the result comes
very close to the "real" exact result.

I believe the result of a particular calculation is and can be
foreseeable. The difficulty here isn't with the calculation, or even
*how* the result is truncated to 64 bits - it's *when* the result is
truncated.
Who wouldn't agree with that (except the microsoft guy who decided that
operator== for fp do not need a compiler warning).

I'm in two minds about that, personally. I can see the argument for a
warning, but at the same time it feels odd.
 
I believe the result of a particular calculation is and can be
foreseeable. The difficulty here isn't with the calculation, or even
*how* the result is truncated to 64 bits - it's *when* the result is
truncated.


This is imho absolutely unforeseeable and depends on the cpu, the jitter and
so on.
I'm in two minds about that, personally. I can see the argument for a
warning, but at the same time it feels odd.

It may feel a bit odd but I do not think one of us ever used == on a
floating point value in a real application.
The only thing I can think of is if (a!=a) to test for NaN but we have
IsNan() for that.
 
cody said:
This is imho absolutely unforeseeable and depends on the cpu, the
jitter and so on.

Yes - but the point I'm making is that it's not the same as the
calculation itself varying in its results.
It may feel a bit odd but I do not think one of us ever used == on a
floating point value in a real application.
The only thing I can think of is if (a!=a) to test for NaN but we have
IsNan() for that.

Sure.
 
cody said:
...

It may feel a bit odd but I do not think one of us ever used == on a
floating point value in a real application.
The only thing I can think of is if (a!=a) to test for NaN but we have
IsNan() for that.

That's the point I've already tried to say: I *do* use tests like "if (a
== -1) a = DoLazyCalculation();" and the like; I'd also guess the equality
operator is implicitly used when you use doubles as keys in a hashtable, or
sort an array of doubles (at least with generics, things like this should
trigger your warning, too). I don't think it would be a good idea to have a
warning *every* time the == operator is used on fp values;

BTW: If you "forbid" the == operator, wouldn't it make sense to apply the
same strictness to other comparison operators? After all, everything said
applies also to comparisons like "Math.Sqrt(2)*Math.Sqrt(2) < 2" as well.

Niki
 
Niki Estner said:
That's the point I've already tried to say: I *do* use tests like "if (a
== -1) a = DoLazyCalculation();" and the like; I'd also guess the equality
operator is implicitly used when you use doubles as keys in a hashtable, or
sort an array of doubles (at least with generics, things like this should
trigger your warning, too).

I don't think the equality operator will be used - the Equals method
will be used. There are subtle differences.
I don't think it would be a good idea to have a
warning *every* time the == operator is used on fp values;

BTW: If you "forbid" the == operator, wouldn't it make sense to apply the
same strictness to other comparison operators? After all, everything said
applies also to comparisons like "Math.Sqrt(2)*Math.Sqrt(2) < 2" as well.

Hmm... interesting point.
 
Jon Skeet said:
I don't think the equality operator will be used - the Equals method
will be used. There are subtle differences.

Yes, you're probably right: After all, support for operators was one of the
weaknesses of .net generics. I just remembered how many compiler
errors/warnings I used to have with C++ generics if someone decided to
implement standard operators in a non-standard way...

But if "==" would issue a warning, while double.Equals would not, wouldn't
that be even more strange? After all, double.Equals will always return the
same value.

The more I think about it, the more I dislike that
"fp-equality-warning"-idea...

Niki
 
But if "==" would issue a warning, while double.Equals would not, wouldn't
that be even more strange? After all, double.Equals will always return the
same value.

Nope - Double.Equals is different to == for NaNs. IEEE specifies that
NaN != NaN, but Object.Equals specifies that x.Equals(x) is true for
all x.

From section 8.2.5.2 of the CLI spec:

<quote>
Note: Although two floating point NaNs are defined by IEC 60559:1989 to
always compare as unequal, the contract for System.Object.Equals,
requires that overrides must satisfy the requirements for an
equivalence operator. Therefore, System.Double.Equals and
System.Single.Equals return True when comparing two NaNs, while the
equality operator returns False in that case, as required by the
standard.
</quote>

Very odd.
 
That's the point I've already tried to say: I *do* use tests like "if (a
== -1) a = DoLazyCalculation();" and the like; I'd also guess the equality
operator is implicitly used when you use doubles as keys in a hashtable, or
sort an array of doubles (at least with generics, things like this should
trigger your warning, too). I don't think it would be a good idea to have a
warning *every* time the == operator is used on fp values;

using -1 is imho a bad idea. what if you actually need -1 in your app?
Negative numbers are very common in floating point (in contrast to
integers). What if you inadvertantly use the float variable with its -1
value?
NaN is just for that purpose: If a value is NaN it stays NaN even if used in
calculations or converted from double to float and vice versa and this way,
you know that something gone wrong if you see NaN in the output.
The following code displays 4 times NaN:

float f = float.NaN;
double d = double.NaN;
Console.WriteLine(double.IsNaN((float)d));
Console.WriteLine(double.IsNaN((double)f));
Console.WriteLine(float.IsNaN((float)d));
Console.WriteLine(float.IsNaN((float)(double)f));

The funny thing is that if you compare a float with and int value -1 the
result returns true even if its value is not -1. The following prints
"hello":

float f1 = -1.00000001f;
float f2 = -0.99999999f;
if (f1==-1 && f2==-1)
{
Console.WriteLine("hello");
}

Btw: using floating point values as hashtable keys seems to be a bad idea.
Even when flooring and ceiling values it doesn't work. The following outputs
2 times false:

float f1 = -0.99999991f;
float f2 = -0.99999999f;
float f3 = 0.99999991f;
float f4 = 0.99999999f;
Console.WriteLine(Math.Ceiling(f1)==Math.Ceiling(f2));
Console.WriteLine(Math.Floor(f3)==Math.Floor(f4));

Only using Math.Round() seems always to work correctly.
BTW: If you "forbid" the == operator, wouldn't it make sense to apply the
same strictness to other comparison operators? After all, everything said
applies also to comparisons like "Math.Sqrt(2)*Math.Sqrt(2) < 2" as well.

Very strange I'd expect the value be smaller as 2 than bigger but it shows
that you cannot rely on exact representation. But I'd never go so far to
issue warnings on all comparison operators for floating point because it
would be simply plain stupid.

Keep in mind that if (f1 < f2) is used very commonly and works fine in
almost every situation (except the programmer makes a wrong assumtion as you
example shows), But in constrast if (f1==f2) will be undefined unless you
use *constants* for *both* values and I cannot imagine a scenario which
requires this which cannot be solved with a better method.

Whatever is said in this thread, one should never assume an exact
representation of floating point variables
even if it seems to work with some values if could fail with other ones or
fail on another platform.
 
But if "==" would issue a warning, while double.Equals would not,
wouldn't
Nope - Double.Equals is different to == for NaNs. IEEE specifies that
NaN != NaN, but Object.Equals specifies that x.Equals(x) is true for
all x.


If == isssues a warning, double.Equals() also should.

Also bare in mind that a Warning doesn't mean "It is wrong" but "It could
possibly be wrong", otherwise it would be an Error.
 
cody said:
using -1 is imho a bad idea. what if you actually need -1 in your app?
Negative numbers are very common in floating point (in contrast to
integers). What if you inadvertantly use the float variable with its -1
value?
NaN is just for that purpose: If a value is NaN it stays NaN even if used
in
calculations or converted from double to float and vice versa and this
way,
you know that something gone wrong if you see NaN in the output.
The following code displays 4 times NaN:

I think this is going slightly off-topic now, but of course "special values"
like these have to be values that cannot be result of a calculation.
Depending on the meaning of the value this could be a negative value (e.g
distances), NaN, a value above 100 (percentages), or whatever. Although NaN
is a good "special value", too, it does have disadvantages, sometimes: AFAIK
Double.Parse can't parse it, converting it to an int will throw an
exception, you only have one single special value, and sometimes NaN can be
result of a calculation...

Niki
 
Back
Top