how can I compare floating point numbers?

[aprivate]

Hi
I have a newbie question.

I have a value which is of type SINGLE



DIM s as single
s=3.7

however when I use the IF statement to compare

if s = 3.7 then
something
endif

It does not work, ie comparision fails.

What should I be doing so that I can
compare variable to floating point values?

Regds
Aprivate

 

[aprivate]

Hi
I found the answer by using
if s = convert.tosingle(3.7) then
something
endif

I guess type checking is very strict in vb.net

Aprivate

 

[Herfried K. Wagner [MVP]]

I have a value which is of type SINGLE



DIM s as single
s=3.7

however when I use the IF statement to compare

if s = 3.7 then
something
endif

It does not work, ie comparision fails.

Exactly comparing two floating point numbers rarely makes sense. Instead
you can determine the absolute difference between the two numbers and check
if it is smaller than a certain threshold.

\\\
If Math.Abs(s - 3.7F) < <small number> Then
...
End If
///

 

[Robin Tucker]

Yes, define an "EPSILON" value, the smallest value under which you can say
the two are more or less the same, then:

If n1 < ( n2 - EPSILON ) or n1 > ( n2 + EPSILON ) Then

.. They aren't more or less the same ..

Else

... They are

End If

 

[Jay B. Harlow [MVP - Outlook]]

aprivate,
In addition to the other comments:

3.7 or 3.7R is a Double literal, 3.7F is Single literal, while 3.7D is a
Decimal literal.

You can use Single.Epsilon or Double.Epsilon for predefined Epsilon values,
otherwise you could define your own.

Something like:

Dim s As Single
s = 3.7

If Math.Abs(s - 3.7F) < Single.Epsilon Then

End If

If Math.Abs(s - 3.7) < Double.Epsilon Then

End If

I would consider encapsulating the comparison in a module someplace if I
needed to use the Epsilon factor a lot...

For details on Epsilon see:

http://msdn.microsoft.com/library/d...f/html/frlrfsystemsingleclassepsilontopic.asp

http://msdn.microsoft.com/library/d...f/html/frlrfSystemDoubleClassEpsilonTopic.asp

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnadvnet/html/vbnet03252003.asp

Hope this helps
Jay


| Hi
| I have a newbie question.
|
| I have a value which is of type SINGLE
|
|
|
| DIM s as single
| s=3.7
|
| however when I use the IF statement to compare
|
| if s = 3.7 then
| something
| endif
|
| It does not work, ie comparision fails.
|
| What should I be doing so that I can
| compare variable to floating point values?
|
| Regds
| Aprivate
|
|