VB 6.3 Int Problem

[Guest]

Since the last update for office 2003, i've been having issues with the int()
function in vb (version 9972 vba retail 6.4.992)

The accounting software that the business i work for writes for dozens of
clients regularly uses x = (int(x*100))/100 to ensure that there are no
fractional cents on the end of prices. It works wonderfully, until one of a
number of values are used. 8.95, 9.95, 2094.95, and a host of other seemingly
random numbers. For these, the end value of x computes as 8.94, 9.94 etc. Is
this something other people have come across??

 

[Cor Ligthert [MVP]]

Chris,

Maybe I read your message wrong, however this is a dotNet VB language
newsgroup.

Probably do you need help on Excel, for which is a very active programmer
newsgroup.

microsoft.public.excel.programmer

If I am right wiht my assuption than I give you much more change on your
question than here.

Cor

 

[Andrew Taylor]

Cor Ligthert [MVP] wrote:

[Without quoting - Original post was about problem with
x = (int(x*100))/100, where 8.95 >>> 8.94 in Excel VBA]

Chris,

Maybe I read your message wrong, however this is a dotNet VB language
newsgroup.

Probably do you need help on Excel, for which is a very active programmer
newsgroup.

microsoft.public.excel.programmer

If I am right wiht my assuption than I give you much more change on your
question than here.

Cor

However, the same problem could occur in .NET:

In Visual Studio Command window:
? 8.95*100
894.99999999999989
? 9.95*100
994.99999999999989


As always with this type of problem, the answer is that you can't rely
on floating point numbers to give exact results.


Andrew Taylor

 

[Herfried K. Wagner [MVP]]

As always with this type of problem, the answer is that you can't rely
on floating point numbers to give exact results.

Additional information:

<URL:http://groups.google.de/group/microsoft.public.dotnet.languages.vb/msg/1d103c8590802792>

 

[AMDRIT]

Here is some more explaination that doesn't require deep thought

Option Explicit

Sub SimpleTest()

Dim SomeNumberA As Double
Dim SomeNumberB As Currency
Dim SomeNumberC As Single

SomeNumberA = 8.950568
SomeNumberB = 8.950568
SomeNumberC = 8.950568

'Floating points are only usefull for scientific notation (Single and
Double)
Debug.Print "Single " & (Int(SomeNumberC * 100)) / 100 & " is a " &
TypeName((Int(SomeNumberC * 100)) / 100)

Debug.Print "Double " & (Int(SomeNumberA * 100)) / 100 & " is a " &
TypeName((Int(SomeNumberA * 100)) / 100)

Debug.Print "Rounded Double " & (Int(Round(SomeNumberA, 2) * 100)) / 100 & "
is a " & TypeName((Int(Round(SomeNumberA, 2) * 100)) / 100)

'Scaled values should be used when precision counts
' Visual basic does not have a decimal value type
' but it does have a CDec (Convert to Decimal) operator
Debug.Print "Decimal " & (Int(CDec(SomeNumberA) * 100)) / 100 & " is a " &
TypeName((Int(CDec(SomeNumberA) * 100)) / 100)

' Currency Value types are the next best thing and will maintain precision
' notice this becomes a double again after all the operations are done
Debug.Print "Currency " & (Int(SomeNumberB * 100)) / 100 & " is a " &
TypeName((Int(SomeNumberB * 100)) / 100)

' A quick work around to your delima
Debug.Print "Rounded Decimal " & Round(CDec(SomeNumberA), 2) & " is a " &
TypeName(Round(CDec(SomeNumberA), 2))

' Another way to skin the cat
Debug.Print "Special " & SpecialReduce(SomeNumberA, 2) & " is a " &
TypeName(SpecialReduce(SomeNumberA, 2))

End Sub

'
Public Function SpecialReduce(InputValue As Variant, Places As Integer) As
Currency

If Not IsNumeric(InputValue) Then
Err.Raise 13, "My Module", "InputValue must be of numeric type."
End If

SpecialReduce = Round(CCur(InputValue), Places)

End Function