5.5555555E02 gives wrong number

[Dymondjack]

Hello again and thanks in advance.

I'm sure this is an error of mine and not vb, but I've got a Double number:

dblFeed = dblFeed / intSpeed

In my example dblFeed calculates to 5.55555E02 rather than 0.05555555....
which I understand is correct for scientific notation. However, I need to
put this into some text based code for a machine, and it is going in as
5.5556 rather than 0.0556

I run a custom function to fix the decimal at 4 places (as much as our
machines will process). Any idea's on how I can convert the sci notation to a
standard number? Here's my code:

'***** CODE START *****
dblFeed = dblFeed / intSpeed
dblFeed = FixedDecimal(dblFeed, 4)

'FixedDecimal Function
'==============================================================================
' Returns a Double with a specified Fixed Decimal Place
' Written by Jack Leach - please do not redistribute without this function
header
'==============================================================================
Public Function FixedDecimal(dblNumber As Double, intPlace As Integer) As
Double
On Error GoTo Error_FixedDecimal
Dim dblResult As Double
'=========================
Dim strNumber As String
Dim strChar As String
Dim intPos As Integer
Dim intLen As Integer
'=========================
strNumber = Str(dblNumber)
intPos = InStr(1, strNumber, ".")
intLen = Len(strNumber)

If intPos <> 0 Then
If intLen > (intPos + intPlace) Then
strNumber = Left(strNumber, intPos + intPlace)
End If
End If

FixedDecimal = CDbl(strNumber)
'=========================
Exit_FixedDecimal:
Exit Function
Error_FixedDecimal:
pbl_Lng = fSetAccessWindow(SW_SHOWNORMAL)
pbl_Lng = 0
MsgBox "Error " & Err.Number & " (" & Err.Description & ") in procedure " _
& "FixedDecimal of Module modProgramConversion"
Resume Exit_FixedDecimal
Resume
End Function

'***** END CODE *****

It the time of writing this function, I had not been able to find a numeric
fixed decimal rounding function. I assume I'm getting the issure due to the
fact that the double is being converted to a string for evaluation. Working
with scientific notation is code is brand new for me. Any help would be
greatly appreciated.

Thanks,
Jack

 

[J_Goddard via AccessMonster.com]

Try using the format function.

For example

format(5.555555e-02,"0.0000") yields 0.0556 as a character string.

You could use format(dblFeed / intSpeed,"0.0000")

John

 

[Dymondjack]

Thank you... hadn't know about the function, that should work perfectly.

 

[Jamie Collins]

It the time of writing this function, I had not been able to find a numeric
fixed decimal rounding function.

Which rounding algorithm? e.g. the Format() expression uses asymmetric
rounding away from zero. See:

Custom Rounding Procedures
http://support.microsoft.com/kb/196652

Jamie.

--