Losing Accuracy in reading numbers

A

AGP

I thought i had this down but i think i am missing something. i read in a
series of strings in CSV format that are actually numbers. i do something
like

Dim gVal as Double
Dim sData as String = "77.05788457660967"

gVal = Double.Parse(gArrCoords(0),
Globalization.NumberFormatInfo.InvariantInfo)
Debug.printline(gVal)

and this is what i get printed
77.057884576609666

This one seems to be small but i thought the double was supposed to hold the
exact number. I think the conversion has to do with the accuarcy built into
the format type but I cant figure out how to convert the exact number.
Suppose this was a scientific app that required extreme accuarcy in reading
the source data. How would i read it or assign it differently?

tia

AGP
 
C

Cor Ligthert[MVP]

AGP,

Simple search on Internet about accuracy and doubles. This is a dead horse.

Cor
 
A

AGP

Gotcha. I've been doing that search for the last hour or so. Will do some
more reading and learning but so far I've learned that the double can hold
15 significant digits of accuarcy. I think I was mistakingly assuming the
range of the double would take care of my numbers but that is not the case.

In my case the documentation for the source data only states that the value
is consisting of floating point value.Looks like one solution is changing my
type to Decimal but i have to weight the performnce hit.

AGP
 
M

Michel Posseth [MCP]

Dim sData As String = "77.05788457660967"



Dim GVal As Decimal = Decimal.Parse(sData,
Globalization.NumberFormatInfo.InvariantInfo)

Debug.WriteLine(GVal)







HTH



Michel
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top