string to double problem

S

sachin

hi everyone .......
could u please help me out of this...........

i have a string with value........

dim str as string="789456123078945612307894561230"

now i need to take the modulos of this number......

dim dbl as double = cdbl(str) mod 256

i get a value of zero.......
but using the microsoft calculator on the desktop i get it as 206

what i see is that when i convert the string to double (ie. cdbl(str))
my string gets converted to 7.8945612307894564E+29 because of which
there is a loss of data and the MODULOS operator on this number return
zero.
where as i should get 206.
please help me out with this problem.....
how do i prevent loss of data when i convert string to double....
 
A

Andrew Morton

sachin said:
i have a string with value........
dim str as string="789456123078945612307894561230"
now i need to take the modulos of this number......

dim dbl as double = cdbl(str) mod 256

i get a value of zero.......
but using the microsoft calculator on the desktop i get it as 206

You're trying to do stuff that numbers in VB aren't designed for. Google for
"arbitrary precision arithmetic", for example a starting point might be:
http://mathforum.org/library/drmath/view/63866.html

Andrew
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top