Problems with Very Large Numbers

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

My app is trying to multiply and add some very large numbers. Here is the line that I am am having problems with:

Seconds = CDbl(PartB) * CDbl(16777216)

Seconds is a double and PartB is a short. Partb is 15 before this line. After the multiplication, the value held in seconds is 549420. This obviously isn't correct. I know that the problem is the conversion between types and how much space vb.net uses for its evaluation. However, I'm not sure how to fix it. Any ideas? Thanks for the help.
 
Hi,

I can't reproduce your error.


Dim PartB As Short = 15
Dim seconds As Double = CDbl(PartB) * CDbl(16777216)
Trace.WriteLine(seconds.ToString)

Trace.WriteLine(15 * 16777216)


Both methods return a value of 251658240.

Ken
--------------
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads


Back
Top