Double is doing incorrect subtraction

N

Nathan Sokalski

I have the following code:

Dim x As Double = 45.333
Dim y As Double = 45
Dim z As Double = x - y
Me.Label1.Text = z.ToString()

The result should obviously be 0.333, but the value displayed in
Me.Label1.Text is:

0.332999999999998

I have seen stuff like this in other situations, but I never managed to
figure out how to fix it. Can somebody help me here?
 
T

Teemu

Nathan Sokalski said:
I have the following code:

Dim x As Double = 45.333
Dim y As Double = 45
Dim z As Double = x - y
Me.Label1.Text = z.ToString()

The result should obviously be 0.333, but the value displayed in
Me.Label1.Text is:

0.332999999999998

I have seen stuff like this in other situations, but I never managed to
figure out how to fix it. Can somebody help me here?

In order to show the correct result you have to format it. For example you
could do this:
Me.Label1.Text =z.ToString("0.000")

Doubles are floating point numbers and that causes this behavior.

If you want to find more information try keyword "IEEE 754" with Google.

-Teemu
 
F

Family Tree Mike

Consider using decimal instead of double if you want to get the answer you
expect below.
 
E

Erik Funkenbusch

I have the following code:

Dim x As Double = 45.333
Dim y As Double = 45
Dim z As Double = x - y
Me.Label1.Text = z.ToString()

The result should obviously be 0.333, but the value displayed in
Me.Label1.Text is:

0.332999999999998

I have seen stuff like this in other situations, but I never managed to
figure out how to fix it. Can somebody help me here?

Double is doing entirely correct subtraction.

The problem is that Double is an implementation of IEEE floating point, and
IEEE floating point is incapable of representing all numbers exactly.

Read this:

http://support.microsoft.com/kb/42980
 
C

Cor Ligthert[MVP]

Nathan,

This is basic

Float, Double or whatever floating poing calculation for mathimatical
programming,
Decimal for business programming,

Cor
 
H

Herfried K. Wagner [MVP]

Nathan Sokalski said:
Dim x As Double = 45.333
Dim y As Double = 45
Dim z As Double = x - y
Me.Label1.Text = z.ToString()

The result should obviously be 0.333, but the value displayed in
Me.Label1.Text is:

0.332999999999998

I have seen stuff like this in other situations, but I never managed to
figure out how to fix it. Can somebody help me here?

In addition to the other replies, take a look at these articles:

IEEE Standard 754 Floating Point Numbers
<URL:http://steve.hollasch.net/cgindex/coding/ieeefloat.html>
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top