Math problem using a Double

[Samuel]

Hi

I make the following calculation 1119.8*0.175 and I assign it to a double
then I round it using the Math.Round(Val,2) method and I get 195.96 instead
of 195.67 as I think it should be and the reason is due to the fact that the
value is the double variable before rounding is 195.96499999999997

Is it inaccuracy of the Double or this is the true result?

Can anyone suggest what can be done if it is the double which is inaccurate

Thank you,
Samuel

 

[Thomas Wieczorek]

Is it inaccuracy of the Double or this is the true result?

This is probably a fixed-point rounding error which you get in any
language. Take a look at http://en.wikipedia.org/wiki/Fixed-point_arithmetic#Precision_loss_and_overflow
for more information

 

[Family Tree Mike]

Hi

I make the following calculation 1119.8*0.175 and I assign it to a double
then I round it using the Math.Round(Val,2) method and I get 195.96 instead
of 195.67 as I think it should be and the reason is due to the fact that the
value is the double variable before rounding is 195.96499999999997

Is it inaccuracy of the Double or this is the true result?

Can anyone suggest what can be done if it is the double which is inaccurate

Thank you,
Samuel

195.96499999999997 rounded to two places should be 195.96. The .dd4 means
it rounds down.

 

[Tom Shelton]

Hi

I make the following calculation 1119.8*0.175 and I assign it to a double
then I round it using the Math.Round(Val,2) method and I get 195.96 instead
of 195.67 as I think it should be and the reason is due to the fact that the
value is the double variable before rounding is 195.96499999999997

Is it inaccuracy of the Double or this is the true result?

Can anyone suggest what can be done if it is the double which is inaccurate

Thank you,
Samuel

Floating point math problem. It's just the way it works Try using a
Decimal type.

 

[Michel Posseth [MCP]]

just Human math VS binary computer math

so doing it the human way will give you the expected result

Dim x As Decimal = 1119.8 * 0.175

x = Math.Round(x, 2, MidpointRounding.AwayFromZero)



hth



michel

 

[Herfried K. Wagner [MVP]]

I make the following calculation 1119.8*0.175 and I assign it to a double
then I round it using the Math.Round(Val,2) method and I get 195.96
instead of 195.67 as I think it should be and the reason is due to the
fact that the value is the double variable before rounding is
195.96499999999997

Is it inaccuracy of the Double or this is the true result?

(Complete) Tutorial to Understand IEEE Floating-Point Errors
<URL:http://support.microsoft.com/?scid=kb;EN-US;42980>

IEEE Standard 754 Floating Point Numbers

Can anyone suggest what can be done if it is the double which is
inaccurate

Take a look at the 'Decimal' type.

 

[Steve Gerrard]

Hi

I make the following calculation 1119.8*0.175 and I assign it to a
double then I round it using the Math.Round(Val,2) method and I get
195.96 instead of 195.67 as I think it should be and the reason is
due to the fact that the value is the double variable before rounding
is 195.96499999999997
Is it inaccuracy of the Double or this is the true result?

Can anyone suggest what can be done if it is the double which is
inaccurate
Thank you,
Samuel

Although it is worthwhile to read up on floating point, in your case, the issue
is really Banker's Rounding, that is, rounding to the nearest even digit.

If you try 1119.8 * 0.175 - .01, you will get 195.95499999999998. This also
rounds to 195.96, because the 6 is even.

As Michel pointed out, you can specify the type of rounding by using x =
Math.Round(x, 2, MidpointRounding.AwayFromZero), which says to round halves up
for positive numbers. Or go old school, and use x = CDbl(x.ToString("0.00")).