Number conversion problem

  • Thread starter Thread starter Ovindo
  • Start date Start date
O

Ovindo

If I do this:

Console.WriteLine(Convert.ToString((int)(0.7f * 100)));
Console.WriteLine(Convert.ToString((int)(0.7d * 100)));

The application output is:
69
70

Can you help me to understand how conversion in c# is working?
Regards
Ovindo
 
Ovindo said:
If I do this:

Console.WriteLine(Convert.ToString((int)(0.7f * 100)));
Console.WriteLine(Convert.ToString((int)(0.7d * 100)));

The application output is:
69
70

Can you help me to understand how conversion in c# is working?
Click to expand...

Sure. First, look at
http://www.pobox.com/~skeet/csharp/floatingpoint.html

Then use DoubleConverter.cs (linked in the article).

Let's look at what 0.7d and 0.7f *really* are:

using System;

public class Test
{
static void Main()
{
Console.WriteLine (DoubleConverter.ToExactString(0.7f));
Console.WriteLine (DoubleConverter.ToExactString(0.7d));
}
}

Results:

0.699999988079071044921875
0.6999999999999999555910790149937383830547332763671875

Now let's look at the actual value of the results you got when
multiplying by 100 (before the cast):

0.7f*100 = 69.9999988079071044921875
0.7d*100 = 70

Now in fact, those calculations are being done at compile time by the
C# compiler, but you get the same results if you make it happen at
runtime.

Casting to an int rounds towards zero, hence why you're getting the
results you are.
 
Jon Skeet said:
Sure. First, look at
http://www.pobox.com/~skeet/csharp/floatingpoint.html

Then use DoubleConverter.cs (linked in the article).

Let's look at what 0.7d and 0.7f *really* are:

using System;

public class Test
{
static void Main()
{
Console.WriteLine (DoubleConverter.ToExactString(0.7f));
Console.WriteLine (DoubleConverter.ToExactString(0.7d));
}
}

Results:

0.699999988079071044921875
0.6999999999999999555910790149937383830547332763671875

Now let's look at the actual value of the results you got when
multiplying by 100 (before the cast):

0.7f*100 = 69.9999988079071044921875
0.7d*100 = 70

Now in fact, those calculations are being done at compile time by the
C# compiler, but you get the same results if you make it happen at
runtime.

Casting to an int rounds towards zero, hence why you're getting the
results you are.
Click to expand...

I still have some problems to understand this.
If
0.7f*100 = 69.9999988079071044921875
why is
0.7d*100 = 70
and not 69.99999999999999555910790149937383830547332763671875?

Regards
Ovindo
 
Ovindo said:
I still have some problems to understand this.
If
0.7f*100 = 69.9999988079071044921875
why is
0.7d*100 = 70
and not 69.99999999999999555910790149937383830547332763671875?
Click to expand...

Because the closest exactly representable double to

69.99999999999999555910790149937383830547332763671875

is 70.

(I recognise that it's odd. You'd expect to be able to keep the same
level of precision just through multiplying by an integer, but I
suspect if you look down at the binary level it makes sense.)
 
Back
Top