Decimal precision ?

  • Thread starter Thread starter Piotr
  • Start date Start date
P

Piotr

Hi

I'm working with Datagrid containg decimal (editable) numbers. The
user-entered numbers are read by decimal.Parse() method. However, Datagrid
formats some decimals as "1", but some as (eg) "1.00" - depends on what the
user had entered. The page has DISABLED viewstate.

So, let someone explain me why the number 1 is not the same as 1.0 or 1.000?
I cannot find any property to set the precision.

The number of decimal digits can be different in different rows, so no
static format option can be used. Currently the only option to get correct
formatting is (a bit stupid) construction like this:
((decimal)DataBinder.Eval(Container.DataItem, "Quantity")).ToString("F" +
((int)DataBinder.Eval(Container.DataItem, "DecimalPlaces")).ToString()

Regards.
 
Hi, Piotr,

Piotr said:
Hi

I'm working with Datagrid containg decimal (editable) numbers. The
user-entered numbers are read by decimal.Parse() method. However, Datagrid
formats some decimals as "1", but some as (eg) "1.00" - depends on what the
user had entered. The page has DISABLED viewstate.

So, let someone explain me why the number 1 is not the same as 1.0 or
Click to expand...
1.000?

There is no difference *between the values* indeed. The difference is in the
way these are stored.

1 is stored as 1 / 10^0
1.0 is stored as 10 / 10^1
1.000 is stored as 1000 / 10^3

See:
http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemDecimalClassTopic.asp

This code illustrates it:

decimal d = decimal.One;
for(int i = 0; i < 20;i++)
{
d = -decimal.Parse(d.ToString("F" + i));
int[] strip = decimal.GetBits(d);
ulong value = Convert.ToUInt64(
((uint)strip[1]).ToString("x") +
((uint)strip[0]).ToString("x"), 16);
Console.WriteLine(
"{0,-22} = {1,-22} / 10^{2,-3} {3}",
d,
value,
(strip[3] & 0xFF0000) >> 16,
(((strip[3] >> 31) & 1) > 0 ? "Negative" : "Positive"));
}

See the topic on GetBits method:
http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemDecimalClassGetBitsTopic.asp
I cannot find any property to set the precision.

The number of decimal digits can be different in different rows, so no
static format option can be used. Currently the only option to get correct
formatting is (a bit stupid) construction like this:
((decimal)DataBinder.Eval(Container.DataItem, "Quantity")).ToString("F" +
((int)DataBinder.Eval(Container.DataItem, "DecimalPlaces")).ToString()
Click to expand...

You can try setting the DataFormatString property alternatively:

"F" + DataBinder.Eval(Container.DataItem, "DecimalPlaces").ToString()

should be fine.

Hope this helps
Martin
 
Back
Top