Repost: Decimal division/remainder fail near Decimal.MaxValue

[Guest]

The following test program illustrates this point.

Decimal Remainder overflows while calculating near MaxValue.
Decimal Divison fails to get correct answer at MaxValue / 10M or ToString()
is rounding on us quietly ?
Analysis in Debugger leads to
Decimal.MaxValue has sign 0, scale 0, mantissa 0xffff ffff ffff ffff ffff
ffff Decimal.MaxValue/10m has sign 0, scale 0, mantissa 0x1999 9999 9999
9999 9999 999A, which is a rounding of the "correct" mantissa of 0x1999 9999
9999 9999 9999 9999.8 The correct representation of the correct value for
the quotient should have sign 0, scale 1, mantissa 0xffff ffff ffff ffff
ffff ffff

The expression Decimal.MaxValue * 0.1m gives the correct answer.

In particular, the boundary is:
79228162514264337567774146561m/10m gives the wrong answer.
79228162514264337567774146560m/10m comes out even so I can't tell.
79228162514264337567774146559m/10m gives the right answer.

The representation of the wrong answer is sign 0, scale 0, mantissa 0xffff
ffff ffff fffa 0000 0001

The scale of the dividend does not seem to make any difference:
(Decimal.MaxValue*0.1m) / 10m also gives the wrong answer on the division.
No matter what power of 10 the divisor is, you only seem to lose one digit
of precision.


Test Program


using System;
namespace test
{
public class MainTest
{
static public void Main()
{
System.Decimal r1 = Decimal.MaxValue;
System.Decimal r2 = 0.1M;
System.Decimal result;
for( int i = 1; i < 6; ++i)
{
try
{
Console.Write( r1.ToString() + " % " + r2.ToString() + " =" );
result = r1 % r2;
Console.WriteLine( result.ToString());
}
catch( System.Exception e )
{
Console.WriteLine( e.Message );
}
r1 /= 10M;
}
}
}
}
#if OUTPUT
79228162514264337593543950335 % 0.1 =Value was either too large or too small
for a Decimal.
7922816251426433759354395034 % 0.1 =Value was either too large or too small
for a Decimal.
792281625142643375935439503.4 % 0.1 =0
79228162514264337593543950.34 % 0.1 =0.04
7922816251426433759354395.034 % 0.1 =0.034
#endif

Same problem occurs in .NET 2.0 (beta) and has been reported in the beta bug
tracking system.

 

[Gary Chang[MSFT]]

Hi,

Currently I am looking for somebody who could help you on it. We will reply
here with more information as soon as possible.
If you have any more concerns on it, please feel free to post here.


Thanks for your understanding!

Best regards,

Gary Chang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.
--------------------

 

[Peter Huang]

Hi,

From the MSDN,
The decimal type is a 128-bit data type suitable for financial and monetary
calculations. The decimal type can represent values ranging from 1.0 ¡Á
10?28 to approximately 7.9 ¡Á 1028 with 28-29 significant digits.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/csspec/html
/vclrfcsharpspec_4_1_7.asp

That is to say, the operation between decimal will only guarantee at most
28 significant digits, but we can not guarantee the 29th digit will be
accurate.
Hope this helps.

Best regards,

Peter Huang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.