C# floating point consistency

  • Thread starter Thread starter news.microsoft.com
  • Start date Start date
N

news.microsoft.com

I spent the wee hours of last night tracking down a lock-up bug that was
causing SorterObjectArray.QuickSort to loop indefinitely. It was comparing
an object with itself, and yet the IComparer function was returning -1! The
function was calculating the area of RectangleF on the object by
multiplying width and height and it turned out that the areas were coming
out different. Why? Because the second calculation is staying in the 80bit
register, while the first calculation was being written and recalled from a
32bit float in ram.

The workaround, in this specific case, is easy. I just compare to see if the
two variables point to the same object, but I'm concerned that C#/.NET
doesnt have a /Op option, like C++ does, to truncate all registers acting as
floats to 32bits. Have I missed it?

Jamie Briant
 
The workaround, in this specific case, is easy. I just compare to see if the
two variables point to the same object, but I'm concerned that C#/.NET
doesnt have a /Op option, like C++ does, to truncate all registers acting as
floats to 32bits. Have I missed it?
Click to expand...

You haven't. You might compare against a maximum threshold rather
than for equality, which is a smart thing to do anyway when you're
using FP calculations. But in general, C# and .NET are just not
particularly well suited to numerical applications.
 
Back
Top