Numeric precision

  • Thread starter Thread starter Edwin E. Smith
  • Start date Start date
E

Edwin E. Smith

I'm just learning C# so for practice, I decided to try and duplicate some of
the functionality of the Microsoft Calculator from scratch.

I used double precision (64 bit) types I.E.

private double window;

but I find that it doesn't count as high as Microsoft Calculator. I
eventuallu get to a point where the window displays "infinity" but the
Microsoft Calculator seems to go on forever displaying numbers with an
exponent.

I tried using decimal types which should give me 128 bits of precision but I
cet a compile error with the following:

private decimal window = 4;

window = Math.Sqrt(window);

// compiler error The best overloaded method match for
'System.Math.Sqrt(double)' has some invalid arguments

I assume that the decimal type is for money only and won't do what I want.
So how can I cet more precision?

Edwin
 
Edwin,
// compiler error The best overloaded method match for
'System.Math.Sqrt(double)' has some invalid arguments
Click to expand...

You need casts in order for the code to compile:

window = (decimal) Math.Sqrt((double) window);

This won't give you the precision you need.
I assume that the decimal type is for money only and won't do what I want.
So how can I get more precision?
Click to expand...

There's no implementation of Sqrt that works on a decimal argument. You
could build your own using Newton-Raphson method for instance.

Regards - Octavio
 
Back
Top