Weird Math Problem

  • Thread starter Thread starter nick
  • Start date Start date
N

nick

I'm having a really weird problem. Take a look at this screenshot.

http://users.aber.ac.uk/njb4/dotnetproblem1.jpg

Notice the 4 watches at the bottom, _totalRecords, MyDataGrid.PageSize
and TotalPages.Text.

Now, you'd think that if _totalRecords was 11, and MyDataGrid.PageSize
was 10... If you divide them, you get 1.1. If you Ceiling 1.1 them
you'd get two.

Unfortunately, it's somehow dividing 11 by 10, but getting 1.0. I have
established that it is actually changing the value, because if i set
the default value of _totalPages to say 0.5, the line with the division
operator will change it to 1.0.

Is this a bug in the math library? Or am I doing something blatantly
wrong?

Thanks!
 
Are your data types all floating point? You want to avoid truncation prior
to your Ceiling() call.

float ratio;
ratio = (float)numerator / (float)denominator
roundedup = Ceiling(ratio)

Tad
 
I'm having a really weird problem. Take a look at this screenshot.

http://users.aber.ac.uk/njb4/dotnetproblem1.jpg

Notice the 4 watches at the bottom, _totalRecords, MyDataGrid.PageSize
and TotalPages.Text.

Now, you'd think that if _totalRecords was 11, and MyDataGrid.PageSize
was 10... If you divide them, you get 1.1. If you Ceiling 1.1 them
you'd get two.
Click to expand...

Nope. Not when both _totalRecords and PageSize are ints. You'd get
integer arithmetic, as specified in the language spec.
Unfortunately, it's somehow dividing 11 by 10, but getting 1.0. I have
established that it is actually changing the value, because if i set
the default value of _totalPages to say 0.5, the line with the division
operator will change it to 1.0.

Is this a bug in the math library? Or am I doing something blatantly
wrong?
Click to expand...

Yup - you're not telling it to do floating point division. Put a cast
in one side or the other and you'll get floating point division:

_totalPages = _totalRecords / (double)MyDataGrid.PageSize;

(Note that you still won't get exactly 1.1, because that can't be
represented by a double, but that's a different story.)

Personally I'd stick with integer arithmetic, changing your code to:

int pageSize = MyDataGrid.PageSize;
int _totalPages = (totalRecords+pageSize-1)/pageSize;
 
Back
Top