Mathmatical Errors

  • Thread starter Thread starter Scott Duncan
  • Start date Start date
S

Scott Duncan

We are working off data in a form...

Basic Question, how can 566.3 - 565.5 = 0.7999878 ???

Underlying Tables are Singles and Form Control Format = "General Number"

Any insight would greatly be appreciated !

SD
 
Scott

Precision.

If you're using single or double precision numbers, Access has to use binary
representations. This introduces "rounding errors".

Consider changing the data type of the underlying fields to Currency (four
places) if you don't need that much precision.
 
Jeff,

Thank you, but all of the Data Types are the same, Single Precision. Seems
like a lot extra code to ensure
a predictive result.

SD
 
Scott said:
Jeff,

Thank you, but all of the Data Types are the same, Single Precision.
Seems like a lot extra code to ensure
a predictive result.

SD
Click to expand...

Necessary.
 
Jeff,

Thank you, but all of the Data Types are the same, Single Precision.
Click to expand...

Jeff's exactly correct.

The problem is that a Single Precision number is, BY ITS VERY NATURE,
an approximation. It's stored as a 24-bit "mantissa", a binary
fraction, and an 8-bit "exponent"; the number is calculated by
multiplying the mantissa by two to the power of the exponent.

Just as it is impossible to accurately represent some numbers - 1/7
say - as a decimal fraction, it is impossible to represent some
numbers - 1/10 for example - as a binary fraction. What is stored is a
24-bit truncated approximation, just as if I were to tell you that 1/7
is equal to 0.1428561 - that's a good approximation but it's off by
0.000000042856142856142856<etcetera>.

Access CURRENCY datatype fields are not stored in this way; instead,
they're stored as a scaled huge integer with exactly four decimal
places.

Note that *all* floating point numbers, in Excel, Fortran, C,
whatever, share this same limitation.
 
Back
Top