PC Review


Reply
Thread Tools Rate Thread

decimal values create inequality

 
 
Sam
Guest
Posts: n/a
 
      23rd Feb 2009
I have an error-checking formula in a spreadsheet:
"If(Sum(A1:A30)<>D1,"ERROR","Balanced"), where A1:A30 are values that are
entered from a source document, while D1 is the Total listed on the same
source document. The purpose is to ensure that all component parts are
correctly entered. All values are formatted as #,##0.00.

There are random times whereby the sum of A1:A30 results in a number such as
25.0000000001 which does not equal the Total of 25 that had been entered
into D1. Since the above formatting displays 25 for both values, the only way
of combatting the issue is to use a Round function for the Sum.

How can this happen when there is no division involved and all components
are entered as dollars and cents?

Excel version 2003.

Your help is greatly appreciated,

Sam
 
Reply With Quote
 
 
 
 
Fred Smith
Guest
Posts: n/a
 
      23rd Feb 2009
This is common with computers. They work in binary, we work in decimal. The
common solution is to round your numbers to the precision required -- in
your case 2 decimal places. So use:
=If(round(Sum(A1:A30),2)<>D1,"ERROR","Balanced")

Regards,
Fred.

"Sam" <(E-Mail Removed)> wrote in message
news:77CB48B8-BB5B-4AD5-97B2-(E-Mail Removed)...
>I have an error-checking formula in a spreadsheet:
> "If(Sum(A1:A30)<>D1,"ERROR","Balanced"), where A1:A30 are values that are
> entered from a source document, while D1 is the Total listed on the same
> source document. The purpose is to ensure that all component parts are
> correctly entered. All values are formatted as #,##0.00.
>
> There are random times whereby the sum of A1:A30 results in a number such
> as
> 25.0000000001 which does not equal the Total of 25 that had been entered
> into D1. Since the above formatting displays 25 for both values, the only
> way
> of combatting the issue is to use a Round function for the Sum.
>
> How can this happen when there is no division involved and all components
> are entered as dollars and cents?
>
> Excel version 2003.
>
> Your help is greatly appreciated,
>
> Sam


 
Reply With Quote
 
David Biddulph
Guest
Posts: n/a
 
      23rd Feb 2009
If your number of 25.00 is formed by adding 12.40 to 12.60, you can find the
source of the problem by calculating the *exact* binary representation of
0.40, for example.
Come back to us when you've done it. :-)
--
David Biddulph


"Sam" <(E-Mail Removed)> wrote in message
news:77CB48B8-BB5B-4AD5-97B2-(E-Mail Removed)...
>I have an error-checking formula in a spreadsheet:
> "If(Sum(A1:A30)<>D1,"ERROR","Balanced"), where A1:A30 are values that are
> entered from a source document, while D1 is the Total listed on the same
> source document. The purpose is to ensure that all component parts are
> correctly entered. All values are formatted as #,##0.00.
>
> There are random times whereby the sum of A1:A30 results in a number such
> as
> 25.0000000001 which does not equal the Total of 25 that had been entered
> into D1. Since the above formatting displays 25 for both values, the only
> way
> of combatting the issue is to use a Round function for the Sum.
>
> How can this happen when there is no division involved and all components
> are entered as dollars and cents?
>
> Excel version 2003.
>
> Your help is greatly appreciated,
>
> Sam



 
Reply With Quote
 
Shane Devenshire
Guest
Posts: n/a
 
      23rd Feb 2009
Hi,

Computers work in binary, we work in decimals which results in
approximations by Excel and any computer.

Here is everything you need to know about this issue (and more):

http://support.microsoft.com/kb/78113/en-us
http://support.microsoft.com/kb/42980
http://support.microsoft.com/kb/214118
http://www.cpearson.com/excel/rounding.htm
http://docs.sun.com/source/806-3568/ncg_goldberg.html

there are innumerable solutions but you alreay have one.

--
If this helps, please click the Yes button

Cheers,
Shane Devenshire


"Sam" wrote:

> I have an error-checking formula in a spreadsheet:
> "If(Sum(A1:A30)<>D1,"ERROR","Balanced"), where A1:A30 are values that are
> entered from a source document, while D1 is the Total listed on the same
> source document. The purpose is to ensure that all component parts are
> correctly entered. All values are formatted as #,##0.00.
>
> There are random times whereby the sum of A1:A30 results in a number such as
> 25.0000000001 which does not equal the Total of 25 that had been entered
> into D1. Since the above formatting displays 25 for both values, the only way
> of combatting the issue is to use a Round function for the Sum.
>
> How can this happen when there is no division involved and all components
> are entered as dollars and cents?
>
> Excel version 2003.
>
> Your help is greatly appreciated,
>
> Sam

 
Reply With Quote
 
Sam
Guest
Posts: n/a
 
      23rd Feb 2009
Thanks for your response David. I'm not sure what you mean by "calculating
the exact binary representation of .40". This formula has produced a
"balanced" flag every day for over 7 months and now found the inequality.
Actually, the numbers that are being summed are: -8.55, .16, 161.00,
-150.00. The actual sum of these numbers is 2.61. When the user entered 2.61
in the check total field the formula matching the two numbers showed an
inequality because the sum function produced a value of 2.61000000001 which
was not apparent because it was formatted as 2 decimal places.

I have seen this happen on rare occasions before. There seems to be no
apparent cause. It doesn't seem reasonable that I should have to use the
Round function every time I want to compare the sum of components to their
total.

"David Biddulph" wrote:

> If your number of 25.00 is formed by adding 12.40 to 12.60, you can find the
> source of the problem by calculating the *exact* binary representation of
> 0.40, for example.
> Come back to us when you've done it. :-)
> --
> David Biddulph
>
>
> "Sam" <(E-Mail Removed)> wrote in message
> news:77CB48B8-BB5B-4AD5-97B2-(E-Mail Removed)...
> >I have an error-checking formula in a spreadsheet:
> > "If(Sum(A1:A30)<>D1,"ERROR","Balanced"), where A1:A30 are values that are
> > entered from a source document, while D1 is the Total listed on the same
> > source document. The purpose is to ensure that all component parts are
> > correctly entered. All values are formatted as #,##0.00.
> >
> > There are random times whereby the sum of A1:A30 results in a number such
> > as
> > 25.0000000001 which does not equal the Total of 25 that had been entered
> > into D1. Since the above formatting displays 25 for both values, the only
> > way
> > of combatting the issue is to use a Round function for the Sum.
> >
> > How can this happen when there is no division involved and all components
> > are entered as dollars and cents?
> >
> > Excel version 2003.
> >
> > Your help is greatly appreciated,
> >
> > Sam

>
>
>

 
Reply With Quote
 
joeu2004
Guest
Posts: n/a
 
      24th Feb 2009
On Feb 23, 12:08 pm, Sam <S...@discussions.microsoft.com> wrote:
> It doesn't seem reasonable that I should have to use
> the Round function every time I want to compare the
> sum of components to their total.


But unfortunately, you do. Well, it is either that or one of two
other approaches, each of which has its detractors. One alternative:
never compare for equality; for example, IF(ABS(SUM(A1:A30)-D1) >=
0.005,"ERROR",...). Another alternative: use the "Precision as
displayed" option under Tools > Options > Calculation.


> Actually, the numbers that are being summed are:
> -8.55, .16, 161.00, -150.00. The actual sum of these
> numbers is 2.61.


What people have been trying to explain to you is: -8.55 and 0.16
cannot be represented exactly in the internal binary representation
used by Excel (and most applications). Instead, they are
-8.55000000000000,0710542735760100185871124267578125 and
0.160000000000000,0033306690738754696212708950042724609375. (The
comma demarcates 15 significant digits to the left.)

But that is not the full story. The rest of the explanation goes far
beyond this tutorial in binary representation. In a nutshell, it has
to do with the magnitude of the pairwise numbers and intermediate
results involved in the computation. For example, whereas SUM
(-8.55,0.16,161,-150) does not exactly equal the internal
representation of 2.61, SUM(-8.55,0.16)+SUM(161,-150) -- aka
(-8.55+0.16)+(161-150) -- is close enough to 2.61 [1] that Excel
considers them to be equal.

The point here is: you can find many examples where you get away with
not rounding before comparison; but you can also find many examples
where you do not get away without it. This mystifies almost everyone;
even those of us who understand the computer science of it get bitten
by this from time to time.

HTH.


Endnotes:

[1] 2.61 is 2.60999999999999,9875655021241982467472553253173828125 .
(-8.55+0.16)+(161-150) is
2.60999999999999,94315658113919198513031005859375 .



----- original posting -----


On Feb 23, 12:08*pm, Sam <S...@discussions.microsoft.com> wrote:
> Thanks for your response David. I'm not sure what you mean by "calculating
> the exact binary representation of .40". This formula has produced a
> "balanced" flag every day for over 7 months and now found the inequality.
> Actually, the numbers that are being summed are: *-8.55, .16, 161.00,
> -150.00. The actual sum of these numbers is 2.61. When the user entered 2..61
> in the check total field the formula matching the two numbers showed an
> inequality because the sum function produced a value of 2.61000000001 which
> was not apparent because it was formatted as 2 decimal places.
>
> I have seen this happen on rare occasions before. There seems to be no
> apparent cause. It doesn't seem reasonable that I should have to use the
> Round function every time I want to compare the sum of components to their
> total.
>
>
>
> "David Biddulph" wrote:
> > If your number of 25.00 is formed by adding 12.40 to 12.60, you can find the
> > source of the problem by calculating the *exact* binary representation of
> > 0.40, for example.
> > Come back to us when you've done it. *:-)
> > --
> > David Biddulph

>
> > "Sam" <S...@discussions.microsoft.com> wrote in message
> >news:77CB48B8-BB5B-4AD5-97B2-(E-Mail Removed)...
> > >I have an error-checking formula in a spreadsheet:
> > > "If(Sum(A1:A30)<>D1,"ERROR","Balanced"), where A1:A30 are values thatare
> > > entered from a source document, while D1 is the Total listed on the same
> > > source document. The purpose is to ensure that all component parts are
> > > correctly entered. All values are formatted as #,##0.00.

>
> > > There are random times whereby the sum of A1:A30 results in a number such
> > > as
> > > 25.0000000001 which does not equal the Total of 25 that had been entered
> > > into D1. Since the above formatting displays 25 for both values, the only
> > > way
> > > of combatting the issue is to use a Round function for the Sum.

>
> > > How can this happen when there is no division involved and all components
> > > are entered as dollars and cents?

>
> > > Excel version 2003.

>
> > > Your help is greatly appreciated,

>
> > > Sam

 
Reply With Quote
 
Bernd P
Guest
Posts: n/a
 
      25th Feb 2009
Hello Sam,

I grew up with the saying "never compare a floating number to zero",
so I would suggest
"If(ABS(Sum(A1:A30)-D1)>1E-14,"ERROR","Balanced")

Regards,
Bernd
 
Reply With Quote
 
 
 
Reply

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: converting text values to number & decimal values.. John W. Vinson Microsoft Access Form Coding 2 15th Jan 2010 07:09 PM
Re: Graph two linear inequality equations Dana DeLouis Microsoft Excel Discussion 0 25th Aug 2008 02:51 AM
How do I write inequality? =?Utf-8?B?TWFyZ2FyZXQ=?= Microsoft Excel Worksheet Functions 11 5th Feb 2005 07:53 PM
Re: Inequality JulieD Microsoft Excel Worksheet Functions 0 26th Aug 2004 04:23 PM
convert hex-values to decimal values Markus Früh Microsoft Excel Programming 6 28th Aug 2003 10:32 AM


Features
 

Advertising
 

Newsgroups
 


All times are GMT +1. The time now is 01:31 PM.