PC Review


Reply
Thread Tools Rate Thread

Calculation/Number Bug in .xls? Any ideas?

 
 
AB
Guest
Posts: n/a
 
      24th Jan 2008
Hi,

I just stumbled across this problem - i can sort it out using a
rounding calculation but anyway i was wondering if anyone had a clue
why it's happening.

so, could you please let me know why this code fires "Not match"?

if (329970.14 + 1012000) = 1341970.14 then
msgbox "Match"
else
msgbox "Not match"
End if

At the same time if i type in a cell
=(329970.14 + 1012000) = 1341970.14
it returns TRUE...

Any idea?
 
Reply With Quote
 
 
 
 
Nigel
Guest
Posts: n/a
 
      24th Jan 2008
Check out.....Floating point numbers

http://support.microsoft.com/kb/214118

--

Regards,
Nigel
(E-Mail Removed)



"AB" <(E-Mail Removed)> wrote in message
news:d137e904-0967-41d3-a7d8-(E-Mail Removed)...
> Hi,
>
> I just stumbled across this problem - i can sort it out using a
> rounding calculation but anyway i was wondering if anyone had a clue
> why it's happening.
>
> so, could you please let me know why this code fires "Not match"?
>
> if (329970.14 + 1012000) = 1341970.14 then
> msgbox "Match"
> else
> msgbox "Not match"
> End if
>
> At the same time if i type in a cell
> =(329970.14 + 1012000) = 1341970.14
> it returns TRUE...
>
> Any idea?


 
Reply With Quote
 
Charles Williams
Guest
Posts: n/a
 
      24th Jan 2008
Because binary floating point numbers (which is what Excel and VBA use) do
not correspond exactly to decimal, this kind of test is very unsafe.

You should use something like this instead
if abs((329970.14 + 1012000) - 1341970.14)<0.000001 then

Or use Currency datatype which will give greater accuracy for up to 2
decimal places
if (CCur(329970.14) + CCur(1012000)) = CCur(1341970.14) then

Excel tries to be cleverer, but it does not always work
=(329970.14+1012000)-1341970.14 returns zero, but
=((329970.14+1012000)-1341970.14) returns a very small number


Charles
__________________________________________________
Outlines for my Sessions at the Australia Excel Users Group
http://www.decisionmodels.com/OZEUC.htm

"AB" <(E-Mail Removed)> wrote in message
news:d137e904-0967-41d3-a7d8-(E-Mail Removed)...
> Hi,
>
> I just stumbled across this problem - i can sort it out using a
> rounding calculation but anyway i was wondering if anyone had a clue
> why it's happening.
>
> so, could you please let me know why this code fires "Not match"?
>
> if (329970.14 + 1012000) = 1341970.14 then
> msgbox "Match"
> else
> msgbox "Not match"
> End if
>
> At the same time if i type in a cell
> =(329970.14 + 1012000) = 1341970.14
> it returns TRUE...
>
> Any idea?



 
Reply With Quote
 
AB
Guest
Posts: n/a
 
      24th Jan 2008
it just proves what a newbie I am...

It's very good to know - thanks a lot Nigel/Charles!!
 
Reply With Quote
 
Dana DeLouis
Guest
Posts: n/a
 
      24th Jan 2008
Just another option is the Currency symbol @:

Sub Demo()
If (329970.14@ + 1012000) = 1341970.14 Then
MsgBox "Match"
Else
MsgBox "Not match"
End If
End Sub

--
Dana DeLouis

"AB" <(E-Mail Removed)> wrote in message
news:f5135ec7-df2c-4329-826b-(E-Mail Removed)...
> it just proves what a newbie I am...
>
> It's very good to know - thanks a lot Nigel/Charles!!


 
Reply With Quote
 
 
 
Reply

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
help with number calculation =?Utf-8?B?aGF0ZW1tYQ==?= Microsoft Access Queries 3 22nd Apr 2007 09:26 PM
PMT calculation is negative number Edward Owen Microsoft Excel Discussion 3 25th Mar 2007 09:01 PM
Number Calculation =?Utf-8?B?S1JEaXRjaA==?= Microsoft Access 2 27th Jul 2006 07:24 PM
Auto number skipped 1000s of #s any ideas why or how to fix =?Utf-8?B?RE1hcmllRA==?= Microsoft Access Queries 3 2nd Jun 2006 10:17 PM
Report with unknown number of varying sections. Ideas? Larry Microsoft Access Reports 8 27th Jan 2004 02:04 AM


Features
 

Advertising
 

Newsgroups
 


All times are GMT +1. The time now is 11:00 AM.