PC Review


Reply
Thread Tools Rate Thread

Floating point division

 
 
Tom C
Guest
Posts: n/a
 
      25th Jan 2012
The following result is from the immediate window in vs2005 just to
illustrate what I am seeing in code. Quite obviously, .08 / .1 does
not equal 0.79999999999999993. What is going on here?

? 8/1
8.0
? .8/1
0.8
? .8/.1
8.0
? .08/.1
0.79999999999999993
? .08/.01
8.0
? .008/.001
8.0
? .008/1
0.008
? .008/.01
0.8
? .008/.1
0.08
? .08/.1
 
Reply With Quote
 
 
 
 
ralph
Guest
Posts: n/a
 
      25th Jan 2012
On Wed, 25 Jan 2012 12:21:40 -0800 (PST), Tom C
<(E-Mail Removed)> wrote:

>The following result is from the immediate window in vs2005 just to
>illustrate what I am seeing in code. Quite obviously, .08 / .1 does
>not equal 0.79999999999999993. What is going on here?
>


Every beginnig developer gets burned by this sooner or later.

http://floating-point-gui.de/

-ralph
 
Reply With Quote
 
 
 
 
Tom C
Guest
Posts: n/a
 
      25th Jan 2012
On Jan 25, 2:49*pm, ralph <(E-Mail Removed)> wrote:
> On Wed, 25 Jan 2012 12:21:40 -0800 (PST), Tom C
>
> <(E-Mail Removed)> wrote:
> >The following result is from the immediate window in vs2005 just to
> >illustrate what I am seeing in code. Quite obviously, .08 / .1 does
> >not equal 0.79999999999999993. What is going on here?

>
> Every beginnig developer gets burned by this sooner or later.
>
> http://floating-point-gui.de/
>
> -ralph


I have probably been programming longer than you are old but aside
from that, I have never seen this and I am looking for the cliff notes
answer not a college degree in math so please don't reply with another
antagonistic comment and link.

thanks
 
Reply With Quote
 
Gene Wirchenko
Guest
Posts: n/a
 
      26th Jan 2012
On Wed, 25 Jan 2012 13:25:33 -0800 (PST), Tom C
<(E-Mail Removed)> wrote:

>On Jan 25, 2:49*pm, ralph <(E-Mail Removed)> wrote:
>> On Wed, 25 Jan 2012 12:21:40 -0800 (PST), Tom C
>>
>> <(E-Mail Removed)> wrote:
>> >The following result is from the immediate window in vs2005 just to
>> >illustrate what I am seeing in code. Quite obviously, .08 / .1 does
>> >not equal 0.79999999999999993. What is going on here?

>>
>> Every beginnig developer gets burned by this sooner or later.
>>
>> http://floating-point-gui.de/
>>
>> -ralph

>
>I have probably been programming longer than you are old but aside
>from that, I have never seen this and I am looking for the cliff notes
>answer not a college degree in math so please don't reply with another
>antagonistic comment and link.


Well, now, you have something to learn about floating-point
arithmetic. The site mentioned above appears to be a good
introduction to the issue.

The antagonism is actually yours. ralph's answer was very good.

You have some reading to do.

Sincerely,

Gene Wirchenko
 
Reply With Quote
 
ralph
Guest
Posts: n/a
 
      26th Jan 2012
On Wed, 25 Jan 2012 13:25:33 -0800 (PST), Tom C
<(E-Mail Removed)> wrote:

>On Jan 25, 2:49*pm, ralph <(E-Mail Removed)> wrote:
>> On Wed, 25 Jan 2012 12:21:40 -0800 (PST), Tom C
>>
>> <(E-Mail Removed)> wrote:
>> >The following result is from the immediate window in vs2005 just to
>> >illustrate what I am seeing in code. Quite obviously, .08 / .1 does
>> >not equal 0.79999999999999993. What is going on here?

>>
>> Every beginnig developer gets burned by this sooner or later.
>>
>> http://floating-point-gui.de/
>>
>> -ralph

>
>I have probably been programming longer than you are old but aside
>from that, I have never seen this and I am looking for the cliff notes
>answer not a college degree in math so please don't reply with another
>antagonistic comment and link.
>
>thanks


I apologize if I ruffled your feathers. Not intended, just wanted to
let you know this is a common issue EVERYONE runs into sooner or
later. Surprised if you have been programming for more years than I
(34 years) you haven't encountered this before.

I did provide the 'Cliff Notes' answer. If you want the real
nitty-gritty go here. (PS: Note the title.)

"What Every Computer Scientist Should Know About Floating-Point
Arithmetic"
http://docs.oracle.com/cd/E19957-01/..._goldberg.html

-ralph
 
Reply With Quote
 
Auric__
Guest
Posts: n/a
 
      26th Jan 2012
Tom C wrote:

> On Jan 25, 2:49*pm, ralph <(E-Mail Removed)> wrote:
>> On Wed, 25 Jan 2012 12:21:40 -0800 (PST), Tom C
>>
>> <(E-Mail Removed)> wrote:
>> >The following result is from the immediate window in vs2005 just to
>> >illustrate what I am seeing in code. Quite obviously, .08 / .1 does
>> >not equal 0.79999999999999993. What is going on here?

>>
>> Every beginnig developer gets burned by this sooner or later.
>>
>> http://floating-point-gui.de/
>>
>> -ralph

>
> I have probably been programming longer than you are old but aside
> from that, I have never seen this and I am looking for the cliff notes
> answer not a college degree in math so please don't reply with another
> antagonistic comment and link.


I've been programming for close to 30 years and I *still* get burned by fp
problems on occasion.

--
DIVINARUM RERUM SCRIPTOR
 
Reply With Quote
 
ralph
Guest
Posts: n/a
 
      27th Jan 2012
On Thu, 26 Jan 2012 02:34:52 +0000 (UTC), "Auric__"
<(E-Mail Removed)> wrote:

>Tom C wrote:
>
>> On Jan 25, 2:49*pm, ralph <(E-Mail Removed)> wrote:
>>> On Wed, 25 Jan 2012 12:21:40 -0800 (PST), Tom C
>>>
>>> <(E-Mail Removed)> wrote:
>>> >The following result is from the immediate window in vs2005 just to
>>> >illustrate what I am seeing in code. Quite obviously, .08 / .1 does
>>> >not equal 0.79999999999999993. What is going on here?
>>>
>>> Every beginnig developer gets burned by this sooner or later.
>>>
>>> http://floating-point-gui.de/
>>>
>>> -ralph

>>
>> I have probably been programming longer than you are old but aside
>> from that, I have never seen this and I am looking for the cliff notes
>> answer not a college degree in math so please don't reply with another
>> antagonistic comment and link.

>
>I've been programming for close to 30 years and I *still* get burned by fp
>problems on occasion.


LOL

Me too!
 
Reply With Quote
 
The Mad Ape
Guest
Posts: n/a
 
      30th Jan 2012
On 25/01/2012 9:31 PM, ralph wrote:
> On Wed, 25 Jan 2012 13:25:33 -0800 (PST), Tom C
> <(E-Mail Removed)> wrote:
>
>> On Jan 25, 2:49 pm, ralph<(E-Mail Removed)> wrote:
>>> On Wed, 25 Jan 2012 12:21:40 -0800 (PST), Tom C
>>>
>>> <(E-Mail Removed)> wrote:
>>>> The following result is from the immediate window in vs2005 just to
>>>> illustrate what I am seeing in code. Quite obviously, .08 / .1 does
>>>> not equal 0.79999999999999993. What is going on here?
>>>
>>> Every beginnig developer gets burned by this sooner or later.
>>>
>>> http://floating-point-gui.de/
>>>
>>> -ralph

>>
>> I have probably been programming longer than you are old but aside
>>from that, I have never seen this and I am looking for the cliff notes
>> answer not a college degree in math so please don't reply with another
>> antagonistic comment and link.
>>
>> thanks

>
> I apologize if I ruffled your feathers. Not intended, just wanted to
> let you know this is a common issue EVERYONE runs into sooner or
> later. Surprised if you have been programming for more years than I
> (34 years) you haven't encountered this before.
>
> I did provide the 'Cliff Notes' answer. If you want the real
> nitty-gritty go here. (PS: Note the title.)
>
> "What Every Computer Scientist Should Know About Floating-Point
> Arithmetic"
> http://docs.oracle.com/cd/E19957-01/..._goldberg.html
>
> -ralph


That hurt my head when I read it. The other article was more on my
level. Fluoride in the water must have dumbed me down

Good to know that I have been doing something right...just didn't knw it
until now.

TMA
 
Reply With Quote
 
 
 
Reply

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Floating point division Tom C Microsoft VB .NET 0 25th Jan 2012 09:22 PM
division in access to round-up if division not whole number ddiel Microsoft Access VBA Modules 10 30th Dec 2009 01:59 AM
Vista speech results in Floating point division by zero exception =?Utf-8?B?RGF2eQ==?= Windows Vista General Discussion 0 27th Feb 2007 11:10 AM
is there a division symbol. As in a maths division symbol =?Utf-8?B?Q3VwQ2FrZQ==?= Microsoft Word Document Management 3 15th Nov 2005 01:38 AM
Passmark Performance Test, Division, Floating Point Division, 2DShapes @(none) Computer Hardware 0 19th Aug 2004 11:57 PM


Features
 

Advertising
 

Newsgroups
 


All times are GMT +1. The time now is 02:01 PM.