Floating point division

[Tom C]

The following result is from the immediate window in vs2005 just to
illustrate what I am seeing in code. Quite obviously, .08 / .1 does
not equal 0.79999999999999993. What is going on here?

? 8/1
8.0
? .8/1
0.8
? .8/.1
8.0
? .08/.1
0.79999999999999993
? .08/.01
8.0
? .008/.001
8.0
? .008/1
0.008
? .008/.01
0.8
? .008/.1
0.08
? .08/.1

 

[ralph]

The following result is from the immediate window in vs2005 just to
illustrate what I am seeing in code. Quite obviously, .08 / .1 does
not equal 0.79999999999999993. What is going on here?

Every beginnig developer gets burned by this sooner or later.

http://floating-point-gui.de/

-ralph

 

[Tom C]

Every beginnig developer gets burned by this sooner or later.

http://floating-point-gui.de/

-ralph

I have probably been programming longer than you are old but aside
from that, I have never seen this and I am looking for the cliff notes
answer not a college degree in math so please don't reply with another
antagonistic comment and link.

thanks

 

[Gene Wirchenko]

I have probably been programming longer than you are old but aside
from that, I have never seen this and I am looking for the cliff notes
answer not a college degree in math so please don't reply with another
antagonistic comment and link.

Well, now, you have something to learn about floating-point
arithmetic. The site mentioned above appears to be a good
introduction to the issue.

The antagonism is actually yours. ralph's answer was very good.

You have some reading to do.

Sincerely,

Gene Wirchenko

 

[ralph]

I have probably been programming longer than you are old but aside
from that, I have never seen this and I am looking for the cliff notes
answer not a college degree in math so please don't reply with another
antagonistic comment and link.

thanks

I apologize if I ruffled your feathers. Not intended, just wanted to
let you know this is a common issue EVERYONE runs into sooner or
later. Surprised if you have been programming for more years than I
(34 years) you haven't encountered this before.

I did provide the 'Cliff Notes' answer. If you want the real
nitty-gritty go here. (PS: Note the title.)

"What Every Computer Scientist Should Know About Floating-Point
Arithmetic"
http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

-ralph

 

[Auric__]

I have probably been programming longer than you are old but aside
from that, I have never seen this and I am looking for the cliff notes
answer not a college degree in math so please don't reply with another
antagonistic comment and link.

I've been programming for close to 30 years and I *still* get burned by fp
problems on occasion.

 

[ralph]

I've been programming for close to 30 years and I *still* get burned by fp
problems on occasion.

LOL

Me too!

 

[The Mad Ape]

I apologize if I ruffled your feathers. Not intended, just wanted to
let you know this is a common issue EVERYONE runs into sooner or
later. Surprised if you have been programming for more years than I
(34 years) you haven't encountered this before.

I did provide the 'Cliff Notes' answer. If you want the real
nitty-gritty go here. (PS: Note the title.)

"What Every Computer Scientist Should Know About Floating-Point
Arithmetic"
http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

-ralph

That hurt my head when I read it. The other article was more on my
level. Fluoride in the water must have dumbed me down

Good to know that I have been doing something right...just didn't knw it
until now.

TMA