Bug in double precision computing?

  • Thread starter Thread starter Daniel Geisenhoff
  • Start date Start date
D

Daniel Geisenhoff

Hi all,

can anyone explain to me why cdbl(2.8+2.16+2.85) does result in
7.81000000005 instead of 7.81 ?

Our configuration:

Win2k Server SP4
PIII-550
Dot Net Framework 1.1
VS 2003

Many thanks in advance.
Daniel.
 
Use the decimal data type. Otherwise you'll get binary rounding, typical
with any real data types. See the Decimal MSDN reference.
Shan
 
You must log in or register to reply here.

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

This is definitely a bug in IIS 6.0 and DotNet Framework 2.0!! 8
IIS not running ASP.NET error!!! 1
Difference in double precision... 2
Error creating new ASP.NET application 8
Weird bug with Math.Floor 1
Very big bug in .NET and Intellisense (VC++)!!! 3
Still can't debug. ARGH! 2
Fix for DataTable.Select bug 3

Back
Top