Excel Calculation Glitch

G

Guest

111,111,111 x 111,111,111 = 12,345,678,987,654,321

Excel 2003 calculates as
111,111,111 x 111,111,111 = 12,345,678,987,654,300


----------------
This post is a suggestion for Microsoft, and Microsoft responds to the
suggestions with the most votes. To vote for this suggestion, click the "I
Agree" button in the message pane. If you do not see the button, follow this
link to open the suggestion in the Microsoft Web-based Newsreader and then
click "I Agree" in the message pane.

http://www.microsoft.com/office/com...74aaf97&dg=microsoft.public.excel.crashesgpfs
 
G

Guest

What you have stated is correct.
Excel can handle only a limited number of digits.
To perform the multiplication with full accuracy, pull-down:

Start > Programs > Accessories > Calculator and do the math.
 
G

Guest

Excel (and most other computer software follows the IEEE standard for double
precision binary representation of numbers. As a result, Excel is clearly
documented (see Help for "specifications" to only give 15 significant digits,
which it did in the OP's example.

The Windows calculator is one of a small minority of programs that do not
follow the IEEE standard. It has sufficient precision to handle the OP's
example, but it too has limited precision.

Jerry
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top