Digit limits

[firebird68]

PRODUCT
Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
(2007-02-22)

QUESTIONS OR COMMENTS
Message: When multiplying 111,111,111 by itself, the Excel result is:
12345678987654300. The real answer when multiplying by hand is :
12345678987654321. Why does Excel round off the last two digits to zero?
Excel appears to rounds off to 15 digits. Why not 16 or more?

 

[VanguardLH]

PRODUCT
Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
(2007-02-22)

QUESTIONS OR COMMENTS
Message: When multiplying 111,111,111 by itself, the Excel result is:
12345678987654300. The real answer when multiplying by hand is :
12345678987654321. Why does Excel round off the last two digits to zero?
Excel appears to rounds off to 15 digits. Why not 16 or more?

"excel" is not in the name of this newsgroup.

http://www.j-walk.com/SS/excel/usertips/tip032.htm
http://support.microsoft.com/kb/65903
http://excel.tips.net/Pages/T001983_Thoughts_and_Ideas_on_Significant_Digits_in_Excel.html
http://precisioncalc.com/What_is_xlPrecision.html

 

[Ken Blake, MVP]

PRODUCT
Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
(2007-02-22)

QUESTIONS OR COMMENTS
Message: When multiplying 111,111,111 by itself, the Excel result is:
12345678987654300. The real answer when multiplying by hand is :
12345678987654321. Why does Excel round off the last two digits to zero?
Excel appears to rounds off to 15 digits. Why not 16 or more?

You're asking an Excel question in a Windows newsgroup. You are far
more likely to get the help you are looking for if you would ask in an
Excel newsgroup; that's where the Excel experts hang out.

 

[AlmostBob]

http://www.smokeylake.com/excel/Data.htm
explains how excel stores numbers, and the precision lost after the 15th
(decimal) digit

--
Adaware http://www.lavasoft.de
spybot http://www.safer-networking.org
AVG free antivirus http://www.grisoft.com
Etrust/Vet/CA.online Antivirus scan
http://www3.ca.com/securityadvisor/virusinfo/scan.aspx
Panda online AntiVirus scan http://www.pandasoftware.com/ActiveScan/
Catalog of removal tools (1)
http://www.pandasoftware.com/download/utilities/
Catalog of removal tools (2)
http://www3.ca.com/securityadvisor/newsinfo/collateral.aspx?CID=40387
Blocking Unwanted Parasites with a Hosts file
http://mvps.org/winhelp2002/hosts.htm
links provided as a courtesy, read all instructions on the pages before use

Grateful thanks to the authors and webmasters
_

 

[RobertVA]

PRODUCT
Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
(2007-02-22)

QUESTIONS OR COMMENTS
Message: When multiplying 111,111,111 by itself, the Excel result is:
12345678987654300. The real answer when multiplying by hand is :
12345678987654321. Why does Excel round off the last two digits to zero?
Excel appears to rounds off to 15 digits. Why not 16 or more?

In most systems the memory available to store a quantity is limited.
When asked to represent large numbers many software applications store
the number as in a manner resembling scientific notation (like
1.23456789876543 X 10^14 BUT there's a significant possibility the
application uses a power of two instead of a power of ten). Since the
space to store the digits is limited, the number gets rounded off. If
the programmer thinks the matter is important enough the problem can be
eliminated, but that would probably at the expense of slower operation
and/or greatly enlarged storage requirements.

Note that for similar reasons there are numbers that cannot be precisely
stated as fractions. These irrational numbers include pi and the square
root of many prime numbers like two and three.

 

[HeyBub]

PRODUCT
Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
(2007-02-22)

QUESTIONS OR COMMENTS
Message: When multiplying 111,111,111 by itself, the Excel result is:
12345678987654300. The real answer when multiplying by hand is :
12345678987654321. Why does Excel round off the last two digits to
zero? Excel appears to rounds off to 15 digits. Why not 16 or more?

Excel uses IEEE double-precision floating point numbers. These numbers are
limited to about 15 significant digits.

As for 16 or more, a limit had to be set somewhere. If the developers had
coded extended precision arithmetic to, oh, say 75 digits of precision, sure
enough somebody would come along and say "why not 76?"