PC Review


Reply
Thread Tools Rate Thread

Digit limits

 
 
firebird68
Guest
Posts: n/a
 
      13th Oct 2008
PRODUCT
Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
(2007-02-22)

QUESTIONS OR COMMENTS
Message: When multiplying 111,111,111 by itself, the Excel result is:
12345678987654300. The real answer when multiplying by hand is :
12345678987654321. Why does Excel round off the last two digits to zero?
Excel appears to rounds off to 15 digits. Why not 16 or more?
--
The Firebird Man
 
Reply With Quote
 
 
 
 
VanguardLH
Guest
Posts: n/a
 
      13th Oct 2008
firebird68 wrote:

> PRODUCT
> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
> (2007-02-22)
>
> QUESTIONS OR COMMENTS
> Message: When multiplying 111,111,111 by itself, the Excel result is:
> 12345678987654300. The real answer when multiplying by hand is :
> 12345678987654321. Why does Excel round off the last two digits to zero?
> Excel appears to rounds off to 15 digits. Why not 16 or more?


"excel" is not in the name of this newsgroup.

http://www.j-walk.com/SS/excel/usertips/tip032.htm
http://support.microsoft.com/kb/65903
http://excel.tips.net/Pages/T001983_..._in_Excel.html
http://precisioncalc.com/What_is_xlPrecision.html
 
Reply With Quote
 
Ken Blake, MVP
Guest
Posts: n/a
 
      13th Oct 2008
On Mon, 13 Oct 2008 04:49:01 -0700, firebird68
<(E-Mail Removed)> wrote:

> PRODUCT
> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
> (2007-02-22)
>
> QUESTIONS OR COMMENTS
> Message: When multiplying 111,111,111 by itself, the Excel result is:
> 12345678987654300. The real answer when multiplying by hand is :
> 12345678987654321. Why does Excel round off the last two digits to zero?
> Excel appears to rounds off to 15 digits. Why not 16 or more?



You're asking an Excel question in a Windows newsgroup. You are far
more likely to get the help you are looking for if you would ask in an
Excel newsgroup; that's where the Excel experts hang out.

--
Ken Blake, Microsoft MVP - Windows Desktop Experience
Please Reply to the Newsgroup
 
Reply With Quote
 
AlmostBob
Guest
Posts: n/a
 
      13th Oct 2008
http://www.smokeylake.com/excel/Data.htm
explains how excel stores numbers, and the precision lost after the 15th
(decimal) digit

--
Adaware http://www.lavasoft.de
spybot http://www.safer-networking.org
AVG free antivirus http://www.grisoft.com
Etrust/Vet/CA.online Antivirus scan
http://www3.ca.com/securityadvisor/virusinfo/scan.aspx
Panda online AntiVirus scan http://www.pandasoftware.com/ActiveScan/
Catalog of removal tools (1)
http://www.pandasoftware.com/download/utilities/
Catalog of removal tools (2)
http://www3.ca.com/securityadvisor/n...aspx?CID=40387
Blocking Unwanted Parasites with a Hosts file
http://mvps.org/winhelp2002/hosts.htm
links provided as a courtesy, read all instructions on the pages before use

Grateful thanks to the authors and webmasters
_

"firebird68" <(E-Mail Removed)> wrote in message
news:44915B4D-2BC9-4CD3-AC45-(E-Mail Removed)...
> PRODUCT
> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
> (2007-02-22)
>
> QUESTIONS OR COMMENTS
> Message: When multiplying 111,111,111 by itself, the Excel result is:
> 12345678987654300. The real answer when multiplying by hand is :
> 12345678987654321. Why does Excel round off the last two digits to zero?
> Excel appears to rounds off to 15 digits. Why not 16 or more?
> --
> The Firebird Man



 
Reply With Quote
 
RobertVA
Guest
Posts: n/a
 
      13th Oct 2008
firebird68 wrote:
> PRODUCT
> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
> (2007-02-22)
>
> QUESTIONS OR COMMENTS
> Message: When multiplying 111,111,111 by itself, the Excel result is:
> 12345678987654300. The real answer when multiplying by hand is :
> 12345678987654321. Why does Excel round off the last two digits to zero?
> Excel appears to rounds off to 15 digits. Why not 16 or more?


In most systems the memory available to store a quantity is limited.
When asked to represent large numbers many software applications store
the number as in a manner resembling scientific notation (like
1.23456789876543 X 10^14 BUT there's a significant possibility the
application uses a power of two instead of a power of ten). Since the
space to store the digits is limited, the number gets rounded off. If
the programmer thinks the matter is important enough the problem can be
eliminated, but that would probably at the expense of slower operation
and/or greatly enlarged storage requirements.

Note that for similar reasons there are numbers that cannot be precisely
stated as fractions. These irrational numbers include pi and the square
root of many prime numbers like two and three.
 
Reply With Quote
 
HeyBub
Guest
Posts: n/a
 
      13th Oct 2008
firebird68 wrote:
> PRODUCT
> Other;Office XP SP3 (2004-10-14) Compatibility pack for Office 2007
> (2007-02-22)
>
> QUESTIONS OR COMMENTS
> Message: When multiplying 111,111,111 by itself, the Excel result is:
> 12345678987654300. The real answer when multiplying by hand is :
> 12345678987654321. Why does Excel round off the last two digits to
> zero? Excel appears to rounds off to 15 digits. Why not 16 or more?


Excel uses IEEE double-precision floating point numbers. These numbers are
limited to about 15 significant digits.

As for 16 or more, a limit had to be set somewhere. If the developers had
coded extended precision arithmetic to, oh, say 75 digits of precision, sure
enough somebody would come along and say "why not 76?"



 
Reply With Quote
 
 
 
Reply

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Color a single digit in a mult-digit number cell =?Utf-8?B?UGh5bGxpcw==?= Microsoft Excel Misc 6 17th Nov 2005 12:46 AM
Excel: let me a color a single digit in a multi-digit cell =?Utf-8?B?UGh5bGxpcw==?= Microsoft Frontpage 6 13th Nov 2005 11:13 PM
When we enter a 16 digit number (credit card) the last digit chan. =?Utf-8?B?Y2VraW5n?= Microsoft Excel Misc 5 8th Dec 2004 11:45 PM
Sorting of mixed zip codes (5 digit and 9 digit) in ascending order =?Utf-8?B?Tk1fRnJlbmNobWFu?= Microsoft Excel Worksheet Functions 3 26th Jan 2004 09:08 PM
Change 2 digit suffix in 14 digit number in a column of numbers. Brad H. Microsoft Excel Worksheet Functions 1 4th Sep 2003 04:35 PM


Features
 

Advertising
 

Newsgroups
 


All times are GMT +1. The time now is 04:31 AM.