Floating Point

Discussion in 'Microsoft Excel Discussion' started by Kathleen Bell, Jan 9, 2006.

  1. Years ago I heard about the way Excel stores numbers. I wrote it off as
    something accountants would never have to deal with since we weren't dealing
    with high precision numbers. Well...

    This past week I tried to upload a journal entry that had about 15K lines.
    The computer rejected it as not balancing. In the ninth and tenth decimal
    places there was a 15. We tried everything. It became a "class project".
    We tried rounding up, rounding down, just plain rounding, truncating. We
    copied as values, copied as text and then recopied as numbers. Finally we
    found a website that talked about the floating point, something which
    something that sort of came out of the recesses of my memory. There was no
    other explanation.

    I split the journal entry into three and somehow the new combination of
    numbers got rid of the phantom number way out there and they went through.
     
    Kathleen Bell, Jan 9, 2006
    #1
    1. Advertisements

  2. Kathleen Bell

    JE McGimpsey Guest

    take a look here:

    http://www.mcgimpsey.com/excel/pennyoff.html

    In article <>,
    "Kathleen Bell" <> wrote:

    > Years ago I heard about the way Excel stores numbers. I wrote it off as
    > something accountants would never have to deal with since we weren't dealing
    > with high precision numbers. Well...
    >
    > This past week I tried to upload a journal entry that had about 15K lines.
    > The computer rejected it as not balancing. In the ninth and tenth decimal
    > places there was a 15. We tried everything. It became a "class project".
    > We tried rounding up, rounding down, just plain rounding, truncating. We
    > copied as values, copied as text and then recopied as numbers. Finally we
    > found a website that talked about the floating point, something which
    > something that sort of came out of the recesses of my memory. There was no
    > other explanation.
    >
    > I split the journal entry into three and somehow the new combination of
    > numbers got rid of the phantom number way out there and they went through.
     
    JE McGimpsey, Jan 9, 2006
    #2
    1. Advertisements

  3. Kathleen Bell

    Guest Guest

    "Kathleen Bell" wrote:
    > Finally we found a website that talked about the
    > floating point, something which something that sort
    > of came out of the recesses of my memory. There
    > was no other explanation.
    >
    > I split the journal entry into three and somehow the
    > new combination of numbers got rid of the phantom
    > number way out there and they went through.


    Do you have a question? Perhaps it would help to read
    the following KB articles:

    http://support.microsoft.com/kb/214118/en-us
    http://support.microsoft.com/kb/78113/en-us
    http://support.microsoft.com/kb/57298/en-us
     
    Guest, Jan 10, 2006
    #3
    1. Advertisements

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dave Peterson

    Re: Floating Point error while merging data from Excel to Word?

    Dave Peterson, Dec 3, 2003, in forum: Microsoft Excel Discussion
    Replies:
    0
    Views:
    493
    Dave Peterson
    Dec 3, 2003
  2. Debra Dalgleish

    Re: Floating Point error while merging data from Excel to Word?

    Debra Dalgleish, Dec 3, 2003, in forum: Microsoft Excel Discussion
    Replies:
    1
    Views:
    245
  3. Don Guillett

    Re: Floating Point error while merging data from Excel to Word?

    Don Guillett, Dec 3, 2003, in forum: Microsoft Excel Discussion
    Replies:
    1
    Views:
    194
  4. Floating CommandButton

    , Feb 6, 2004, in forum: Microsoft Excel Discussion
    Replies:
    4
    Views:
    203
  5. Printing floating point numbers

    , Mar 1, 2005, in forum: Microsoft Excel Discussion
    Replies:
    4
    Views:
    190
    Ron Rosenfeld
    Mar 2, 2005
Loading...

Share This Page