Strange BUG in teh Framework?

[csmba]

thanks Jon, I appreciate your time. your answers are long and informative.

I understand everything you said and I am not claiming that anything should
have been different. the behavior which I could not understand was not the
fact that single and double are not a perfect representation, but the one
summarized in the next simple example:
Dim xSng As Single = 6547.972
Dim xDbl As Double = 6547.972

Dim d As Double = xDbl * CDbl(yInt)
Dim s As Single = xSng * CSng(yInt)

both d and s are calculated "correctly".
so I don't see how to explain this with your previous post: you say double
can not represent 6547.972 and neither can single (and that I was lucky that
a single gave me a right result). by why then would I get the correct answer
if I start from a double, and only face the problem of 6547.97216796875 when
it is a cast from single to double?

as for the business use in my case: I do not as you say need infinitely
precise numbers, all I need actually is 3 digits to the right precision.
this number represents the length in seconds of a file, and since it is a
8000 samples/sec, this is all the precision I need. I know I can probably
treat it all as a integer counting in some small unit such as milliseconds,
but the data I get is in float so transforming it to INT might have the same
problems has I have here...

hernan.

 

[Guest]

Further to the other replies if this is 'Really' a problem consider using a
Decimal instead of Double or Single

hth

guy

 

[Jay B. Harlow [MVP - Outlook]]

csmba,

all I need actually is 3 digits to the right precision.
this number represents the length in seconds of a file, and since it is a
8000 samples/sec, this is all the precision I need.

Consider using Decimal.

Dim x As Decimal = 6547.972D
Dim y As Integer = 8000

Dim d As Decimal = x * y

Hope this helps
Jay

 

[Jon Skeet [C# MVP]]

thanks Jon, I appreciate your time. your answers are long and informative.

So long as they're actually helping - there's nothing worse than
writing a long answer which only makes things *less* clear!

I understand everything you said and I am not claiming that anything should
have been different. the behavior which I could not understand was not the
fact that single and double are not a perfect representation, but the one
summarized in the next simple example:
Dim xSng As Single = 6547.972
Dim xDbl As Double = 6547.972

Dim d As Double = xDbl * CDbl(yInt)
Dim s As Single = xSng * CSng(yInt)

both d and s are calculated "correctly".
so I don't see how to explain this with your previous post: you say double
can not represent 6547.972 and neither can single (and that I was lucky that
a single gave me a right result). by why then would I get the correct answer
if I start from a double, and only face the problem of 6547.97216796875 when
it is a cast from single to double?

Just chance. The error in truncating the result of the multiplication
happened to go the other way from the error in the original
representation - two "wrongs" happened to make a "right".

as for the business use in my case: I do not as you say need infinitely
precise numbers, all I need actually is 3 digits to the right precision.
this number represents the length in seconds of a file, and since it is a
8000 samples/sec, this is all the precision I need. I know I can probably
treat it all as a integer counting in some small unit such as milliseconds,
but the data I get is in float so transforming it to INT might have the same
problems has I have here...

How are you getting it as a float though? From a database, or parsing
it from a file, or what?

So long as you don't do very many operations, you'll certainly keep 3
significant digits correctly, but you might want to consider using
decimal instead, as others have suggested.

 

[Kevin Gale]

No. The lesson is not to assume that floating point arithmetic will be
accurate (in the sense the OP expected) in the first case.

Yup, however it's rather sad that this fact that has been known since the
first days of computing needs to be rediscovered on a daily basis.

Yes grasshopper, water is wet and floating point math is not exact.
and yes I was the grasshopper myself once.

In my area of software the gotcha most programmers get bit by is assuming
you can calculate a duration by subtracting the previously saved time from
the current time. The problem of course is the clock can change for lots of
reasons beyond the application's control. So instead of turning on that
output for ten seconds it turns on for an hour and ten seconds when the
clock turns back because of daylight savings times. It seems every new
programmer makes this same mistake.

So floating point math is not exact and you can't calculate durations from a
time of day clock.

-- Kevin

-- Kevin

 

[csmba]

I know about Decimal, it is not the first choice becuase of performance.
can you comment on my other quesion? which was why this works:acording to what you said, the double version should not have worked... but
it does.
h.

 

[csmba]

well... I get it from a COM object (BLHaaA)
it is a c++ object, that loads a wav file to memory and I need to ask it
what is the length of the file. it returns a float.

Decimal just seems soooo slow. is there a trick to use a single but tell it
to keep only 3 points precision? in that case, I will not have a problem
when I multiply by 8000.

h.

 

[Jon Skeet [C# MVP]]

well... I get it from a COM object (BLHaaA)

Oh dear On the other hand, it means you've probably done the
floating point conversion already, so any attempts to make it
"accurate" again (by using Decimal) are likely to be fruitless. Maybe
I'm wrong though... it depends on how exactly the COM object is doing
things, I guess.

it is a c++ object, that loads a wav file to memory and I need to ask it
what is the length of the file. it returns a float.

That's the length of the file in seconds?

Decimal just seems soooo slow.

Well, it's significantly slower than float - but is this actually the
bottleneck in your app?

is there a trick to use a single but tell it to keep only 3 points
precision? in that case, I will not have a problem when I multiply by
8000.

No, but you could always round the float afterwards. I'm not sure how I
see why that's a good thing though, to be honest.

 

[csmba]

yes, this 6547.972 is the length in seconds of a wav file.
and because of the calculation I am using (8000 is the sampling rate), all
that is actually true is the .xxx digits. anything that the double adds to
it like the .xxx167... is just wrong. and was never there to begin with (the
number is a perfect factor of 8000).
so since you said that "actually" the number is stored as 6547.972164... if
I can ignore the digits behind the first 3 I get exactly what I want.... (I
can do it of course by multiplying by 1000, then flooring the number to a
int, then dividing by 1000 again)--> but if I end up storing it ina single,
and he again adds those random 167.. it will do no good.

Decimal is out of the question, I am doing hundred of thousands of such
calculation per file.

while you said it is luck that
dim d as double = 6547.972
works, I find it more then strange. it can not be luck since I tested many
different numbers, all work the same: they all work when set like above. but
some return "strange" values when cast from single to double.
it is as if the problem is in the casting.

as for rounding the float: what I wanted is a way to guarantee that
6547.972 * 8000 gives the right answer. it seems like a fair thing to do. I
am surprised that except for the suggestion of using decimal, I did not find
any real answer to this.
again, it actually does work, as long as the 6547.972 value is stored in a
double or in a single. it FAILS only when it is stored in a single and CAST
to a double as in CDbl(sing)*8000 where sing is a single parameter with
6547.972 as a value.

H.

 

[Daniel O'Connell [C# MVP]]

yes, this 6547.972 is the length in seconds of a wav file.
and because of the calculation I am using (8000 is the sampling rate), all
that is actually true is the .xxx digits. anything that the double adds to
it like the .xxx167... is just wrong. and was never there to begin with
(the
number is a perfect factor of 8000).

You can't ask for number of samples or parse it yourself? It seems to me you
are basically doing a sample to length to sample conversion.

I do not believe that there is ever a circumstance where a wav file actually
stores length, just sample size and then audio data length, so the data you
are getting is basically

dataSize/(sampleSize*samplesPerSecond*channels)

or something similar and getting a float out of it. Does the COM object do
anything other than parse the wav file?

 

[Jay B. Harlow [MVP - Outlook]]

csmba

can you comment on my other quesion?

See my original post, Jon does a significantly better job then I at
explaining it. Reading Jon's & the other responses I believe the "why" has
been covered.

I know about Decimal, it is not the first choice becuase of performance.

It sounds like you are worrying about the wrong end of performance. Have you
profiled the difference between calculating the value in Decimal & Double or
even Long or Integer, have you proven via Profiling (see below) in your
program that using Decimal is a performance problem in this specific
routine?

Remember the 80/20 rule. That is 80% of the execution time of your program
is spent in 20% of your code. I will optimize (worry about performance) the
20% once that 20% has been identified & proven to be a performance problem
via profiling (CLR Profiler is one profiling tool).

For info on the 80/20 rule & optimizing only the 20% see Martin Fowler's
article "Yet Another Optimization Article" at
http://martinfowler.com/ieeeSoftware/yetOptimization.pdf


Info on the CLR Profiler:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag/html/scalenethowto13.asp

http://msdn.microsoft.com/library/d...y/en-us/dndotnet/html/highperfmanagedapps.asp



If you want "3 digits to the right precision" then I would recommend
Decimal, if you want performance then I would recommend Double. If you
really want you cake (precision) & eat it also (performance), then you may
want to consider not using Floating Point (Decimal, Single or Double), as
Jon and other have demonstrated, Single & Double floating point is inexact.
Decimal is more precise, however has this perceived performance problem. (I
say perceived, as I suspect using it in most cases will not cause any
significant performance issues).
"Length in seconds of a file", really sounds like a Performance Counter. If
I needed "length/time" I would use Performance Counters...


Hope this helps
Jay

I know about Decimal, it is not the first choice becuase of performance.
can you comment on my other quesion? which was why this works:acording to what you said, the double version should not have worked...
but
it does.
h.

<<snip>>

 

[Jon Skeet [C# MVP]]

yes, this 6547.972 is the length in seconds of a wav file.
and because of the calculation I am using (8000 is the sampling rate), all
that is actually true is the .xxx digits. anything that the double adds to
it like the .xxx167... is just wrong. and was never there to begin with (the
number is a perfect factor of 8000).
so since you said that "actually" the number is stored as 6547.972164... if
I can ignore the digits behind the first 3 I get exactly what I want.... (I
can do it of course by multiplying by 1000, then flooring the number to a
int, then dividing by 1000 again)--> but if I end up storing it ina single,
and he again adds those random 167.. it will do no good.

You haven't said *why* you need it to be exact. What are you doing with
the result? If you treat it as only being exact to 3 significant
figures, it should be fine. (I would use double rather than single
though, as 6547.972 is getting close to the number of significant
digits you can rely on in a single.)

Alternatively, why not just keep it as an int, scaled up by 8000? It
really sounds like using binary floating point is definitely the wrong
way of proceeding.

Decimal is out of the question, I am doing hundred of thousands of such
calculation per file.

Hundreds of thousands of decimal operations still doesn't take very
long. Have you actually *tried* it?

while you said it is luck that
dim d as double = 6547.972
works, I find it more then strange. it can not be luck since I tested many
different numbers, all work the same: they all work when set like above. but
some return "strange" values when cast from single to double.
it is as if the problem is in the casting.

No, the problem is that the multiplication gives a result which is
accurate to the number of significant figures that single can
represent, but not accurate to the number of significant figures that
double can represent. I suspect this is still luck though, and it will
*certainly* become a problem if you deal with longer files where the
number of significant digits that single can cope with gets too small.

as for rounding the float: what I wanted is a way to guarantee that
6547.972 * 8000 gives the right answer. it seems like a fair thing to do.

Not when 6547.972 can't be exactly represented as a binary floating
point number to start with.

I am surprised that except for the suggestion of using decimal, I did not find
any real answer to this.
again, it actually does work, as long as the 6547.972 value is stored in a
double or in a single. it FAILS only when it is stored in a single and CAST
to a double as in CDbl(sing)*8000 where sing is a single parameter with
6547.972 as a value.

Yes - we've explained why. When you cast the single to a double, you're
using the less accurate representation of 6547.972, but then
multiplying that and keeping many significant digits of the result.

 

[csmba]

You haven't said *why* you need it to be exact
Well, to make all of this make sense... (this is for both Jon and Daniel
last posts)
the COM holds the wav file, and its data. my "client" wants to process a
portion of the wav file. in this case he wants to process the window 6000
sec to end of file, which he gets from the COM as a float 6547.972. now we
can say that it all boils to the fact the COM should have never return such
a float. I am not sure what are the COM alternatives... but that wasn't my
game either.

I then create a new array of samples to do computations on (calculate energy
for example), the size of that array has to be calculated from the length of
the window and the sample rate (8000). I then fill this "window" array with
values from the original array of samples returned by the COM. the COM holds
m_samples which is of correct size: 52383776.
when I fill up my window array, I have to put in its last cell the value
from the last cell on m_samples.
the "bad" calculation caused me to try and get the cell value 52383777
instead of 52383776, which is pass the last sample in the wav file...
no. I, as the client have my own reasons to which windows on the file I need
to process, and they are ruled by TIME. for example, I am told that in the
last 10 seconds of the file something happens, or that between the 15 and 55
seconds of the call... so I need to make a copy of the m_samples, but only
the size of (timeEnd-TimeStart)*8000 and here is where I got the wrong
calculation (not exactly, check the first posts for the exact line of code).

Does the COM object do anything other than parse the wav file

well, all I need from it is the array of samples, and the length of the call
and sample rate.
are you going to suggest I just get array, sample rate and maybe if he is so
kind the length of the array? I will then have to calculate the length in
seconds anyway... so I don't see how is that different (unless I store it in
a decimal of course).

OK, to focused this, and since everyone have been very helpful: I think we
all got the issue about double and single and precision. I also got the fact
Decimal is better (yet not supported in c++ or at least in COM).

I "solved" my problem (maybe badly I assume Jon will claim) by forcing the
calculation to be in singles. avoiding the cast to double solved my problem
in this case. I understand that if the number is much bigger, it may not
(Jon's position).
the only advice then I ask is, how should I "correctly" handle this case?

h.

as for the 80/20 rules: Jon you are absolutely right.

 

[Jon Skeet [C# MVP]]

Well, to make all of this make sense... (this is for both Jon and Daniel
last posts)
the COM holds the wav file, and its data. my "client" wants to process a
portion of the wav file. in this case he wants to process the window 6000
sec to end of file, which he gets from the COM as a float 6547.972.

Do you have to get it in that form? Can you not work it out accurately
(using only integers) from other information? That would be the best
bet.

well, all I need from it is the array of samples, and the length of the call
and sample rate.
are you going to suggest I just get array, sample rate and maybe if he is so
kind the length of the array? I will then have to calculate the length in
seconds anyway... so I don't see how is that different (unless I store it in
a decimal of course).

If you ask for the sample rate and the total number of samples, and you
know (as an integer) the number of seconds you want to start at, I
don't see where any floating point maths needs to be involved.

I "solved" my problem (maybe badly I assume Jon will claim) by forcing the
calculation to be in singles. avoiding the cast to double solved my problem
in this case. I understand that if the number is much bigger, it may not
(Jon's position).
the only advice then I ask is, how should I "correctly" handle this
case?

Try to avoid using floating point at all. If you've basically got a
floating point number somewhere which is some integer / 8000, and you
want to get back to the same integer, then rather than multiplying the
floating point number by 8000, you should try to get the original
integer directly.

 

[Peter Huang]

Hi

As I said before, float number is approximate number.
From the document we will know that
A single number has at lease 6 Significant digits Number
A single number has at lease 15 Significant digits Number
Type Significant digits
single 6 ¨C 7
double 15 ¨C 16

When we convert single to double the error is caused by the fact that the
single stored a single number less accurate than double.
That is to say, when we stored 6547.972 in single, 6547.972 will be stored
with single algorithm, so do the double.
Because double will display more accurate number than single, the error
between the "single version" and the "double version" is difference between
a and b.
double a = 6547.972
single c =6547.972
double b =cdbl(c)

So for now, I think we can try to convert all the single number to double
number at the very beginnning of the procedure that we will process the
single number. So that all the consequential operation will result in
double operation which will keep at least 15 Significant digits.(Here the
6547.972 has 7 Significant digits which is same with 6547972, 65.47972
....)

Hope this help.

Best regards,

Peter Huang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.