S
Steve
Hi,
I've developed a testing application that stores formatted results in a
database. Recently it was requested that I add the ability to generate
graphs from the raw, un formatted test results (100,000+ float values)
I don't intend to store all of the 100,000 datapoints, but rather a subset
of say 250. Due to the volume of testing that we need and the volume of
results stored, I need to be VERY careful with data size and keep things as
lean and small as possible.
Given the example data: (-5.2485, 5.9875, -1.2548, 7.5842)
Those samples represent our data value boundaries, they won't get any bigger
than that, they are signed and they are floating point.
I don't need the precision of a float, the above values would be fine to be
stored as:
-5.29
5.99
-1.25
7.58
A couple of ideas I have come up with is to try compressing the array of 250
floats and storing the compressed binary in the DB BLOB field. I also
though about breaking the values into an sbyte and a byte like this:
-5.29
represented as:
sbyte l = -5;
byte r = 29;
This would save me to bytes from using a float. If I then compressed this
"byte data" I could save even more space.
Anyway, I have never done any of this before and wanted to invite you guys
to offer any suggestions or past experiences you might of had with doing
something like this.
As you can see, I'm not real sure what I'm doing ;0)
Thanks for reading,
Steve
I've developed a testing application that stores formatted results in a
database. Recently it was requested that I add the ability to generate
graphs from the raw, un formatted test results (100,000+ float values)
I don't intend to store all of the 100,000 datapoints, but rather a subset
of say 250. Due to the volume of testing that we need and the volume of
results stored, I need to be VERY careful with data size and keep things as
lean and small as possible.
Given the example data: (-5.2485, 5.9875, -1.2548, 7.5842)
Those samples represent our data value boundaries, they won't get any bigger
than that, they are signed and they are floating point.
I don't need the precision of a float, the above values would be fine to be
stored as:
-5.29
5.99
-1.25
7.58
A couple of ideas I have come up with is to try compressing the array of 250
floats and storing the compressed binary in the DB BLOB field. I also
though about breaking the values into an sbyte and a byte like this:
-5.29
represented as:
sbyte l = -5;
byte r = 29;
This would save me to bytes from using a float. If I then compressed this
"byte data" I could save even more space.
Anyway, I have never done any of this before and wanted to invite you guys
to offer any suggestions or past experiences you might of had with doing
something like this.
As you can see, I'm not real sure what I'm doing ;0)
Thanks for reading,
Steve
