decimal bug on data entry
Hi board members,
we use a single cube to enter distribution keys. In one example is just entered 0,24 and 0,76.
After a refresh i got the following values:
Same procedure works using 1,24 and 1,76.
I there any explanation for this behavior?
Best
Paul
Answers
-
Some suggestions:
Set the cube as double precision.
Or reduce the decimals visibility in the data view
0 -
Dear Paul Stegmann,
problem is not really BOARD related, it´s general problem. Following link and explanation hopefully give you some more Information.
Like Daniele Di Lorenzo suggested using DOUBLE Cube brings you double precision concerning your fugures,
http://support.microsoft.com/kb/214118
This link to Microsoft Support site explains the issue as follows:
"The IEEE 754 standard is a method of storing floating-point numbers in a compact way that is easy to manipulate. This standard is used by Intel coprocessors and most PC-based programs that implement floating-point math.
IEEE 754 specifies that numbers be stored in binary format to reduce storage requirements and allow the built-in binary arithmetic instructions that are available on all microprocessors to process the data in a relatively rapid fashion. However, some numbers that are simple, nonrepeating decimal numbers are converted into repeating binary numbers that cannot be stored with perfect accuracy.
For example, the number 1/10 can be represented in a decimal number system with a simple decimal:
.1
However, the same number in binary format becomes the repeating binary decimal:
.0001100011000111000111 (and so on)
This number cannot be represented in a finite amount of space. Therefore, this number is rounded down by approximately -2.78E-17 when it is stored.If several arithmetic operations are performed to obtain a given result, these rounding errors may be cumulative."
Hope it helps.
regards
Alexander Kappes
0 -
Thx - i'll switch to double.
0
