decimal bug on data entry
Hi board members,
we use a single cube to enter distribution keys. In one example is just entered 0,24 and 0,76.
After a refresh i got the following values:
Same procedure works using 1,24 and 1,76.
I there any explanation for this behavior?
Best
Paul
Answers
-
Some suggestions:
Set the cube as double precision.
Or reduce the decimals visibility in the data view
0 -
Dear Paul Stegmann,
problem is not really BOARD related, it´s general problem. Following link and explanation hopefully give you some more Information.
Like Daniele Di Lorenzo suggested using DOUBLE Cube brings you double precision concerning your fugures,
http://support.microsoft.com/kb/214118
This link to Microsoft Support site explains the issue as follows:
"The IEEE 754 standard is a method of storing floating-point numbers in a compact way that is easy to manipulate. This standard is used by Intel coprocessors and most PC-based programs that implement floating-point math.
IEEE 754 specifies that numbers be stored in binary format to reduce storage requirements and allow the built-in binary arithmetic instructions that are available on all microprocessors to process the data in a relatively rapid fashion. However, some numbers that are simple, nonrepeating decimal numbers are converted into repeating binary numbers that cannot be stored with perfect accuracy.
For example, the number 1/10 can be represented in a decimal number system with a simple decimal:
.1
However, the same number in binary format becomes the repeating binary decimal:
.0001100011000111000111 (and so on)
This number cannot be represented in a finite amount of space. Therefore, this number is rounded down by approximately -2.78E-17 when it is stored.If several arithmetic operations are performed to obtain a given result, these rounding errors may be cumulative."
Hope it helps.
regards
Alexander Kappes
0 -
Thx - i'll switch to double.
0
Categories
- All Categories
- 1.8K Forums
- 1.7K Platform
- 109 Academy
- 283 Resources
- 3 Community Captains
- 1 About Community Captains
- 1 Meet the Community Captains
- 1 Topics & Thought Starters
- Board Manual
- 50 Best Practices
- 42 How-To Guides
- 25 Board Advocacy Program
- 159 Blog
- 4 Groups Hub
- 4 About Groups
- New Community Members
- DACH
- Japan
- Academy
- 2 Board Academy
- 7 ILT/VILT Course Catalogue
- 13 e-Learning Course Catalogue
- 4 Academy Forum
- 1.1K Idea Exchange
- 310 Partner Hub
- 2 Solution Sales Toolkits
- 92 Support
- 14 FAQ's
- Customer Support Portal
- 54 Support Articles
- BEAP