Dataflow execution methods: high performance in depth

Board Support
Board Support Employee
500 Likes Photogenic
edited February 2023 in Support Articles

BOARD Dataflow calculation methods

The execution time of a single BOARD Data Flow can depend on many factors, like the size of the dataset, selections applied, sparsities etc...

However, one very important element to be considered is the type of the algorithm that the BOARD engine adopts to perform the calculation.

BOARD Data Flows may use different types of algorithms, depending on the algorithm used and infocubes structures.

The fastest calculation method is named "HBMP" and the slowest is named "CellBased".

 

The Database log file contains information about the type of algorithm being used.

 

When performance issues are detected, you can utilize the following techniques to transform Data Flows from using CellBased to using HBMP.

 

Enable High Performance Mode

Ticking the “High Performance Mode” checkbox increases the application of the HBMP algorithm to the maximum breadth of Data Flow operations.

 

High Performance Mode will only invoke HBMP algorithm under the following conditions:

Functions

  • Time functions are not admitted
  • Rules are not admitted
  • Refer To functions are not admitted

Formulas 

  • Formula must not include a constant value (e.g. b=a+1)

Infocubes 

  • Only numeric cube data types are admitted.
  • All cubes in the layout must have all the dimensions of the Data Flow’s “Target” cube, even if at a different level of aggregation.

 

 

Data-flow Layout Tips

Complex dataflow layouts, might further reduce the adoption of the HBMP algorithm. In some cases it may be necessary to "break" the complex layout into several "simpler" steps that conform to the above requirements, allowing the HBMP algorithm application.

Several HBMP Data Flows take less time than a single CellBased Data Flow.

 

The following examples show common cases of layout simplification to achieve the dataflow execution through HBMP.

 

1 - Remove the "Refer To" block using a preliminary step where a combination of select and Data Flow replicates the "Refer To" operation.

 

 

 

 

 2 - Reconcile all cubes to the same sparsity structure. ("bxcxd" and "bxc" are different sparsities combinations of entities "b", "c", "d")

 

 

 

 3 - Reconcile all cubes to the same hierarchical entities. ("b" and "B" are 2 entities belonging to the same hierarchy where "B" is parent of "b")

 

 

Note:

  • The High Performance Mode checkbox is set to true by default in Board 10.0.3 and all subsequent releases. Older Data Flows must be manually updated.
  • If you encounter data calculation failures after implementing these changes, remove the checkbox to perform the calculation in the traditional manner.
Tagged:

Categories