Data Loading Strategies

Copy/Paste Tool for Data Reader's ETL
This proposal introduces a copy and paste functionality for ETL configurations within Data Reader objects in Board. Currently, when working with multiple Data Readers, developers often face the tedious task of manually configuring each ETL setting individually, which in some cases, is common for some Data Readers. This…
How can I flag all periods between 2 dates?
Hi, I need to load some discount values from a table in SQL but the table does not have a month column. Instead, we have 2 separate columns; for Start Date and End Date. In normal circumstances, I would just take the last value from each of these dates and subtract the two lists, leaving me with the period between being…
Filename substitution formula in Data Readers
Hello everyone, It could be useful for developers to have the following functionality implemented in Board: A new substitution formula @filename that returns the name of the file being loaded in the Data Reader. This formula could then be used in the ETL. Usage example: The client's cubes have the dimension "Month of…
Substitution formulas in Data Reader path
Hello everyone, It could be useful for developers to have the following functionality implemented in Board: Have already existing substitution formulas recognized on the path field of Data Readers. Usage example: By using @database on the Data Reader's path, when duplicating a database, and renaming it, we wouldn't have to…
ASCII Data Reader - Load first/last rows
Hello everyone, It could be useful for developers to have the following functionality implemented in Board: Add a new setting on the Data Reader "properties" tab that allows developers to specify a certain numer of rows to be loaded, and choosing wether to load the first or last X rows of the file, and ignore the rest of…
Prevent Objects from loading without ATO
Dear Community I have a screen with 3 Data views. First two dataviews show aggregated values. Users can click on these values and afterwards in data view 3 there is a detailed data. Now I want the third dataview only load when an entity has a number that is small enough. Has anyone any Idea how to set it up without ATO? I…
Add Smart import to cube analysis and Impact analysis
A simple idea this time : Currently, there is no reference to Smart Imports when a cube make cube analysis or Impact analysis in the DB. We can see the cube is included on a screen, but no reference about where. Sometimes, the loaded cube is also displayed on a dataview and then it becomes hard to see the difference. Would…
V12 - Smart import & max items
Hello everyone, As a neophyte, I am currently experiencing a few problems. Indeed, most of my models are still on a pre-V12 version and I'd like to understand the advantages of the V12 on certain points. Optimization with Smart Import in Board V12: What are the best strategies for taking full advantage of the "smart…
New Datasource: SAP Data Lake
Customers in DACH-Region are mainly driven by SAP. SAP Cloud is increasingly being used by customers. SAP data and non-SAP data is stored there. It is necessary to read and write data from SAP Data Lake. Connect to Data Lake Relational Engine Using the .NET Driver
Probably The Best Alternative to CSV Storage: Parquet Data
1. What is your idea? Board support a lot of datasources. Probably the most used are csv files. However, with ever increasing amounts of data, working with csv files becomes a pain. Parquet files can be an alternative. Head-to-head comparison CSV Parquet Row-based storage format. A hybrid of Row-based and column-based…
Add option to set all 'Modes' in a Data Reader to 'Add New Item' automatically
Hi, I think it'd be useful and efficient a have a button to switch all the modes automatically from 'Discard New Item'/'Read' to 'Add New Item'/'Replace'. This would be very useful in new data models or when loading a new entity/hierarchy as you do not have to manually change the mode for each field area in the data…
Temporary DataReader in Procedures
Hello together, to reduce the number of DataReaders, it would be helpful if DataReaders can be created temporarily in a procedure. This would also significantly increase flexibility in combination with temporary cubes. Best regards Christoph S.
Transferring data directly from one database to another
Situation: Assume that planning takes place in different databases for various reasons. Now the results of the planning are to be merged into one database. One possibility is to extract the data from the planning databases into a CSV and read it back into the reporting database. Request: It would be nice to have a function…
Smart Import Enhancement
The last releases contained a lot of news about the Smart Import Object. The functions that the object now brings with it are great. Still, I would like to see two more features: moving columns (without having to rename and remap columns every time a new column is added in between. if rows are rejected, it would be great…
Extract Cubes in Parquet-File
1. What is your idea? Board has multiple steps to extract data from cubes: Extract Cube Export Data View to file Export Dataset Extract all cubes All these steps save data as text or csv-files. We could save disk-space and increase performance by using parquet-files. Head-to-head comparison CSV Parquet Row-based storage…
How to manage cloud tasks from on premise server - Orchestrator
Hello community i'm going to move to cloud and I know it's now possible execute a board procedure through an API call. Since I have various tasks to copy files to/from the cloud, to be synchronyzed with Board procedures, I would evaluate possible solutions to manage processes that combines batches to transfer files (with…
How to hide/ignore columns of an Excel data source when using the Smart Import Object
Is it possible to hide/ignore columns of an Excel data source if they are not relevant for the import when using the file upload of the Smart Import Object (e.g. descriptions or comments)? In the following table, columns A,B (entities) and D,E (values) have to be imported, columns C,F have to be ignored.
Data Load Guidelines
1. Abstract While developers are focused on data model creation and data integration, they need to remember that in addition to the data model structure, they have to fill this structure with data. Entities and Cubes have to be filled with meaningful data to be seen by the users. This document provides a standardized…
Data Transformation
1. Abstract The data transformation process can be done in three different ways: 1. with SQL transformations directly in the data source or in the Data Reader Object 2. with the out-of-the-box Board ETL function 3. after the data has been imported into Board by using the dataflow engine. The type and number of data sources…

Data Loading Strategies

Discussion List