Best practice data import and update CSV

ThomasHalberstadt · 2024-11-04T07:06:15+00:00

There was an error rendering this rich post.

Hello, I'm new to the board world and am having a bit of a hard time finding the right way to import weekly updated data from a .csv file.
What would be the correct/best procedure to import the data and update it weekly. I'm really missing the common thread, where should I start?
First create the entities or cubes or data readers or what is the best way to proceed? In the .csv there are various text columns and a currency column.

I have completed eLearning 100, but unfortunately I still don't have the right approach on how to set it up now.

Thanks

Find more posts tagged with

Cube

Entity

Data Reader

Accepted answers

All comments

Helmut Heimann

Hi @ThomasHalberstadt ,

the order should be the following:

create entities / relationships
create cubes using appropriate entities as dimensions
create master data reader to populate the entities / relationships → run them
check for missing relationships, create any needed indirect relation, normalize
create transactional data reader to pouplate the cubes → run them
check data in a screen building a new dataview displaying the cube's values
check entity data in the screen selection mask

Best,
Helmut

Helmut Heimann

additional comment: you might need to run the data readers from within a procedure to apply some selections so as not to clear existing data - you'd always want historical data to remain unchanged…

ThomasHalberstadt

https://community.board.com/discussion/comment/18842#Comment_18842

Hi @Helmut Heimann What is the difference between points 3 and 5?

Thank you

Helmut Heimann

https://community.board.com/discussion/comment/18852#Comment_18852

Hi @ThomasHalberstadt ,

Step 3 is about Masterdata, Step 5 about transactional data.
It's best practice to always load masterdata first and transactional data afterwards so you'd have the dimensions already populated—you wouldn't want to load transactions for non-existing masterdata.

Usually, transactional records should be discarded where there is no relation to any dimensional value. Thus, you wouldn't populate the entities with members that don't carry any transactional information.

Best,
Helmut