Best practice data import and update CSV

ThomasHalberstadt

Hello, I'm new to the board world and am having a bit of a hard time finding the right way to import weekly updated data from a .csv file.
What would be the correct/best procedure to import the data and update it weekly. I'm really missing the common thread, where should I start?
First create the entities or cubes or data readers or what is the best way to proceed? In the .csv there are various text columns and a currency column.

I have completed eLearning 100, but unfortunately I still don't have the right approach on how to set it up now.

Thanks

Helmut Heimann

Hi @ThomasHalberstadt ,

the order should be the following:

1. create entities / relationships
2. create cubes using appropriate entities as dimensions
3. create master data reader to populate the entities / relationships → run them
4. check for missing relationships, create any needed indirect relation, normalize
5. create transactional data reader to pouplate the cubes → run them
6. check data in a screen building a new dataview displaying the cube's values
7. check entity data in the screen selection mask

Best,
Helmut

Helmut Heimann

additional comment: you might need to run the data readers from within a procedure to apply some selections so as not to clear existing data - you'd always want historical data to remain unchanged…

ThomasHalberstadt

https://community.board.com/discussion/comment/18842#Comment_18842

Hi @Helmut Heimann What is the difference between points 3 and 5?

Thank you

Helmut Heimann

https://community.board.com/discussion/comment/18852#Comment_18852

Hi @ThomasHalberstadt ,

Step 3 is about Masterdata, Step 5 about transactional data.
It's best practice to always load masterdata first and transactional data afterwards so you'd have the dimensions already populated—you wouldn't want to load transactions for non-existing masterdata.

Usually, transactional records should be discarded where there is no relation to any dimensional value. Thus, you wouldn't populate the entities with members that don't carry any transactional information.

Best,
Helmut