Data Reader limits ?

Alexis Dufraux · October 3

Hi,

I would like to know if there are any Text File Data Reader limits in terms of size of the data loaded. I'll explain : I have this DR which loads only the highlighted cubes without any error message.

If I delete this 2 cubes, keeping the others, it only loads the 2 cubes highlighted following.

The file I am trying to load has 58 columns and 567k rows, so I am wondering if there are any limits concerning the size of the file and the number of cubes we can load.

Thanks

Leone Scaburri · October 3

Hi @Alexis Dufraux,

it doesn't seem to be a limit of the DataReader. Could it be you have multiple records with the same key that could zero out your key figures?

Lucas Charello · October 3

Hello Alexis,

Did you check for a log file? Any rejected loading on your DR page?

Best,

Lucas

Alexis Dufraux · October 3

https://community.board.com/discussion/comment/18656#Comment_18656

Hi @Leone Scaburri, I checked my file and I do not have multiple records, every key is unique.

Alexis Dufraux · October 3

https://community.board.com/discussion/comment/18657#Comment_18657

Hi @Lucas Charello, no I do not have any log file after loading data.

Adrien Albrespit · October 3

Hi Alexis,

To get the log file, you must activate it in the data reader configuration:

Best regards,

Adrien

Alexis Dufraux · October 3

https://community.board.com/discussion/comment/18662#Comment_18662

Hello @Adrien Albrespit, the check was active and I still did not get any log file.

Helmut Heimann · October 4

https://community.board.com/discussion/18439/data-reader-limits

Hi @Alexis Dufraux ,

can you please check that you don't have any NULL values in the files you're trying to load?

Such values often lead to discarded records with no log at all, and since you can't operate on NULL values in the reader's ETL you'll have to get rid of those beforehand.

Best,
Helmut

Alexis Dufraux · October 4

https://community.board.com/discussion/comment/18664#Comment_18664

Hi @Helmut Heimann, I do not have any NULL values in my file. The fact is that this Data Readers only load the first 2 cubes, and when I delete those, and let the others, it keeps loading the first 2 cubes of the DR (which did not load on the first DR).

Maybe I should open a ticket to the support ?

Helmut Heimann · October 4

Hi @Alexis Dufraux ,

most likely the error is due to entities in the same reader, not cubes. Do the 4 cubes share any dimensions?
Did you check the relationships for any missing (indirect) relations?

Best,
Helmut