Data Reader to Cube, can't find log file

another newbie question, thanks in advance for anyone that can assist !

I have several dimensions / entities + a flat file for a cube, in CSV file on AWS S3
I am able to load the dimensions with no issue
I load the csv cube file, check the 'log' and 'stop on error' boxes
My log file is not in the same S3 bucket with "_LOG"
221k files are valid, 594k are discarded - I'm trying to figure out why those won't load - also, the cube seems completely empty when I try to create a basic report on the 221k files that it says were not discarded
am i missing some obvious config?

------------------------------
Matt Brown
Manager, Business Intelligence
Ebsco Industries Inc.
United States
------------------------------

Find more posts tagged with

Accepted answers

All comments

Stephane Joly

Hi Matt,
It could be an issue with a special character and especially if in one of the lines (probably 221879) there a line return character. It could also be an issu with the format of a field, for instance if you have a non numeric character in a numeric field (event a "space" , "-", etc.)

Hope this helps.

Stéphane Joly

------------------------------
Stephane Joly
Manager
CGI France
France
------------------------------
-------------------------------------------

Matthew Brown

Thank you for the response, Stephane.

Great input, I'll check it.

Any ideas around why the log file might not be there in S3 AWS bucket?

And how does the "Stop on error" work? Similar to a debugger?

------------------------------
Matt Brown
Manager, Business Intelligence
Ebsco Industries Inc.
United States
------------------------------
-------------------------------------------

Stephane Joly

My understanding is it just stops,meaning it should import data until their is an error and then stops reading the file. Therefore in this case I imagine their is not log or only for the first line.

------------------------------
Stephane Joly
Manager
CGI France
France
------------------------------
-------------------------------------------

Nico Weber

Hello Matt,

In the bug fix notes for the latest Board 2023 Spring Release, I found an entry that I think fits perfectly with the problem you describe.

The bug fix note is as follows:

Data reader. In the case of a Text file Data reader, the log file with all discarded records was not created in the same path as the source text file if the latter was on a supported cloud storage service (Amazon S3, Azure Blob Storage, Google Cloud Storage). The issue has been fixed and a new behavior has been put in place: the log file is now temporarily hosted in the default storage path in the Board cloud environment and Board will try to upload it on the cloud storage path where the source text file resides.
- If this process is successful, the temporary file in the Board cloud environment will be deleted.
- If this process fails, then the log file is maintained in the default storage path in the Board cloud environment

So an upgrade to the 2023 Spring Release may solve this problem.

Kind regards
Nico

------------------------------
Nico Weber
Senior Associate
PricewaterhouseCoopers GmbH Wirtschaftsprüfungsgesellschaft
Germany
------------------------------
-------------------------------------------