How to use the API to exchange data between Board databases ?

Etienne CAUSSE
Etienne CAUSSE Customer
Fourth Anniversary 250 Up Votes 100 Comments 100 Likes
edited April 2021 in Platform

Hi all,
I have been wondering recently, with the new Board API functions, is it possibe to use it to read data from a Board database to another, or within the same database (to manipulate data through mapping and / or ETL functions that would not be possible through dataflows) ?

If yes, how would you do that ?

Thanks,

Etienne

Answers

  • Etienne CAUSSE
    Etienne CAUSSE Customer
    Fourth Anniversary 250 Up Votes 100 Comments 100 Likes
    edited May 2021
    Ping to my favorite specialists @Antonio Speca @Helmut Heimann @Björn Reuber, any idea ? :)-------------------------------------------
  • Helmut Heimann
    Helmut Heimann Employee, Community Captain
    500 Comments 250 Likes Fourth Anniversary 25 Answers
    edited September 2021
    Hi @Etienne CAUSSE,

    I don't think I fully understand your request.
    What would be different if you'd use extract and re-import via flatfiles?
    The only thing I can think of would be the use of an external ETL tool to manipulate your data and then - why not do the proper ETL manipulations before loading data into Board?
    Could you please shed a bit more light on your use case?

    Kind regards,
    Helmut

    ------------------------------
    Helmut Heimann
    Senior Consultant
    Board Community
    Germany
    ------------------------------
    -------------------------------------------
  • Etienne CAUSSE
    Etienne CAUSSE Customer
    Fourth Anniversary 250 Up Votes 100 Comments 100 Likes
    edited September 2021

    Hi @Helmut Heimann,
    You are right, this would be equivalent to an export/reimport, except I have some cases with very large volumes (>10M records with 8 to 10 dimensions) and I am wondering if doing this fully in memory through the API would be faster (and also avoid creating big files and potentially saturating storage - it happened in the past).
    Any experience on that front ?
    Thanks,

    Etienne

    -------------------------------------------
  • Helmut Heimann
    Helmut Heimann Employee, Community Captain
    500 Comments 250 Likes Fourth Anniversary 25 Answers
    edited September 2021
    Hi @Etienne CAUSSE,

    yeah, understood.
    But, whether the API call will be faster in execution compared to the export/reimport is something, I can't answer generally.
    I've only just used the API to call procedures or extracting small (!) layouts, which worked pretty fast.
    And please bear in mind that using the server's memory could easily saturate that as well.
    So, no experience I can come up with to help, sorry.

    Kind regards
    Helmut

    ------------------------------
    Helmut Heimann
    Senior Consultant
    Board Community
    Germany
    ------------------------------
    -------------------------------------------

Categories