New feature: Multi- company export BC2ADLS extension

New feature: Multi- company export BC2ADLS extension

Maybe you already know. But you can export your data to an Azure Data Lake with the tool microsoft/bc2adls: Exporting data from Dynamics 365 Business Central to Azure data lake storage (github.com)

If you need some more information I made some blog about this:
Azure Data Lake – Discover Microsoft Business Central (bertverbeek.nl)
From Data Archive to Azure Data Lake – Discover Microsoft Business Central (bertverbeek.nl)
But also give a presentation on bctechdays 2022:

But there is a new feature added to the extension! With pull request Multi company parallel exports by DuttaSoumya · Pull Request #47 · microsoft/bc2adls (github.com) you can do know a multi- company export to your Data Lake.
This means that you can trigger the export for multiple companies at ones!

If you want to use this option you have to first export it for one company. After this is done you can switch the option “Multi- Company export” to true and export the data for all companies (in each company you want to export you must trigger the export).

There are some limits were you must think of:

  • Operational limits
    Because each sessions starts with startsession. That is some limits.
    Operation Limits in Dynamics 365 Business Central – Business Central | Microsoft Learn
    Are you going over that limits those sesisons are pushed in a queue to pick up if there is one free.
    Otherwise it will export in the next export trigger.
  • Entity jsons in Data Lake
    The .json files must be the same. So you cannot change the schema.
    If you want to do that you must disable the “Multi- company export” and changge the schema. Run an export and switch the option “Multi- company export” to on again.
  • Disable and enable
    Disabling and enabling a table to not be exported or exported again is a schema change. So this is not allowed when you have the switch “Multi- company export” set to true.
  • Export of database tables issue
    When you export tables that has the property “DataPerCompany” set to false in some scenario’s you export the same table twice (or as many companies you have). This will be resolved in the Azure Synapse Pipelines.

2 COMMENTS

dataguy

Hey,

we are using the official depreceated version of bc2adls from microsoft. So if I am using Multi Company Export, but i want to add a new column to a table, what do have to do as this would be a schema change?

Would i need to trop the table (or can i just delete the manifest of the table, the entity folder within the data and deltas folder) and then do a new export? or what is the correct way to deal with it?

    Bert Verbeek

    Hi,
    What you have to do is the following:
    – Disable the option “Multi company export”
    – Add / Remove the fields that you want
    – Enable the option “Multi company export”
    – Reset the table
    – Delete the source in your data lake
    – Export the table

    This is the process for the old extension.

Leave a Reply

Your email address will not be published. Required fields are marked *


The reCAPTCHA verification period has expired. Please reload the page.