Скачать с ютуб Fabric UG Switzerland February 2024 - Using Metadata Driven Orchestration to Migrate a DWH to Fabric в хорошем качестве

Fabric UG Switzerland February 2024 - Using Metadata Driven Orchestration to Migrate a DWH to Fabric

Using Metadata Driven Orchestration to Migrate a DWH to Fabric Or customer has a bespoke ERP system with over 200 Tables. 130 of which were needed to form the foundation for a DWH that feeds Analytics Reports. The current Setup used a mix of SQL, SSIS and Cognos Analytics to provide the DWH and Analysis of the data. The Challenge: Migrate the whole setup within three months to Fabric and Power BI. Thanks to a brilliant post on Metadata driven Pipelines for Microsoft Fabric by Jean Hayes we were able to build up the bronze layer in record time. The hard part was migrating the 600+ SQL scripts needed to transform the data for Analytics purposes. In this talk we’ll present the final Medallion Architecture for the Fabric Solution, describe the different components that came to play (Pipelines. Lakehouse, Warehouse, Dataflow Gen 2, Orchestration, Azure SQL, Replication, Notebooks, Semantic Models), and discuss some of the workarounds we had to come up with to navigate the murky waters of Fabric still being in preview for the duration of the project. We’ll finish up with some insights into Devops and why we still coin it Ducktape-Devops in Fabric.