Скачать с ютуб ELT with dlt 2.0. Go-to-production в хорошем качестве

ELT with dlt 2.0. Go-to-production

In this workshop, hosted by Alena and Hiba from the dlltHub, you'll dive deep into advanced data pipeline techniques. Covering essential topics like Custom Incremental Loading, Schema Configuration, Data Contracts, Performance Optimization, and how to work with Custom Sources and Destinations, this session is designed for data professionals looking to enhance their ETL (Extract, Transform, Load) skills. The workshop also explores deployment strategies with GitHub Actions, Dagster, and AWS Lambda, showing how to automate data workflows using cutting-edge technologies. Whether you're building custom APIs, optimizing pipeline performance, or deploying machine learning models, this workshop has everything you need to elevate your data engineering game. 🔑 Key Topics Covered: Custom Incremental Loading Methods Schema Configuration & Data Contracts Performance Optimization Techniques for ETL Pipelines Custom Source & Destination Integrations using REST API Deployment using GitHub Actions, Dagster, and AWS Lambda Error Handling with Tracing, Logging, and Retries 👥 Who Should Watch? This workshop is perfect for data engineers, data scientists, and developers who want to automate their workflows, optimize performance, and learn deployment strategies for data pipelines. Whether you're a beginner looking to learn more about ETL processes or a seasoned professional aiming to refine your deployment strategies, this session is for you. 📚 Resources: Learn more about dltHub: www.dlthub.com Sign up for future sessions here: https://dlthub.com/events Subscribe to our channel for more tutorials and deep-dive workshops on data pipelines, machine learning, and automation! :) 0:00 Introduction 2:02 Alena & Hiba Introduce themselves 4:09 Recap from Part 1 8:07 Custom Incremental Loading 17:52 Schema Configuration & Data Contracts 26:33 Tracing,Logging & Retries 36:46 Retries 44:22 Bonus: DBT Runner 54:38 Performance Optimization 1:22:11 Custom Source: REST API Helpers 1:43:08 Custom Destinations 1:44:36 Custom Destinations Example: Notion 2:20:13 Thank you :) #DataPipeline #ETL #DataEngineering #MachineLearning #APIDevelopment #GitHubActions #AWSLambda #Dagster #DataScience #DataOptimization #IncrementalLoading #SchemaManagement #CustomAPIs #ErrorHandling #PerformanceOptimization #LoggingAndTracing #PythonProgramming #PythonETL #CloudAutomation #DatabaseManagement #RESTAPI #DeployDataPipelines #DataContracts #DataTransformation #GitHubTutorial #DataScientist #OpenSource #MachineLearningPipeline #DataAutomation #BigData #CloudComputing #DevOps #WorkflowAutomation #APIPipelines #SoftwareEngineering #PythonDevelopers #AdvancedETL #CustomIntegrations #PythonTutorial #CodingWorkshop #APIDeveloper #TechWorkshop #DatabaseOptimization #ScalableSystems #DataAnalysis #TechEducation #CustomDestinations #RESTClient #CloudArchitecture #DataIntegration #AutomatedPipelines #ServerlessComputing