Program Outline
Azure Data Engineering with Databricks
A hands-on data engineering program covering end-to-end pipeline development on Azure, from raw ingestion through transformation to serving. Participants work with Apache Spark, Delta Lake, and Azure Data Factory under real pipeline design constraints, not simplified examples.
Delivery
Virtual or On-site
Duration
5 days
Best Fit
Audience Profile
Who This Program Is For
Data engineers and architects building or modernizing data platforms on Azure
Overview
Program Summary
Build scalable data pipelines using Azure Databricks, Delta Lake, and Apache Spark. Covers medallion architecture and ADF integration.
Course Outline
Complete Module Sequence
Review the full module sequence for this program, including the primary topic coverage in each module where available.
1Module 1
Delta Lake and Medallion Architecture
+
Module 1
Delta Lake and Medallion Architecture
Design and implement Bronze-Silver-Gold pipeline patterns using Delta Lake, schema management, and incremental processing for scalable data platforms.
- implement Bronze-Silver-Gold pipeline patterns using Delta Lake
- schema management
- and incremental processing for scalable data platforms
2Module 2
Apache Spark Optimization
+
Module 2
Apache Spark Optimization
Write and tune Spark jobs for cost-efficient processing at enterprise scale, covering partitioning, caching, and cluster configuration.
- tune Spark jobs for cost-efficient processing at enterprise scale
- covering partitioning
- and cluster configuration
3Module 3
Azure Data Factory Orchestration
+
Module 3
Azure Data Factory Orchestration
Build ADF-triggered pipeline workflows that invoke Databricks compute, monitor job runs, and handle failures with retry and alerting logic.
- Build ADF-triggered pipeline workflows that invoke Databricks compute
- monitor job runs
- and handle failures with retry
- alerting logic
4Module 4
Data Quality and Unity Catalog
+
Module 4
Data Quality and Unity Catalog
Implement data quality checks, lineage tracking, and catalog-level governance using Databricks Unity Catalog for enterprise data reliability.
- Implement data quality checks
- lineage tracking
- and catalog-level governance using Databricks Unity Catalog for enterprise data reliability
Coverage Areas
Topic Coverage
Coverage Item 1
Delta Lake and medallion architecture design
Coverage Item 2
Apache Spark optimization and performance tuning
Coverage Item 3
Azure Data Factory orchestration with Databricks
Coverage Item 4
Data quality and Unity Catalog governance
Customization
Adapt This Program for Your Team
We can adapt this program around your team structure, platform priorities, delivery goals, and the scenarios your people need to work through in practice.
- •Align dataset and schema to team's existing sources
- •Include Unity Catalog configuration and access control
- •Add structured streaming module (Event Hubs or Kafka)
