Airflow 3: Advanced DAG Authoring Take your Apache Airflow skills to the next level. This course dives deep into the powerful features of Airflow 3 that separate beginners from production-ready data engineers.
You’ll master the TaskFlow API—from the basics to advanced patterns like dynamic DAG generation, task groups, pools, and resource management. Learn how to build flexible, scalable pipelines using dynamic task mapping with advanced techniques like reduce, expand, and more. Explore modern scheduling strategies including assets, conditional scheduling, and event-driven pipelines with services like AWS SQS. Plus, discover how to integrate AI into your workflows using LLMs, the AI SDK, and human-in-the-loop approval patterns.
What you’ll learn:
- Write clean, Pythonic DAGs using the TaskFlow API
- Generate DAGs dynamically and reuse tasks like a pro
- Master dynamic task mapping for flexible, data-driven workflows
- Schedule pipelines using assets, event-driven triggers, and continuous scheduling
- Integrate AI and LLMs directly into your Airflow tasks
- Implement human-in-the-loop workflows for AI approvals
Every video has its corresponding source code so it’s easy for you to follow along.
Who this course is for: Data engineers and developers with foundational Airflow knowledge who want to write more efficient, maintainable, and production-grade DAGs using Airflow 3’s latest features.
I hope you are ready for the course. Let’s do it!
Marc Lamberti
