Agentic Workloads on Airflow: Observable, Retryable, and Auditable by Design

Wed, 15 Apr 2026 00:00:00 +0000

A question like “How does AI tool usage vary across Airflow versions?” has a natural SQL shape: one cross-tabulation, one result. A question like “What does a typical Airflow deployment look like for practitioners who are actively using AI in their workflow?” does not. It requires querying executor type, deployment method, cloud provider, and Airflow version independently, each filtered to the same respondent group, then synthesizing the results into a coherent picture. No single query returns the answer. The answer emerges from the relationship between all of them.

Ask Your Survey Anything: Building AI Analysis Pipelines with Airflow 3

Wed, 15 Apr 2026 00:00:00 +0000

The 2025 Airflow Community Survey collected responses from nearly 6,000 practitioners across 168 questions. You can open a spreadsheet and filter, or write SQL by hand. But what if you could just ask a question and have Airflow figure out the query, run it, and bring the result back for your approval?

This post builds two pipelines that do exactly that, using the apache-airflow-providers-common-ai provider for Airflow 3.

The first pipeline is interactive: a human reviews the question before it reaches the LLM and approves the result before the DAG finishes. The second is scheduled: it downloads fresh survey data, validates the schema, runs the query unattended, and emails the result.

Tutorial on Apache Airflow

Agentic Workloads on Airflow: Observable, Retryable, and Auditable by Design

Ask Your Survey Anything: Building AI Analysis Pipelines with Airflow 3