• Home
• How-to Guides

How-to Guides

Setting up the sandbox in the Quick Start section was easy; building a production-grade environment requires a bit more work!

These how-to guides will step you through common tasks in using and configuring an Airflow environment.

• Add tags to DAGs and use it for filtering in the UI
• Add Owner Links to DAG
• Setting Configuration Options
• Set up a Database Backend
  • Choosing database backend
  • Database URI
  • Setting up a SQLite Database
  • Setting up a PostgreSQL Database
  • Setting up a MySQL Database
  • Setting up a MsSQL Database
  • Other configuration options
  • Initialize the database
  • What’s next?
• Using Operators
  • BashOperator
  • BashSensor
  • BranchDateTimeOperator
  • FileSensor
  • PythonOperator
  • PythonVirtualenvOperator
  • ExternalPythonOperator
  • ShortCircuitOperator
  • PythonSensor
  • TimeDeltaSensor
  • TimeDeltaSensorAsync
  • TimeSensor
  • TimeSensorAsync
  • BranchDayOfWeekOperator
  • DayOfWeekSensor
  • Cross-DAG Dependencies
• Customizing DAG Scheduling with Timetables
  • Timetable Registration
  • Define Scheduling Logic
  • Parameterized Timetables
  • Timetable Display in UI
  • Timetable Description Display in UI
  • Changing generated run_id
• Customize view of Apache Hive Metastore from Airflow web UI
  • Custom view Registration
• Customizing the UI
  • Customizing state colours
  • Customizing DAG UI Header and Airflow Page Titles
  • Add custom alert messages on the dashboard
• Creating a custom Operator
  • Hooks
  • User interface
  • Templating
  • Define an operator extra link
  • Sensors
• Creating Custom @task Decorators
• (Optional) Adding IDE auto-completion support
• Export dynamic environment variables available for operators to use
• Managing Connections
  • Storing connections in environment variables
  • Storing connections in a Secrets Backend
  • Storing connections in the database
  • Custom connection fields
  • URI format
• Managing Variables
  • Storing Variables in Environment Variables
  • Securing Variables
• Running Airflow behind a reverse proxy
• Running Airflow with systemd
• Using the Test Mode Configuration
• Define an operator extra link
  • Add or override Links to Existing Operators
• Email Configuration
  • Send email using SendGrid
  • Send email using AWS SES
• Dynamic DAG Generation
  • Dynamic DAGs with environment variables
  • Generating Python code with embedded meta-data
  • Dynamic DAGs with external configuration from a structured data file
  • Registering dynamic DAGs
• Running Airflow in Docker
  • Before you begin
  • Fetching docker-compose.yaml
  • Initializing Environment
  • Cleaning-up the environment
  • Running Airflow
  • Accessing the environment
  • Cleaning up
  • Using custom images
  • Networking
  • FAQ: Frequently asked questions
  • What’s Next?
  • Environment variables supported by Docker Compose

Previous Next

Was this entry helpful?