How-to Guides

Setting up the sandbox in the Quick Start section was easy; building a production-grade environment requires a bit more work!

These how-to guides will step you through common tasks in using and configuring an Airflow environment.

• Setting Configuration Options
• Initializing a Database Backend
• Using Operators
  • BashOperator
  • PythonOperator
  • Google Cloud Platform Operators
• Managing Connections
  • Creating a Connection with the UI
  • Editing a Connection with the UI
  • Creating a Connection with Environment Variables
  • Connection Types
• Securing Connections
• Writing Logs
  • Writing Logs Locally
  • Writing Logs to Amazon S3
  • Writing Logs to Azure Blob Storage
  • Writing Logs to Google Cloud Storage
• Scaling Out with Celery
• Scaling Out with Dask
• Scaling Out with Mesos (community contributed)
  • Tasks executed directly on mesos slaves
  • Tasks executed in containers on mesos slaves
• Running Airflow with systemd
• Running Airflow with upstart
• Using the Test Mode Configuration