Google

Airflow has support for the Google service.

All hooks are based on airflow.providers.google.common.hooks.base_google.GoogleBaseHook. Some integration also use airflow.providers.google.common.hooks.discovery_api.

See the Google Cloud connection type documentation to configure connections to Google services.

Google Cloud

Airflow has extensive support for the Google Cloud.

Note

You can learn how to use Google Cloud integrations by analyzing the source code of the Google Cloud example DAGs

Services

These integrations allow you to perform various operations within the Google Cloud.

Google Cloud Key Management Service (KMS)

Hooks

airflow.providers.google.cloud.hooks.kms.

Provider

apache-airflow-providers-google

Product documentation

Google Cloud Key Management Service (KMS)

Google Deployment Manager

Hooks

airflow.providers.google.cloud.hooks.gdm.

Provider

apache-airflow-providers-google

Product documentation

Google Deployment Manager

Transfers

These integrations allow you to copy data from/to Google Cloud.

Google Cloud Storage (GCS) to Amazon Simple Storage Service (S3)

Source product documentation

Google Cloud Storage (GCS)

Target product documentation

Amazon Simple Storage Service (S3)

Python API

airflow.providers.amazon.aws.transfers.gcs_to_s3

Provider

apache-airflow-providers-amazon

Amazon Glacier to Google Cloud Storage (GCS)

Source product documentation

Amazon Glacier

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Amazon Glacier Transfer Operator

Python API

airflow.providers.amazon.aws.transfers.glacier_to_gcs

Provider

apache-airflow-providers-amazon

Presto to Google Cloud Storage (GCS)

Source product documentation

Presto

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Presto to Google Cloud Storage Transfer Operator

Python API

airflow.providers.google.cloud.transfers.presto_to_gcs

Provider

apache-airflow-providers-google

Trino to Google Cloud Storage (GCS)

Source product documentation

Trino

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Trino to Google Cloud Storage Transfer Operator

Python API

airflow.providers.google.cloud.transfers.trino_to_gcs

Provider

apache-airflow-providers-google

SQL to Google Cloud Storage (GCS)

Target product documentation

Google Cloud Storage (GCS)

Python API

airflow.providers.google.cloud.transfers.sql_to_gcs

Provider

apache-airflow-providers-google

Google Cloud Storage (GCS) to Google Drive

Source product documentation

Google Cloud Storage (GCS)

Target product documentation

Google Drive

Operator guide

Google Cloud Storage to Google Drive Transfer Operators

Python API

airflow.providers.google.suite.transfers.gcs_to_gdrive

Provider

apache-airflow-providers-google

Google Drive to Google Cloud Storage (GCS)

Source product documentation

Google Drive

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Google Drive to Google Cloud Storage Transfer Operator

Python API

airflow.providers.google.cloud.transfers.gdrive_to_gcs

Provider

apache-airflow-providers-google

Microsoft SQL Server (MSSQL) to Google Cloud Storage (GCS)

Source product documentation

Microsoft SQL Server (MSSQL)

Target product documentation

Google Cloud Storage (GCS)

Python API

airflow.providers.google.cloud.transfers.mssql_to_gcs

Provider

apache-airflow-providers-google

Microsoft Azure FileShare to Google Cloud Storage (GCS)

Source product documentation

Microsoft Azure FileShare

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Transfers data from Azure FileShare Storage to Google Cloud Storage

Python API

airflow.providers.google.cloud.transfers.azure_fileshare_to_gcs

Provider

apache-airflow-providers-google

Apache Cassandra to Google Cloud Storage (GCS)

Source product documentation

Apache Cassandra

Target product documentation

Google Cloud Storage (GCS)

Python API

airflow.providers.google.cloud.transfers.cassandra_to_gcs

Provider

apache-airflow-providers-google

Google Spreadsheet to Google Cloud Storage (GCS)

Source product documentation

Google Spreadsheet

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Google Sheets to Google Cloud Storage Transfer Operators

Python API

airflow.providers.google.cloud.transfers.sheets_to_gcs

Provider

apache-airflow-providers-google

Amazon Simple Storage Service (S3) to Google Cloud Storage (GCS)

Source product documentation

Amazon Simple Storage Service (S3)

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Transfer Data from Amazon S3 to Google Cloud Storage

Python API

airflow.providers.google.cloud.transfers.s3_to_gcs

Provider

apache-airflow-providers-google

Google Cloud Storage (GCS) to SSH File Transfer Protocol (SFTP)

Source product documentation

Google Cloud Storage (GCS)

Target product documentation

SSH File Transfer Protocol (SFTP)

Operator guide

Google Cloud Storage Transfer Operator to SFTP

Python API

airflow.providers.google.cloud.transfers.gcs_to_sftp

Provider

apache-airflow-providers-google

PostgreSQL to Google Cloud Storage (GCS)

Source product documentation

PostgreSQL

Target product documentation

Google Cloud Storage (GCS)

Python API

airflow.providers.google.cloud.transfers.postgres_to_gcs

Provider

apache-airflow-providers-google

Google BigQuery to MySQL

Source product documentation

Google BigQuery

Target product documentation

MySQL

Python API

airflow.providers.google.cloud.transfers.bigquery_to_mysql

Provider

apache-airflow-providers-google

Google BigQuery to Microsoft SQL Server (MSSQL)

Source product documentation

Google BigQuery

Target product documentation

Microsoft SQL Server (MSSQL)

Python API

airflow.providers.google.cloud.transfers.bigquery_to_mssql

Provider

apache-airflow-providers-google

Google Cloud Storage (GCS) to Google BigQuery

Source product documentation

Google Cloud Storage (GCS)

Target product documentation

Google BigQuery

Python API

airflow.providers.google.cloud.transfers.gcs_to_bigquery

Provider

apache-airflow-providers-google

Google Cloud Storage (GCS) to Google Cloud Storage (GCS)

Source product documentation

Google Cloud Storage (GCS)

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Transfer data in Google Cloud Storage

Python API

airflow.providers.google.cloud.transfers.gcs_to_gcs

Provider

apache-airflow-providers-google

Facebook Ads to Google Cloud Storage (GCS)

Source product documentation

Facebook Ads

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Facebook Ads To GCS Operators

Python API

airflow.providers.google.cloud.transfers.facebook_ads_to_gcs

Provider

apache-airflow-providers-google

SSH File Transfer Protocol (SFTP) to Google Cloud Storage (GCS)

Source product documentation

SSH File Transfer Protocol (SFTP)

Target product documentation

Google Cloud Storage (GCS)

Operator guide

SFTP to Google Cloud Storage Transfer Operator

Python API

airflow.providers.google.cloud.transfers.sftp_to_gcs

Provider

apache-airflow-providers-google

Microsoft Azure Data Lake Storage to Google Cloud Storage (GCS)

Source product documentation

Microsoft Azure Data Lake Storage

Target product documentation

Google Cloud Storage (GCS)

Python API

airflow.providers.google.cloud.transfers.adls_to_gcs

Provider

apache-airflow-providers-google

Google BigQuery to Google BigQuery

Source product documentation

Google BigQuery

Target product documentation

Google BigQuery

Python API

airflow.providers.google.cloud.transfers.bigquery_to_bigquery

Provider

apache-airflow-providers-google

MySQL to Google Cloud Storage (GCS)

Source product documentation

MySQL

Target product documentation

Google Cloud Storage (GCS)

Operator guide

MySQL To Google Cloud Storage Operator

Python API

airflow.providers.google.cloud.transfers.mysql_to_gcs

Provider

apache-airflow-providers-google

Oracle to Google Cloud Storage (GCS)

Source product documentation

Oracle

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Oracle To Google Cloud Storage Operator

Python API

airflow.providers.google.cloud.transfers.oracle_to_gcs

Provider

apache-airflow-providers-google

Google Cloud Storage (GCS) to Google Spreadsheet

Source product documentation

Google Cloud Storage (GCS)

Target product documentation

Google Spreadsheet

Operator guide

Google Cloud Storage to Google Sheets Transfer Operators

Python API

airflow.providers.google.suite.transfers.gcs_to_sheets

Provider

apache-airflow-providers-google

Google BigQuery to Google Cloud Storage (GCS)

Source product documentation

Google BigQuery

Target product documentation

Google Cloud Storage (GCS)

Python API

airflow.providers.google.cloud.transfers.bigquery_to_gcs

Provider

apache-airflow-providers-google

Salesforce to Google Cloud Storage (GCS)

Source product documentation

Salesforce

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Salesforce To GCS Operators

Python API

airflow.providers.google.cloud.transfers.salesforce_to_gcs

Provider

apache-airflow-providers-google

Microsoft SQL Server (MSSQL) to Google Cloud Storage (GCS)

Source product documentation

Microsoft SQL Server (MSSQL)

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Microsoft SQL Server To Google Cloud Storage Operator

Python API

airflow.providers.google.cloud.transfers.mssql_to_gcs

Provider

apache-airflow-providers-google

Microsoft Azure Blob Storage to Google Cloud Storage (GCS)

Source product documentation

Microsoft Azure Blob Storage

Target product documentation

Google Cloud Storage (GCS)

Operator guide

Azure Blob Storage Transfer Operator

Python API

airflow.providers.microsoft.azure.transfers.azure_blob_to_gcs

Provider

apache-airflow-providers-microsoft-azure

Other Google operators and hooks

Google Calendar

Hooks

airflow.providers.google.suite.hooks.calendar.

Provider

apache-airflow-providers-google

Product documentation

Google Calendar

Was this entry helpful?