Airflow Summit 2021 is coming July 8-16. Register now!

Upload data from Local Filesystem to Google Cloud Storage

The Google Cloud Storage (GCS) is used to store large data from various applications. This page shows how to upload data from local filesystem to GCS.

Prerequisite Tasks

LocalFilesystemToGCSOperator

LocalFilesystemToGCSOperator allows you to upload data from local filesystem to GCS.

When you use this operator, you can optionally compress the data being uploaded.

Below is an example of using this operator to upload a file to GCS.

airflow/providers/google/cloud/example_dags/example_local_to_gcs.pyView Source

    upload_file = LocalFilesystemToGCSOperator(
        task_id="upload_file",
        src=PATH_TO_UPLOAD_FILE,
        dst=DESTINATION_FILE_LOCATION,
        bucket=BUCKET_NAME,
    )

Reference

For further information, look at:

Was this entry helpful?