Upload data from Local Filesystem to Google Cloud Storage

The Google Cloud Storage (GCS) is used to store large data from various applications. This page shows how to upload data from local filesystem to GCS.

Prerequisite Tasks

LocalFileSystemToGCSOperator

LocalFilesystemToGCSOperator allows you to upload data from local filesystem to GCS.

When you use this operator, you can optionally compress the data being uploaded.

Below is an example of using this operator to upload a file to GCS.

airflow/providers/google/cloud/example_dags/example_local_to_gcs.py

    upload_file = LocalFilesystemToGCSOperator(
        task_id="upload_file",
        src=PATH_TO_UPLOAD_FILE,
        dst=DESTINATION_FILE_LOCATION,
        bucket=BUCKET_NAME,
    )

Reference

For further information, look at:

Was this entry helpful?