tests.system.providers.google.cloud.gcs.example_firestore

Example Airflow DAG that shows interactions with Google Cloud Firestore.

Prerequisites

This example uses two Google Cloud projects:

  • GCP_PROJECT_ID - It contains a bucket and a firestore database.

  • G_FIRESTORE_PROJECT_ID - it contains the Data Warehouse based on the BigQuery service.

Saving in a bucket should be possible from the G_FIRESTORE_PROJECT_ID project. Reading from a bucket should be possible from the GCP_PROJECT_ID project.

The bucket and dataset should be located in the same region.

If you want to run this example, you must do the following:

  1. Create Google Cloud project and enable the BigQuery API

  2. Create the Firebase project

  3. Create a bucket in the same location as the Firebase project

  4. Grant Firebase admin account permissions to manage BigQuery. This is required to create a dataset.

  5. Create a bucket in Firebase project and

  6. Give read/write access for Firebase admin to bucket to step no. 5.

  7. Create collection in the Firestore database.

Module Contents

tests.system.providers.google.cloud.gcs.example_firestore.ENV_ID[source]
tests.system.providers.google.cloud.gcs.example_firestore.DAG_ID = 'example_google_firestore'[source]
tests.system.providers.google.cloud.gcs.example_firestore.GCP_PROJECT_ID[source]
tests.system.providers.google.cloud.gcs.example_firestore.FIRESTORE_PROJECT_ID[source]
tests.system.providers.google.cloud.gcs.example_firestore.BUCKET_NAME[source]
tests.system.providers.google.cloud.gcs.example_firestore.DATASET_NAME[source]
tests.system.providers.google.cloud.gcs.example_firestore.EXPORT_DESTINATION_URL[source]
tests.system.providers.google.cloud.gcs.example_firestore.EXPORT_PREFIX[source]
tests.system.providers.google.cloud.gcs.example_firestore.EXPORT_COLLECTION_ID[source]
tests.system.providers.google.cloud.gcs.example_firestore.DATASET_LOCATION[source]
tests.system.providers.google.cloud.gcs.example_firestore.create_bucket[source]
tests.system.providers.google.cloud.gcs.example_firestore.test_run[source]

Was this entry helpful?