Google Cloud Tasks¶

Firestore in Datastore mode is a NoSQL document database built for automatic scaling, high performance, and ease of application development.

For more information about the service visit Cloud Tasks product documentation

Prerequisite Tasks¶

Queue operations¶

Create queue¶

To create new Queue use CloudTasksQueueCreateOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

create_queue = CloudTasksQueueCreateOperator(
    location=LOCATION,
    task_queue=Queue(stackdriver_logging_config=dict(sampling_ratio=0.5)),
    queue_name=QUEUE_ID,
    retry=Retry(maximum=10.0),
    timeout=5,
    task_id="create_queue",
)

Delete queue¶

To delete Queue use CloudTasksQueueDeleteOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

delete_queue = CloudTasksQueueDeleteOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task_id="delete_queue",
)

Resume queue¶

To resume Queue use CloudTasksQueueResumeOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

resume_queue = CloudTasksQueueResumeOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task_id="resume_queue",
)

Pause queue¶

To pause Queue use CloudTasksQueuePauseOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

pause_queue = CloudTasksQueuePauseOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task_id="pause_queue",
)

Purge queue¶

To purge Queue use CloudTasksQueuePurgeOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

purge_queue = CloudTasksQueuePurgeOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task_id="purge_queue",
)

Get queue¶

To get Queue use CloudTasksQueueGetOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

get_queue = CloudTasksQueueGetOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task_id="get_queue",
)

get_queue_result = BashOperator(
    task_id="get_queue_result",
    bash_command=f"echo {get_queue.output}",
)

Update queue¶

To update Queue use CloudTasksQueueUpdateOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

update_queue = CloudTasksQueueUpdateOperator(
    task_queue=Queue(stackdriver_logging_config=dict(sampling_ratio=1)),
    location=LOCATION,
    queue_name=QUEUE_ID,
    update_mask=FieldMask(paths=["stackdriver_logging_config.sampling_ratio"]),
    task_id="update_queue",
)

List queues¶

To list all Queues use CloudTasksQueuesListOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

list_queue = CloudTasksQueuesListOperator(location=LOCATION, task_id="list_queue")

Tasks operations¶

Create task¶

To create new Task in a particular queue use CloudTasksTaskCreateOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

create_task = CloudTasksTaskCreateOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task=TASK,
    task_name=TASK_NAME,
    retry=Retry(maximum=10.0),
    timeout=5,
    task_id="create_task_to_run",
)

Get task¶

To get the Tasks in a particular queue use CloudTasksTaskGetOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

tasks_get = CloudTasksTaskGetOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task_name=TASK_NAME,
    task_id="tasks_get",
)

Run task¶

To run the Task in a particular queue use CloudTasksTaskRunOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

run_task = CloudTasksTaskRunOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task_name=TASK_NAME,
    task_id="run_task",
)

List tasks¶

To list all Tasks in a particular queue use CloudTasksTasksListOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

list_tasks = CloudTasksTasksListOperator(location=LOCATION, queue_name=QUEUE_ID, task_id="list_tasks")

Delete task¶

To delete the Task from particular queue use CloudTasksTaskDeleteOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

create_task = CloudTasksTaskCreateOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task=TASK,
    task_name=TASK_NAME,
    retry=Retry(maximum=10.0),
    timeout=5,
    task_id="create_task_to_run",
)

References¶

For further information, take a look at: