Google Cloud Tasks

Firestore in Datastore mode is a NoSQL document database built for automatic scaling, high performance, and ease of application development.

For more information about the service visit Cloud Tasks product documentation

Prerequisite Tasks

Queue operations

Create queue

To create new Queue use CloudTasksQueueCreateOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

create_queue = CloudTasksQueueCreateOperator(
    location=LOCATION,
    task_queue=Queue(stackdriver_logging_config=dict(sampling_ratio=0.5)),
    queue_name=QUEUE_ID,
    retry=Retry(maximum=10.0),
    timeout=5,
    task_id="create_queue",
)

Delete queue

To delete Queue use CloudTasksQueueDeleteOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

delete_queue = CloudTasksQueueDeleteOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task_id="delete_queue",
)

Resume queue

To resume Queue use CloudTasksQueueResumeOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

resume_queue = CloudTasksQueueResumeOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task_id="resume_queue",
)

Pause queue

To pause Queue use CloudTasksQueuePauseOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

pause_queue = CloudTasksQueuePauseOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task_id="pause_queue",
)

Purge queue

To purge Queue use CloudTasksQueuePurgeOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

purge_queue = CloudTasksQueuePurgeOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task_id="purge_queue",
)

Get queue

To get Queue use CloudTasksQueueGetOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

get_queue = CloudTasksQueueGetOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task_id="get_queue",
)

get_queue_result = BashOperator(
    task_id="get_queue_result",
    bash_command=f"echo {get_queue.output}",
)

Update queue

To update Queue use CloudTasksQueueUpdateOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

update_queue = CloudTasksQueueUpdateOperator(
    task_queue=Queue(stackdriver_logging_config=dict(sampling_ratio=1)),
    location=LOCATION,
    queue_name=QUEUE_ID,
    update_mask=FieldMask(paths=["stackdriver_logging_config.sampling_ratio"]),
    task_id="update_queue",
)

List queues

To list all Queues use CloudTasksQueuesListOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

list_queue = CloudTasksQueuesListOperator(location=LOCATION, task_id="list_queue")

Tasks operations

Create task

To create new Task in a particular queue use CloudTasksTaskCreateOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

create_task = CloudTasksTaskCreateOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task=TASK,
    task_name=TASK_NAME,
    retry=Retry(maximum=10.0),
    timeout=5,
    task_id="create_task_to_run",
)

Get task

To get the Tasks in a particular queue use CloudTasksTaskGetOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

tasks_get = CloudTasksTaskGetOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task_name=TASK_NAME,
    task_id="tasks_get",
)

Run task

To run the Task in a particular queue use CloudTasksTaskRunOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

run_task = CloudTasksTaskRunOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task_name=TASK_NAME,
    task_id="run_task",
)

List tasks

To list all Tasks in a particular queue use CloudTasksTasksListOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

list_tasks = CloudTasksTasksListOperator(location=LOCATION, queue_name=QUEUE_ID, task_id="list_tasks")

Delete task

To delete the Task from particular queue use CloudTasksTaskDeleteOperator

airflow/providers/google/cloud/example_dags/example_tasks.py[source]

create_task = CloudTasksTaskCreateOperator(
    location=LOCATION,
    queue_name=QUEUE_ID,
    task=TASK,
    task_name=TASK_NAME,
    retry=Retry(maximum=10.0),
    timeout=5,
    task_id="create_task_to_run",
)

References

For further information, take a look at:

Was this entry helpful?