• Home
• Google Operators
• Google Cloud Operators
• Google Compute Engine Operators

Google Compute Engine Operators

Prerequisite Tasks

To use these operators, you must do a few things:

• Select or create a Cloud Platform project using the Cloud Console.

• Enable billing for your project, as described in the Google Cloud documentation.

• Enable the API, as described in the Cloud Console documentation.

• Install API libraries via pip.

  pip install 'apache-airflow[google]'
  

  Detailed information is available for Installation.

• Setup a Google Cloud Connection.

ComputeEngineInsertInstanceOperator

Use the ComputeEngineInsertInstanceOperator to create new Google Compute Engine instance.

Using the operator

The code to create the operator:

tests/system/providers/google/cloud/compute/example_compute.py

gce_instance_insert = ComputeEngineInsertInstanceOperator(
    task_id="gcp_compute_create_instance_task",
    project_id=PROJECT_ID,
    zone=LOCATION,
    body=GCE_INSTANCE_BODY,
)

You can also create the operator without project id - project id will be retrieved from the Google Cloud connection id used:

tests/system/providers/google/cloud/compute/example_compute.py

gce_instance_insert2 = ComputeEngineInsertInstanceOperator(
    task_id="gcp_compute_create_instance_task_2",
    zone=LOCATION,
    body=GCE_INSTANCE_BODY,
)

Templating

template_fields: Sequence[str] = (
    "body",
    "project_id",
    "zone",
    "request_id",
    "gcp_conn_id",
    "api_version",
    "impersonation_chain",
    "resource_id",
)

More information

See Google Compute Engine API documentation to insert an instance.

ComputeEngineInsertInstanceFromTemplateOperator

Use the ComputeEngineInsertInstanceFromTemplateOperator to create new Google Compute Engine instance based on specified instance template.

Using the operator

The code to create the operator:

tests/system/providers/google/cloud/compute/example_compute.py

gce_instance_insert_from_template = ComputeEngineInsertInstanceFromTemplateOperator(
    task_id="gcp_compute_create_instance_from_template_task",
    project_id=PROJECT_ID,
    zone=LOCATION,
    body=GCE_INSTANCE_FROM_TEMPLATE_BODY,
    source_instance_template=f"global/instanceTemplates/{TEMPLATE_NAME}",
)

You can also create the operator without project id - project id will be retrieved from the Google Cloud connection id used:

tests/system/providers/google/cloud/compute/example_compute.py

gce_instance_insert_from_template2 = ComputeEngineInsertInstanceFromTemplateOperator(
    task_id="gcp_compute_create_instance_from_template_task_2",
    zone=LOCATION,
    body=GCE_INSTANCE_FROM_TEMPLATE_BODY,
    source_instance_template=f"global/instanceTemplates/{TEMPLATE_NAME}",
)

Templating

template_fields: Sequence[str] = (
    "body",
    "source_instance_template",
    "project_id",
    "zone",
    "request_id",
    "gcp_conn_id",
    "api_version",
    "impersonation_chain",
    "resource_id",
)

More information

See Google Compute Engine API documentation to insert an instance from template.

ComputeEngineDeleteInstanceOperator

Use the ComputeEngineDeleteInstanceOperator to delete an existing Google Compute Engine instance.

Using the operator

You can create the operator without project id - project id will be retrieved from the Google Cloud connection id used. The code to create the operator:

tests/system/providers/google/cloud/compute/example_compute.py

gce_instance_delete = ComputeEngineDeleteInstanceOperator(
    task_id="gcp_compute_delete_instance_task",
    zone=LOCATION,
    resource_id=GCE_INSTANCE_NAME,
    trigger_rule=TriggerRule.ALL_DONE,
)

Templating

template_fields: Sequence[str] = (
    "zone",
    "resource_id",
    "request_id",
    "project_id",
    "gcp_conn_id",
    "api_version",
    "impersonation_chain",
)

More information

See Google Compute Engine API documentation to delete an instance.

ComputeEngineStartInstanceOperator

Use the ComputeEngineStartInstanceOperator to start an existing Google Compute Engine instance.

Using the operator

The code to create the operator:

tests/system/providers/google/cloud/compute/example_compute.py

gce_instance_start = ComputeEngineStartInstanceOperator(
    task_id="gcp_compute_start_task",
    project_id=PROJECT_ID,
    zone=LOCATION,
    resource_id=GCE_INSTANCE_NAME,
)

You can also create the operator without project id - project id will be retrieved from the Google Cloud connection id used:

tests/system/providers/google/cloud/compute/example_compute.py

gce_instance_start2 = ComputeEngineStartInstanceOperator(
    task_id="gcp_compute_start_task_2",
    zone=LOCATION,
    resource_id=GCE_INSTANCE_NAME,
)

Templating

template_fields: Sequence[str] = (
    "project_id",
    "zone",
    "resource_id",
    "gcp_conn_id",
    "api_version",
    "impersonation_chain",
)

More information

See Google Compute Engine API documentation to start an instance.

ComputeEngineStopInstanceOperator

Use the operator to stop Google Compute Engine instance.

For parameter definition, take a look at ComputeEngineStopInstanceOperator

Using the operator

The code to create the operator:

tests/system/providers/google/cloud/compute/example_compute.py

gce_instance_stop = ComputeEngineStopInstanceOperator(
    task_id="gcp_compute_stop_task",
    project_id=PROJECT_ID,
    zone=LOCATION,
    resource_id=GCE_INSTANCE_NAME,
    trigger_rule=TriggerRule.ALL_DONE,
)

You can also create the operator without project id - project id will be retrieved from the Google Cloud connection used:

tests/system/providers/google/cloud/compute/example_compute.py

gce_instance_stop2 = ComputeEngineStopInstanceOperator(
    task_id="gcp_compute_stop_task_2",
    zone=LOCATION,
    resource_id=GCE_INSTANCE_NAME,
    trigger_rule=TriggerRule.ALL_DONE,
)

Templating

template_fields: Sequence[str] = (
    "project_id",
    "zone",
    "resource_id",
    "gcp_conn_id",
    "api_version",
    "impersonation_chain",
)

More information

See Google Compute Engine API documentation to stop an instance.

ComputeEngineSetMachineTypeOperator

Use the operator to change machine type of a Google Compute Engine instance.

For parameter definition, take a look at ComputeEngineSetMachineTypeOperator.

Arguments

Using the operator

The code to create the operator:

tests/system/providers/google/cloud/compute/example_compute.py

gce_set_machine_type = ComputeEngineSetMachineTypeOperator(
    task_id="gcp_compute_set_machine_type",
    project_id=PROJECT_ID,
    zone=LOCATION,
    resource_id=GCE_INSTANCE_NAME,
    body={"machineType": f"zones/{LOCATION}/machineTypes/{SHORT_MACHINE_TYPE_NAME}"},
)

You can also create the operator without project id - project id will be retrieved from the Google Cloud connection used:

tests/system/providers/google/cloud/compute/example_compute.py

gce_set_machine_type2 = ComputeEngineSetMachineTypeOperator(
    zone=LOCATION,
    resource_id=GCE_INSTANCE_NAME,
    body={"machineType": f"zones/{LOCATION}/machineTypes/{SHORT_MACHINE_TYPE_NAME}"},
    task_id="gcp_compute_set_machine_type_2",
)

Templating

template_fields: Sequence[str] = (
    "project_id",
    "zone",
    "resource_id",
    "body",
    "gcp_conn_id",
    "api_version",
    "impersonation_chain",
)

More information

See Google Compute Engine API documentation to set the machine type.

ComputeEngineDeleteInstanceTemplateOperator

Use the operator to delete Google Compute Engine instance template.

For parameter definition, take a look at ComputeEngineDeleteInstanceTemplateOperator.

Using the operator

The code to create the operator:

You can create the operator without project id - project id will be retrieved from the Google Cloud connection used. The code to create the operator:

tests/system/providers/google/cloud/compute/example_compute_igm.py

gce_instance_template_old_delete = ComputeEngineDeleteInstanceTemplateOperator(
    task_id="gcp_compute_delete_old_template_task",
    resource_id=TEMPLATE_NAME,
    trigger_rule=TriggerRule.ALL_DONE,
)

Templating

template_fields: Sequence[str] = (
    "resource_id",
    "request_id",
    "project_id",
    "gcp_conn_id",
    "api_version",
    "impersonation_chain",
)

More information

See Google Compute Engine API documentation to delete a template.

ComputeEngineInsertInstanceTemplateOperator

Use the operator to create Google Compute Engine instance template.

For parameter definition, take a look at ComputeEngineInsertInstanceTemplateOperator.

Using the operator

The code to create the operator:

tests/system/providers/google/cloud/compute/example_compute_igm.py

gce_instance_template_insert = ComputeEngineInsertInstanceTemplateOperator(
    task_id="gcp_compute_create_template_task",
    project_id=PROJECT_ID,
    body=INSTANCE_TEMPLATE_BODY,
)

You can also create the operator without project id - project id will be retrieved from the Google Cloud connection used:

tests/system/providers/google/cloud/compute/example_compute_igm.py

gce_instance_template_insert2 = ComputeEngineInsertInstanceTemplateOperator(
    task_id="gcp_compute_create_template_task_2",
    body=INSTANCE_TEMPLATE_BODY,
)

Templating

template_fields: Sequence[str] = (
    "body",
    "project_id",
    "request_id",
    "gcp_conn_id",
    "api_version",
    "impersonation_chain",
    "resource_id",
)

More information

See Google Compute Engine API documentation to create a new template.

ComputeEngineCopyInstanceTemplateOperator

Use the operator to copy an existing Google Compute Engine instance template applying a patch to it.

For parameter definition, take a look at ComputeEngineCopyInstanceTemplateOperator.

Using the operator

The code to create the operator:

tests/system/providers/google/cloud/compute/example_compute_igm.py

gce_instance_template_copy = ComputeEngineCopyInstanceTemplateOperator(
    task_id="gcp_compute_igm_copy_template_task",
    project_id=PROJECT_ID,
    resource_id=TEMPLATE_NAME,
    body_patch=INSTANCE_TEMPLATE_BODY_UPDATE,
)

You can also create the operator without project id - project id will be retrieved from the Google Cloud connection used:

tests/system/providers/google/cloud/compute/example_compute_igm.py

gce_instance_template_copy2 = ComputeEngineCopyInstanceTemplateOperator(
    task_id="gcp_compute_igm_copy_template_task_2",
    resource_id=TEMPLATE_NAME,
    body_patch=INSTANCE_TEMPLATE_BODY_UPDATE,
)

Templating

template_fields: Sequence[str] = (
    "project_id",
    "resource_id",
    "request_id",
    "gcp_conn_id",
    "api_version",
    "impersonation_chain",
)

More information

See Google Compute Engine API documentation to create a new instance with an existing template.

ComputeEngineInsertInstanceGroupManagerOperator

Use the operator to create a Compute Engine Instance Group Manager.

For parameter definition, take a look at ComputeEngineInsertInstanceGroupManagerOperator.

Arguments

Using the operator

The code to create the operator:

tests/system/providers/google/cloud/compute/example_compute_igm.py

    gce_igm_insert = ComputeEngineInsertInstanceGroupManagerOperator(
        task_id="gcp_compute_create_group_task",
        zone=LOCATION,
        body=INSTANCE_GROUP_MANAGER_BODY,
        project_id=PROJECT_ID,
    )

You can also create the operator without project id - project id will be retrieved from the Google Cloud connection used:

tests/system/providers/google/cloud/compute/example_compute_igm.py

gce_igm_insert2 = ComputeEngineInsertInstanceGroupManagerOperator(
    task_id="gcp_compute_create_group_task_2",
    zone=LOCATION,
    body=INSTANCE_GROUP_MANAGER_BODY,
)

Templating

template_fields: Sequence[str] = (
    "project_id",
    "body",
    "zone",
    "request_id",
    "gcp_conn_id",
    "api_version",
    "impersonation_chain",
    "resource_id",
)

More information

See Google Compute Engine API documentation to create a group instance.

ComputeEngineDeleteInstanceGroupManagerOperator

Use the operator to delete a Compute Engine Instance Group Manager.

For parameter definition, take a look at ComputeEngineDeleteInstanceGroupManagerOperator.

Arguments

Using the operator

You can create the operator without project id - project id will be retrieved from the Google Cloud connection used. The code to create the operator:

tests/system/providers/google/cloud/compute/example_compute_igm.py

gce_igm_delete = ComputeEngineDeleteInstanceGroupManagerOperator(
    task_id="gcp_compute_delete_group_task",
    resource_id=INSTANCE_GROUP_MANAGER_NAME,
    zone=LOCATION,
    trigger_rule=TriggerRule.ALL_DONE,
)

Templating

template_fields: Sequence[str] = (
    "project_id",
    "resource_id",
    "zone",
    "request_id",
    "gcp_conn_id",
    "api_version",
    "impersonation_chain",
)

More information

See Google Compute Engine API documentation to delete a group instance.

ComputeEngineInstanceGroupUpdateManagerTemplateOperator

Use the operator to update a template in Google Compute Engine Instance Group Manager.

For parameter definition, take a look at ComputeEngineInstanceGroupUpdateManagerTemplateOperator.

Arguments

Using the operator

The code to create the operator:

tests/system/providers/google/cloud/compute/example_compute_igm.py

gce_instance_group_manager_update_template = ComputeEngineInstanceGroupUpdateManagerTemplateOperator(
    task_id="gcp_compute_igm_group_manager_update_template",
    project_id=PROJECT_ID,
    resource_id=INSTANCE_GROUP_MANAGER_NAME,
    zone=LOCATION,
    source_template=SOURCE_TEMPLATE_URL,
    destination_template=DESTINATION_TEMPLATE_URL,
    update_policy=UPDATE_POLICY,
)

You can also create the operator without project id - project id will be retrieved from the Google Cloud connection used:

tests/system/providers/google/cloud/compute/example_compute_igm.py

gce_instance_group_manager_update_template2 = ComputeEngineInstanceGroupUpdateManagerTemplateOperator(
    task_id="gcp_compute_igm_group_manager_update_template_2",
    resource_id=INSTANCE_GROUP_MANAGER_NAME,
    zone=LOCATION,
    source_template=SOURCE_TEMPLATE_URL,
    destination_template=DESTINATION_TEMPLATE_URL,
)

Templating

template_fields: Sequence[str] = (
    "project_id",
    "resource_id",
    "zone",
    "request_id",
    "source_template",
    "destination_template",
    "gcp_conn_id",
    "api_version",
    "impersonation_chain",
)

Troubleshooting

You might find that your ComputeEngineInstanceGroupUpdateManagerTemplateOperator fails with missing permissions. To execute the operation, the service account requires the permissions that theService Account User role provides (assigned via Google Cloud IAM).

More information

See Google Compute Engine API documentation to manage a group instance.

Reference

For further information, look at:

• Google Cloud API Documentation

• Product Documentation

Previous Next

Was this entry helpful?