Google Compute Engine Operators¶
Prerequisite Tasks¶
To use these operators, you must do a few things:
Select or create a Cloud Platform project using the Cloud Console.
Enable billing for your project, as described in the Google Cloud documentation.
Enable the API, as described in the Cloud Console documentation.
Install API libraries via pip.
pip install 'apache-airflow[google]'Detailed information is available for Installation.
ComputeEngineInsertInstanceOperator¶
Use the
ComputeEngineInsertInstanceOperator
to create new Google Compute Engine instance.
Using the operator¶
The code to create the operator:
gce_instance_insert = ComputeEngineInsertInstanceOperator(
task_id="gcp_compute_create_instance_task",
project_id=PROJECT_ID,
zone=LOCATION,
body=GCE_INSTANCE_BODY,
)
You can also create the operator without project id - project id will be retrieved from the Google Cloud connection id used:
gce_instance_insert2 = ComputeEngineInsertInstanceOperator(
task_id="gcp_compute_create_instance_task_2",
zone=LOCATION,
body=GCE_INSTANCE_BODY,
)
Templating¶
template_fields: Sequence[str] = (
"body",
"project_id",
"zone",
"request_id",
"gcp_conn_id",
"api_version",
"impersonation_chain",
"resource_id",
)
More information¶
See Google Compute Engine API documentation to insert an instance.
ComputeEngineInsertInstanceFromTemplateOperator¶
Use the
ComputeEngineInsertInstanceFromTemplateOperator
to create new Google Compute Engine instance based on specified instance template.
Using the operator¶
The code to create the operator:
gce_instance_insert_from_template = ComputeEngineInsertInstanceFromTemplateOperator(
task_id="gcp_compute_create_instance_from_template_task",
project_id=PROJECT_ID,
zone=LOCATION,
body=GCE_INSTANCE_FROM_TEMPLATE_BODY,
source_instance_template=f"global/instanceTemplates/{TEMPLATE_NAME}",
)
You can also create the operator without project id - project id will be retrieved from the Google Cloud connection id used:
gce_instance_insert_from_template2 = ComputeEngineInsertInstanceFromTemplateOperator(
task_id="gcp_compute_create_instance_from_template_task_2",
zone=LOCATION,
body=GCE_INSTANCE_FROM_TEMPLATE_BODY,
source_instance_template=f"global/instanceTemplates/{TEMPLATE_NAME}",
)
Templating¶
template_fields: Sequence[str] = (
"body",
"source_instance_template",
"project_id",
"zone",
"request_id",
"gcp_conn_id",
"api_version",
"impersonation_chain",
"resource_id",
)
More information¶
See Google Compute Engine API documentation to insert an instance from template.
ComputeEngineDeleteInstanceOperator¶
Use the
ComputeEngineDeleteInstanceOperator
to delete an existing Google Compute Engine instance.
Using the operator¶
You can create the operator without project id - project id will be retrieved from the Google Cloud connection id used. The code to create the operator:
gce_instance_delete = ComputeEngineDeleteInstanceOperator(
task_id="gcp_compute_delete_instance_task",
zone=LOCATION,
resource_id=GCE_INSTANCE_NAME,
trigger_rule=TriggerRule.ALL_DONE,
)
Templating¶
template_fields: Sequence[str] = (
"zone",
"resource_id",
"request_id",
"project_id",
"gcp_conn_id",
"api_version",
"impersonation_chain",
)
More information¶
See Google Compute Engine API documentation to delete an instance.
ComputeEngineStartInstanceOperator¶
Use the
ComputeEngineStartInstanceOperator
to start an existing Google Compute Engine instance.
Using the operator¶
The code to create the operator:
gce_instance_start = ComputeEngineStartInstanceOperator(
task_id="gcp_compute_start_task",
project_id=PROJECT_ID,
zone=LOCATION,
resource_id=GCE_INSTANCE_NAME,
)
You can also create the operator without project id - project id will be retrieved from the Google Cloud connection id used:
gce_instance_start2 = ComputeEngineStartInstanceOperator(
task_id="gcp_compute_start_task_2",
zone=LOCATION,
resource_id=GCE_INSTANCE_NAME,
)
Templating¶
template_fields: Sequence[str] = (
"project_id",
"zone",
"resource_id",
"gcp_conn_id",
"api_version",
"impersonation_chain",
)
More information¶
See Google Compute Engine API documentation to start an instance.
ComputeEngineStopInstanceOperator¶
Use the operator to stop Google Compute Engine instance.
For parameter definition, take a look at
ComputeEngineStopInstanceOperator
Using the operator¶
The code to create the operator:
gce_instance_stop = ComputeEngineStopInstanceOperator(
task_id="gcp_compute_stop_task",
project_id=PROJECT_ID,
zone=LOCATION,
resource_id=GCE_INSTANCE_NAME,
trigger_rule=TriggerRule.ALL_DONE,
)
You can also create the operator without project id - project id will be retrieved from the Google Cloud connection used:
gce_instance_stop2 = ComputeEngineStopInstanceOperator(
task_id="gcp_compute_stop_task_2",
zone=LOCATION,
resource_id=GCE_INSTANCE_NAME,
trigger_rule=TriggerRule.ALL_DONE,
)
Templating¶
template_fields: Sequence[str] = (
"project_id",
"zone",
"resource_id",
"gcp_conn_id",
"api_version",
"impersonation_chain",
)
More information¶
See Google Compute Engine API documentation to stop an instance.
ComputeEngineSetMachineTypeOperator¶
Use the operator to change machine type of a Google Compute Engine instance.
For parameter definition, take a look at
ComputeEngineSetMachineTypeOperator
.
Arguments¶
Using the operator¶
The code to create the operator:
gce_set_machine_type = ComputeEngineSetMachineTypeOperator(
task_id="gcp_compute_set_machine_type",
project_id=PROJECT_ID,
zone=LOCATION,
resource_id=GCE_INSTANCE_NAME,
body={"machineType": f"zones/{LOCATION}/machineTypes/{SHORT_MACHINE_TYPE_NAME}"},
)
You can also create the operator without project id - project id will be retrieved from the Google Cloud connection used:
gce_set_machine_type2 = ComputeEngineSetMachineTypeOperator(
zone=LOCATION,
resource_id=GCE_INSTANCE_NAME,
body={"machineType": f"zones/{LOCATION}/machineTypes/{SHORT_MACHINE_TYPE_NAME}"},
task_id="gcp_compute_set_machine_type_2",
)
Templating¶
template_fields: Sequence[str] = (
"project_id",
"zone",
"resource_id",
"body",
"gcp_conn_id",
"api_version",
"impersonation_chain",
)
More information¶
See Google Compute Engine API documentation to set the machine type.
ComputeEngineDeleteInstanceTemplateOperator¶
Use the operator to delete Google Compute Engine instance template.
For parameter definition, take a look at
ComputeEngineDeleteInstanceTemplateOperator
.
Using the operator¶
The code to create the operator:
You can create the operator without project id - project id will be retrieved from the Google Cloud connection used. The code to create the operator:
gce_instance_template_old_delete = ComputeEngineDeleteInstanceTemplateOperator(
task_id="gcp_compute_delete_old_template_task",
resource_id=TEMPLATE_NAME,
trigger_rule=TriggerRule.ALL_DONE,
)
Templating¶
template_fields: Sequence[str] = (
"resource_id",
"request_id",
"project_id",
"gcp_conn_id",
"api_version",
"impersonation_chain",
)
More information¶
See Google Compute Engine API documentation to delete a template.
ComputeEngineInsertInstanceTemplateOperator¶
Use the operator to create Google Compute Engine instance template.
For parameter definition, take a look at
ComputeEngineInsertInstanceTemplateOperator
.
Using the operator¶
The code to create the operator:
gce_instance_template_insert = ComputeEngineInsertInstanceTemplateOperator(
task_id="gcp_compute_create_template_task",
project_id=PROJECT_ID,
body=INSTANCE_TEMPLATE_BODY,
)
You can also create the operator without project id - project id will be retrieved from the Google Cloud connection used:
gce_instance_template_insert2 = ComputeEngineInsertInstanceTemplateOperator(
task_id="gcp_compute_create_template_task_2",
body=INSTANCE_TEMPLATE_BODY,
)
Templating¶
template_fields: Sequence[str] = (
"body",
"project_id",
"request_id",
"gcp_conn_id",
"api_version",
"impersonation_chain",
"resource_id",
)
More information¶
See Google Compute Engine API documentation to create a new template.
ComputeEngineCopyInstanceTemplateOperator¶
Use the operator to copy an existing Google Compute Engine instance template applying a patch to it.
For parameter definition, take a look at
ComputeEngineCopyInstanceTemplateOperator
.
Using the operator¶
The code to create the operator:
gce_instance_template_copy = ComputeEngineCopyInstanceTemplateOperator(
task_id="gcp_compute_igm_copy_template_task",
project_id=PROJECT_ID,
resource_id=TEMPLATE_NAME,
body_patch=INSTANCE_TEMPLATE_BODY_UPDATE,
)
You can also create the operator without project id - project id will be retrieved from the Google Cloud connection used:
gce_instance_template_copy2 = ComputeEngineCopyInstanceTemplateOperator(
task_id="gcp_compute_igm_copy_template_task_2",
resource_id=TEMPLATE_NAME,
body_patch=INSTANCE_TEMPLATE_BODY_UPDATE,
)
Templating¶
template_fields: Sequence[str] = (
"project_id",
"resource_id",
"request_id",
"gcp_conn_id",
"api_version",
"impersonation_chain",
)
More information¶
See Google Compute Engine API documentation to create a new instance with an existing template.
ComputeEngineInsertInstanceGroupManagerOperator¶
Use the operator to create a Compute Engine Instance Group Manager.
For parameter definition, take a look at
ComputeEngineInsertInstanceGroupManagerOperator
.
Arguments¶
Using the operator¶
The code to create the operator:
gce_igm_insert = ComputeEngineInsertInstanceGroupManagerOperator(
task_id="gcp_compute_create_group_task",
zone=LOCATION,
body=INSTANCE_GROUP_MANAGER_BODY,
project_id=PROJECT_ID,
)
You can also create the operator without project id - project id will be retrieved from the Google Cloud connection used:
gce_igm_insert2 = ComputeEngineInsertInstanceGroupManagerOperator(
task_id="gcp_compute_create_group_task_2",
zone=LOCATION,
body=INSTANCE_GROUP_MANAGER_BODY,
)
Templating¶
template_fields: Sequence[str] = (
"project_id",
"body",
"zone",
"request_id",
"gcp_conn_id",
"api_version",
"impersonation_chain",
"resource_id",
)
More information¶
See Google Compute Engine API documentation to create a group instance.
ComputeEngineDeleteInstanceGroupManagerOperator¶
Use the operator to delete a Compute Engine Instance Group Manager.
For parameter definition, take a look at
ComputeEngineDeleteInstanceGroupManagerOperator
.
Arguments¶
Using the operator¶
You can create the operator without project id - project id will be retrieved from the Google Cloud connection used. The code to create the operator:
gce_igm_delete = ComputeEngineDeleteInstanceGroupManagerOperator(
task_id="gcp_compute_delete_group_task",
resource_id=INSTANCE_GROUP_MANAGER_NAME,
zone=LOCATION,
trigger_rule=TriggerRule.ALL_DONE,
)
Templating¶
template_fields: Sequence[str] = (
"project_id",
"resource_id",
"zone",
"request_id",
"gcp_conn_id",
"api_version",
"impersonation_chain",
)
More information¶
See Google Compute Engine API documentation to delete a group instance.
ComputeEngineInstanceGroupUpdateManagerTemplateOperator¶
Use the operator to update a template in Google Compute Engine Instance Group Manager.
For parameter definition, take a look at
ComputeEngineInstanceGroupUpdateManagerTemplateOperator
.
Arguments¶
Using the operator¶
The code to create the operator:
gce_instance_group_manager_update_template = ComputeEngineInstanceGroupUpdateManagerTemplateOperator(
task_id="gcp_compute_igm_group_manager_update_template",
project_id=PROJECT_ID,
resource_id=INSTANCE_GROUP_MANAGER_NAME,
zone=LOCATION,
source_template=SOURCE_TEMPLATE_URL,
destination_template=DESTINATION_TEMPLATE_URL,
update_policy=UPDATE_POLICY,
)
You can also create the operator without project id - project id will be retrieved from the Google Cloud connection used:
gce_instance_group_manager_update_template2 = ComputeEngineInstanceGroupUpdateManagerTemplateOperator(
task_id="gcp_compute_igm_group_manager_update_template_2",
resource_id=INSTANCE_GROUP_MANAGER_NAME,
zone=LOCATION,
source_template=SOURCE_TEMPLATE_URL,
destination_template=DESTINATION_TEMPLATE_URL,
)
Templating¶
template_fields: Sequence[str] = (
"project_id",
"resource_id",
"zone",
"request_id",
"source_template",
"destination_template",
"gcp_conn_id",
"api_version",
"impersonation_chain",
)
Troubleshooting¶
You might find that your ComputeEngineInstanceGroupUpdateManagerTemplateOperator fails with missing permissions. To execute the operation, the service account requires the permissions that theService Account User role provides (assigned via Google Cloud IAM).
More information¶
See Google Compute Engine API documentation to manage a group instance.