Google Cloud Build Operators

The GoogleCloud Build is a service that executes your builds on Google Cloud infrastructure. Cloud Build can import source code from Google Cloud Storage, Cloud Source Repositories, execute a build to your specifications, and produce artifacts such as Docker containers or Java archives.

Build configuration overview

In order to trigger a build, it is necessary to pass the build configuration.

airflow/providers/google/cloud/example_dags/example_cloud_build.pyView Source

create_build_from_storage_body = {
    "source": {"storageSource": GCP_SOURCE_ARCHIVE_URL},
    "steps": [
            "name": "",
            "args": ["build", "-t", f"$PROJECT_ID/{GCP_SOURCE_BUCKET_NAME}", "."],
    "images": [f"$PROJECT_ID/{GCP_SOURCE_BUCKET_NAME}"],

The source code for the build can come from Google Cloud Build Storage:

"source": {"storageSource": {"bucket": "bucket-name", "object": "object-name.tar.gz"}},

It is also possible to specify it using the URL:

"source": {"storageSource": "gs://bucket-name/object-name.tar.gz"},

In addition, a build can refer to source stored in Google Cloud Source Repositories.

"source": {
    "repoSource": {
        "projectId": "airflow-project",
        "repoName": "airflow-repo",
        "branchName": "master",

It is also possible to specify it using the URL:

"source": {"repoSource": ""},

It is also possible to specify it using a YAML or JSON format.

airflow/providers/google/cloud/example_dags/example_cloud_build.pyView Source

    create_build_from_file = CloudBuildCreateBuildOperator(
        params={'name': 'Airflow'},

Read Build Configuration Overview to understand all the fields you can include in a build config file.

Trigger a build

Trigger a build is performed with the CloudBuildCreateBuildOperator operator.

airflow/providers/google/cloud/example_dags/example_cloud_build.pyView Source

create_build_from_storage = CloudBuildCreateBuildOperator(
    task_id="create_build_from_storage", project_id=GCP_PROJECT_ID, body=create_build_from_storage_body

You can use Jinja templating with body, gcp_conn_id, api_version, impersonation_chain parameters which allows you to dynamically determine values. The result is saved to XCom, which allows it to be used by other operators.

airflow/providers/google/cloud/example_dags/example_cloud_build.pyView Source

create_build_from_storage_result = BashOperator(
    bash_command="echo '{{ task_instance.xcom_pull('create_build_from_storage')['images'][0] }}'",

Was this entry helpful?