airflow.contrib.hooks.datastore_hook

Module Contents

class airflow.contrib.hooks.datastore_hook.DatastoreHook(datastore_conn_id='google_cloud_datastore_default', delegate_to=None, api_version='v1')[source]

Bases: airflow.contrib.hooks.gcp_api_base_hook.GoogleCloudBaseHook

Interact with Google Cloud Datastore. This hook uses the Google Cloud Platform connection.

This object is not threads safe. If you want to make multiple requests simultaneously, you will need to create a hook per thread.

Parameters

api_version (str) – The version of the API it is going to connect to.

get_conn(self, version=None)[source]

Establishes a connection to the Google API.

Returns

a Google Cloud Datastore service object.

Return type

Resource

allocate_ids(self, partial_keys)[source]

Allocate IDs for incomplete keys.

Parameters

partial_keys (list) – a list of partial keys.

Returns

a list of full keys.

Return type

list

begin_transaction(self)[source]

Begins a new transaction.

Returns

a transaction handle.

Return type

str

commit(self, body)[source]

Commit a transaction, optionally creating, deleting or modifying some entities.

Parameters

body (dict) – the body of the commit request.

Returns

the response body of the commit request.

Return type

dict

lookup(self, keys, read_consistency=None, transaction=None)[source]

Lookup some entities by key.

Parameters
  • keys (list) – the keys to lookup.

  • read_consistency (str) – the read consistency to use. default, strong or eventual. Cannot be used with a transaction.

  • transaction (str) – the transaction to use, if any.

Returns

the response body of the lookup request.

Return type

dict

rollback(self, transaction)[source]

Roll back a transaction.

Parameters

transaction (str) – the transaction to roll back.

run_query(self, body)[source]

Run a query for entities.

Parameters

body (dict) – the body of the query request.

Returns

the batch of query results.

Return type

dict

get_operation(self, name)[source]

Gets the latest state of a long-running operation.

Parameters

name (str) – the name of the operation resource.

Returns

a resource operation instance.

Return type

dict

delete_operation(self, name)[source]

Deletes the long-running operation.

Parameters

name (str) – the name of the operation resource.

Returns

none if successful.

Return type

dict

poll_operation_until_done(self, name, polling_interval_in_seconds)[source]

Poll backup operation state until it’s completed.

Parameters
  • name (str) – the name of the operation resource

  • polling_interval_in_seconds (int) – The number of seconds to wait before calling another request.

Returns

a resource operation instance.

Return type

dict

export_to_storage_bucket(self, bucket, namespace=None, entity_filter=None, labels=None)[source]

Export entities from Cloud Datastore to Cloud Storage for backup.

Note

Keep in mind that this requests the Admin API not the Data API.

Parameters
  • bucket (str) – The name of the Cloud Storage bucket.

  • namespace (str) – The Cloud Storage namespace path.

  • entity_filter (dict) – Description of what data from the project is included in the export.

  • labels (dict of str) – Client-assigned labels.

Returns

a resource operation instance.

Return type

dict

import_from_storage_bucket(self, bucket, file, namespace=None, entity_filter=None, labels=None)[source]

Import a backup from Cloud Storage to Cloud Datastore.

Note

Keep in mind that this requests the Admin API not the Data API.

Parameters
  • bucket (str) – The name of the Cloud Storage bucket.

  • file (str) – the metadata file written by the projects.export operation.

  • namespace (str) – The Cloud Storage namespace path.

  • entity_filter (dict) – specify which kinds/namespaces are to be imported.

  • labels (dict of str) – Client-assigned labels.

Returns

a resource operation instance.

Return type

dict