`airflow.contrib.hooks.datastore_hook`¶

Module Contents¶

class airflow.contrib.hooks.datastore_hook.DatastoreHook(datastore_conn_id='google_cloud_datastore_default', delegate_to=None, api_version='v1')[source]¶

Bases: airflow.contrib.hooks.gcp_api_base_hook.GoogleCloudBaseHook

Interact with Google Cloud Datastore. This hook uses the Google Cloud Platform connection.

This object is not threads safe. If you want to make multiple requests simultaneously, you will need to create a hook per thread.

Parameters: api_version (str) – The version of the API it is going to connect to.

get_conn(self, version=None)[source]¶

Establishes a connection to the Google API.

Returns: a Google Cloud Datastore service object.
Return type: Resource

allocate_ids(self, partial_keys)[source]¶

Allocate IDs for incomplete keys.

Parameters: partial_keys (list) – a list of partial keys.
Returns: a list of full keys.
Return type: list

begin_transaction(self)[source]¶

Begins a new transaction.

Returns: a transaction handle.
Return type: str

commit(self, body)[source]¶

Commit a transaction, optionally creating, deleting or modifying some entities.

Parameters: body (dict) – the body of the commit request.
Returns: the response body of the commit request.
Return type: dict

lookup(self, keys, read_consistency=None, transaction=None)[source]¶

Lookup some entities by key.

Parameters

keys (list) – the keys to lookup.
read_consistency (str) – the read consistency to use. default, strong or eventual. Cannot be used with a transaction.
transaction (str) – the transaction to use, if any.

Returns

the response body of the lookup request.

Return type

dict

rollback(self, transaction)[source]¶

Roll back a transaction.

Parameters: transaction (str) – the transaction to roll back.

run_query(self, body)[source]¶

Run a query for entities.

Parameters: body (dict) – the body of the query request.
Returns: the batch of query results.
Return type: dict

get_operation(self, name)[source]¶

Gets the latest state of a long-running operation.

Parameters: name (str) – the name of the operation resource.
Returns: a resource operation instance.
Return type: dict

delete_operation(self, name)[source]¶

Deletes the long-running operation.

Parameters: name (str) – the name of the operation resource.
Returns: none if successful.
Return type: dict

poll_operation_until_done(self, name, polling_interval_in_seconds)[source]¶

Poll backup operation state until it’s completed.

Parameters

name (str) – the name of the operation resource
polling_interval_in_seconds (int) – The number of seconds to wait before calling another request.

Returns

a resource operation instance.

Return type

dict

export_to_storage_bucket(self, bucket, namespace=None, entity_filter=None, labels=None)[source]¶

Export entities from Cloud Datastore to Cloud Storage for backup.

Note

Keep in mind that this requests the Admin API not the Data API.

Parameters

bucket (str) – The name of the Cloud Storage bucket.
namespace (str) – The Cloud Storage namespace path.
entity_filter (dict) – Description of what data from the project is included in the export.
labels (dict of str) – Client-assigned labels.

Returns

a resource operation instance.

Return type

dict

import_from_storage_bucket(self, bucket, file, namespace=None, entity_filter=None, labels=None)[source]¶

Import a backup from Cloud Storage to Cloud Datastore.

Note

Keep in mind that this requests the Admin API not the Data API.

Parameters

bucket (str) – The name of the Cloud Storage bucket.
file (str) – the metadata file written by the projects.export operation.
namespace (str) – The Cloud Storage namespace path.
entity_filter (dict) – specify which kinds/namespaces are to be imported.
labels (dict of str) – Client-assigned labels.

Returns

a resource operation instance.

Return type

dict

airflow.contrib.hooks.datastore_hook¶

Module Contents¶

`airflow.contrib.hooks.datastore_hook`¶