airflow.contrib.hooks.cassandra_hook

Module Contents

class airflow.contrib.hooks.cassandra_hook.CassandraHook(cassandra_conn_id='cassandra_default')[source]

Bases: airflow.hooks.base_hook.BaseHook, airflow.utils.log.logging_mixin.LoggingMixin

Hook used to interact with Cassandra

Contact points can be specified as a comma-separated string in the ‘hosts’ field of the connection.

Port can be specified in the port field of the connection.

If SSL is enabled in Cassandra, pass in a dict in the extra field as kwargs for ssl.wrap_socket(). For example:

{
    'ssl_options' : {
        'ca_certs' : PATH_TO_CA_CERTS
    }
}

Default load balancing policy is RoundRobinPolicy. To specify a different LB policy:

- DCAwareRoundRobinPolicy
    {
        'load_balancing_policy': 'DCAwareRoundRobinPolicy',
        'load_balancing_policy_args': {
            'local_dc': LOCAL_DC_NAME,                      // optional
            'used_hosts_per_remote_dc': SOME_INT_VALUE,     // optional
        }
     }
- WhiteListRoundRobinPolicy
    {
        'load_balancing_policy': 'WhiteListRoundRobinPolicy',
        'load_balancing_policy_args': {
            'hosts': ['HOST1', 'HOST2', 'HOST3']
        }
    }
- TokenAwarePolicy
    {
        'load_balancing_policy': 'TokenAwarePolicy',
        'load_balancing_policy_args': {
            'child_load_balancing_policy': CHILD_POLICY_NAME, // optional
            'child_load_balancing_policy_args': { ... }       // optional
        }
    }

For details of the Cluster config, see cassandra.cluster.

get_conn(self)[source]

Returns a cassandra Session object

get_cluster(self)[source]
shutdown_cluster(self)[source]

Closes all sessions and connections associated with this Cluster.

static get_lb_policy(policy_name, policy_args)[source]
table_exists(self, table)[source]

Checks if a table exists in Cassandra

Parameters

table (str) – Target Cassandra table. Use dot notation to target a specific keyspace.

record_exists(self, table, keys)[source]

Checks if a record exists in Cassandra

Parameters

  • table (str) – Target Cassandra table. Use dot notation to target a specific keyspace.

  • keys (dict) – The keys and their values to check the existence.

Was this entry helpful?