airflow.providers.google.cloud.secrets.secret_manager

Objects relating to sourcing connections from Google Cloud Secrets Manager

Module Contents

airflow.providers.google.cloud.secrets.secret_manager.SECRET_ID_PATTERN = ^[a-zA-Z0-9-_]*$[source]
class airflow.providers.google.cloud.secrets.secret_manager.CloudSecretManagerBackend(connections_prefix: str = 'airflow-connections', variables_prefix: str = 'airflow-variables', config_prefix: str = 'airflow-config', gcp_keyfile_dict: Optional[dict] = None, gcp_key_path: Optional[str] = None, gcp_scopes: Optional[str] = None, project_id: Optional[str] = None, sep: str = '-', **kwargs)[source]

Bases: airflow.secrets.BaseSecretsBackend, airflow.utils.log.logging_mixin.LoggingMixin

Retrieves Connection object from Google Cloud Secrets Manager

Configurable via airflow.cfg as follows:

[secrets]
backend = airflow.providers.google.cloud.secrets.secret_manager.CloudSecretManagerBackend
backend_kwargs = {"connections_prefix": "airflow-connections", "sep": "-"}

For example, if the Secrets Manager secret id is airflow-connections-smtp_default, this would be accessible if you provide {"connections_prefix": "airflow-connections", "sep": "-"} and request conn_id smtp_default.

If the Secrets Manager secret id is airflow-variables-hello, this would be accessible if you provide {"variables_prefix": "airflow-variables", "sep": "-"} and request Variable Key hello.

The full secret id should follow the pattern “[a-zA-Z0-9-_]”.

Parameters

  • connections_prefix (str) – Specifies the prefix of the secret to read to get Connections. If set to None (null), requests for connections will not be sent to GCP Secrets Manager

  • variables_prefix (str) – Specifies the prefix of the secret to read to get Variables. If set to None (null), requests for variables will not be sent to GCP Secrets Manager

  • config_prefix (str) – Specifies the prefix of the secret to read to get Airflow Configurations containing secrets. If set to None (null), requests for configurations will not be sent to GCP Secrets Manager

  • gcp_key_path (str) – Path to Google Cloud Service Account key file (JSON). Mutually exclusive with gcp_keyfile_dict. use default credentials in the current environment if not provided.

  • gcp_keyfile_dict (dict) – Dictionary of keyfile parameters. Mutually exclusive with gcp_key_path.

  • gcp_scopes (str) – Comma-separated string containing OAuth2 scopes

  • project_id (str) – Project ID to read the secrets from. If not passed, the project ID from credentials will be used.

  • sep (str) – Separator used to concatenate connections_prefix and conn_id. Default: “-“

client(self)[source]

Cached property returning secret client.

Returns

Secrets client

_is_valid_prefix_and_sep(self)[source]
get_conn_uri(self, conn_id: str)[source]

Get secret value from the SecretManager.

Parameters

conn_id (str) – connection id

get_variable(self, key: str)[source]

Get Airflow Variable from Environment Variable

Parameters

key – Variable Key

Returns

Variable Value

get_config(self, key: str)[source]

Get Airflow Configuration

Parameters

key – Configuration Option Key

Returns

Configuration Option Value

_get_secret(self, path_prefix: str, secret_id: str)[source]

Get secret value from the SecretManager based on prefix.

Parameters

  • path_prefix (str) – Prefix for the Path to get Secret

  • secret_id (str) – Secret Key

Was this entry helpful?