tests.system.google.cloud.bigquery.example_bigquery_to_mssql

Example Airflow DAG for Google BigQuery service.

This DAG relies on the following OS environment variables

  • AIRFLOW__API__GOOGLE_KEY_PATH - Path to service account key file. Note, you can skip this variable if you run this DAG in a Composer environment.

Attributes

ENV_ID

PROJECT_ID

DAG_ID

REGION

ZONE

NETWORK

CONNECTION_ID

CONNECTION_TYPE

BIGQUERY_DATASET_NAME

BIGQUERY_TABLE

INSERT_ROWS_QUERY

SCHEMA

DB_PORT

DB_USER_NAME

DB_USER_PASSWORD

SETUP_MSSQL_COMMAND

SQL_TABLE

SQL_CREATE_TABLE

GCE_MACHINE_TYPE

GCE_INSTANCE_NAME

GCE_INSTANCE_BODY

FIREWALL_RULE_NAME

CREATE_FIREWALL_RULE_COMMAND

DELETE_FIREWALL_RULE_COMMAND

DELETE_PERSISTENT_DISK_COMMAND

log

create_bigquery_dataset

test_run

Module Contents

tests.system.google.cloud.bigquery.example_bigquery_to_mssql.ENV_ID[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.PROJECT_ID[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.DAG_ID = 'bigquery_to_mssql'[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.REGION = 'us-west2'[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.ZONE = 'us-west2-a'[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.NETWORK = 'default'[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.CONNECTION_ID = ''[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.CONNECTION_TYPE = 'mssql'[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.BIGQUERY_DATASET_NAME = ''[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.BIGQUERY_TABLE = 'table_42'[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.INSERT_ROWS_QUERY = "INSERT INTO .table_42 (emp_name, salary) VALUES ('emp 1', 10000), ('emp 2', 15000);"[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.SCHEMA[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.DB_PORT = 1433[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.DB_USER_NAME = 'sa'[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.DB_USER_PASSWORD = '5FHq4fSZ85kK6g0n'[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.SETUP_MSSQL_COMMAND = Multiline-String[source]
Show Value
"""
sudo apt update &&
sudo apt install -y docker.io &&
sudo docker run -e ACCEPT_EULA=Y -e MSSQL_SA_PASSWORD=5FHq4fSZ85kK6g0n -p 1433:1433     -d mcr.microsoft.com/mssql/server:2022-latest
"""
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.SQL_TABLE = 'test_table'[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.SQL_CREATE_TABLE = Multiline-String[source]
Show Value
"""if not exists (select * from sys.tables where sys.tables.name='test_table' and sys.tables.type='U')
    create table test_table (
        emp_name VARCHAR(8),
        salary INT
    )
"""
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.GCE_MACHINE_TYPE = 'n1-standard-1'[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.GCE_INSTANCE_NAME = ''[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.GCE_INSTANCE_BODY[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.FIREWALL_RULE_NAME = ''[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.CREATE_FIREWALL_RULE_COMMAND[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.DELETE_FIREWALL_RULE_COMMAND[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.DELETE_PERSISTENT_DISK_COMMAND[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.log[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.create_bigquery_dataset[source]
tests.system.google.cloud.bigquery.example_bigquery_to_mssql.test_run[source]

Was this entry helpful?