Package apache-airflow-providers-databricks¶

Databricks

This is detailed commit list of changes for versions provider package: databricks. For high-level changelog, see package information including changelog.

4.2.0¶

Latest change: 2023-05-18

Commit	Committed	Subject
abea189022	2023-05-18	`Use '__version__' in providers not 'version' (#31393)`
f5aed58d9f	2023-05-18	`Fixing circular import error in providers caused by airflow version check (#31379)`
d9ff55cf6d	2023-05-16	`Prepare docs for May 2023 wave of Providers (#31252)`
fdc7a31aeb	2023-05-13	`Add 'DatabricksPartitionSensor' (#30980)`
edd7133a13	2023-05-09	`Add conditional output processing in SQL operators (#31136)`
3df0be0f6f	2023-05-08	`Add cancel all runs functionality to Databricks hook (#31038)`
ac46902154	2023-05-04	`Move TaskInstanceKey to a separate file (#31033)`
0a30706aa7	2023-05-03	`Use 'AirflowProviderDeprecationWarning' in providers (#30975)`
eef5bc7f16	2023-05-03	`Add full automation for min Airflow version for providers (#30994)`
a7eb32a5b2	2023-04-30	`Bump minimum Airflow version in providers (#30917)`
9409446097	2023-04-24	`Add cli cmd to list the provider trigger info (#30822)`
ecb9a9ea78	2023-04-23	`Add retry param in databrics async operator (#30744)`
9bebf85e24	2023-04-22	`Add repair job functionality to databricks hook (#30786)`
7d02277ae1	2023-04-22	`Deprecate databricks async operator (#30761)`

4.1.0¶

Latest change: 2023-04-21

Commit	Committed	Subject
e46ce78b66	2023-04-21	`Prepare docs for adhoc release of providers (#30787)`
37cf0506b5	2023-04-14	`Add delete inactive run functionality to databricks provider (#30646)`
1e311cf036	2023-04-11	`Databricks SQL sensor (#30477)`
d23a3bbed8	2023-04-04	`Add mechanism to suspend providers (#30422)`

4.0.1¶

Latest change: 2023-04-02

Commit	Committed	Subject
55dbf1ff1f	2023-04-02	`Prepare docs for April 2023 wave of Providers (#30378)`
c3867781e0	2023-03-08	`adding trigger info to provider yaml (#29950)`
c405ecb63e	2023-03-07	`DatabricksSubmitRunOperator to support taskflow (#29840)`

4.0.0¶

Latest change: 2022-11-26

Commit	Committed	Subject
25bdbc8e67	2022-11-26	`Updated docs for RC3 wave of providers (#27937)`
db5375bea7	2022-11-26	`Fixing the behaviours of SQL Hooks and Operators finally (#27912)`
2e20e9f7eb	2022-11-24	`Prepare for follow-up relase for November providers (#27774)`
80c327bd3b	2022-11-24	`Bump common.sql provider to 1.3.1 (#27888)`
ea306c9462	2022-11-24	`Fix errors in Databricks SQL operator introduced when refactoring (#27854)`
a343bba1e3	2022-11-23	`Fix templating fields and do_xcom_push in DatabricksSQLOperator (#27868)`

3.4.0¶

Latest change: 2022-11-15

Commit	Committed	Subject
12c3c39d1a	2022-11-15	`pRepare docs for November 2022 wave of Providers (#27613)`
00af5c007e	2022-11-14	`Replace urlparse with urlsplit (#27389)`
eb06c65556	2022-11-11	`Use new job search API for triggering Databricks job by name (#27446)`
9ab1a6a3e7	2022-10-27	`Update old style typing (#26872)`
78b8ea2f22	2022-10-24	`Move min airflow version to 2.3.0 for all providers (#27196)`
2a34dc9e84	2022-10-23	`Enable string normalization in python formatting - providers (#27205)`
ecd4d6654f	2022-10-18	`Add SQLExecuteQueryOperator (#25717)`

3.3.0¶

Latest change: 2022-09-28

Commit	Committed	Subject
f8db64c35c	2022-09-28	`Update docs for September Provider's release (#26731)`
89e44c46ad	2022-09-27	`Remove duplicated connection-type within the provider (#26628)`
06acf40a43	2022-09-13	`Apply PEP-563 (Postponed Evaluation of Annotations) to non-core airflow (#26289)`
5066844513	2022-09-09	`D400 first line should end with period batch02 (#25268)`
25a9c6a905	2022-08-26	`DatabricksSubmitRunOperator dbt task support (#25623)`
9535ec0bba	2022-08-22	`Databricks: fix provider name in the User-Agent string (#25873)`
ca9229b6fe	2022-08-18	`Add common-sql lower bound for common-sql (#25789)`

3.2.0¶

Latest change: 2022-08-15

Commit	Committed	Subject
7d0525a55b	2022-08-15	`Prepare documentation for RC4 release of providers (#25720)`
4d32f61fd0	2022-08-12	`Databricks: Fix provider for Airflow 2.2.x (#25674)`
e5ac6c7cfb	2022-08-10	`Prepare docs for new providers release (August 2022) (#25618)`
52f2f5bfa8	2022-08-07	`Databricks: update user-agent string (#25578)`
0255a0a5e7	2022-08-04	`Do not convert boolean values to string in deep_string_coerce function (#25394)`
679a85325a	2022-08-03	`Correctly handle output of the failed tasks (#25427)`
82f842ffc5	2022-07-28	`updated documentation for databricks operator (#24599)`
54a8c4fd2a	2022-07-27	`More improvements in the Databricks operators (#25260)`
7438707747	2022-07-26	`Improved telemetry for Databricks provider (#25115)`
df00436569	2022-07-22	`Unify DbApiHook.run() method with the methods which override it (#23971)`
2f70daf5ac	2022-07-18	`Databricks: fix test_connection implementation (#25114)`

3.1.0¶

Latest change: 2022-07-13

Commit	Committed	Subject
d2459a241b	2022-07-13	`Add documentation for July 2022 Provider's release (#25030)`
8dfe7bf5ff	2022-07-11	`Added databricks_conn_id as templated field (#24945)`
acaa0635c8	2022-07-08	`Automatically detect if non-lazy logging interpolation is used (#24910)`
46bbfdade0	2022-07-07	`Move all SQL classes to common-sql provider (#24836)`
96b01a8012	2022-07-05	`Remove "bad characters" from our codebase (#24841)`
0de31bd73a	2022-06-29	`Move provider dependencies to inside provider folders (#24672)`
510a6bab45	2022-06-28	`Remove 'hook-class-names' from provider.yaml (#24702)`
ed37c3a0e8	2022-06-28	`Add 'test_connection' method to Databricks hook (#24617)`
9c59831ee7	2022-06-21	`Update providers to use functools compat for ''cached_property'' (#24582)`

3.0.0¶

Latest change: 2022-06-09

Commit	Committed	Subject
dcdcf3a2b8	2022-06-09	`Update release notes for RC2 release of Providers for May 2022 (#24307)`
717a7588bc	2022-06-07	`Update package description to remove double min-airflow specification (#24292)`
aeabe994b3	2022-06-07	`Prepare docs for May 2022 provider's release (#24231)`
027b707d21	2022-06-05	`Add explanatory note for contributors about updating Changelog (#24229)`
ddf9013098	2022-06-05	`AIP-47 - Migrate databricks DAGs to new design #22442 (#24203)`
acf89510cd	2022-06-04	`fix: DatabricksSubmitRunOperator and DatabricksRunNowOperator cannot define .json as template_ext (#23622) (#23641)`
92ddcf4ac6	2022-05-27	`Introduce 'flake8-implicit-str-concat' plugin to static checks (#23873)`
6150d28323	2022-05-22	`Add Deferrable Databricks operators (#19736)`
cf5a78e91c	2022-05-22	`Fix UnboundLocalError when sql is empty list in DatabricksSqlHook (#23815)`
d0a5b3a4f2	2022-05-13	`Add git_source to DatabricksSubmitRunOperator (#23620)`

2.7.0¶

Latest change: 2022-05-12

Commit	Committed	Subject
75c60923e0	2022-05-12	`Prepare provider documentation 2022.05.11 (#23631)`
428a439953	2022-05-09	`Clean up in-line f-string concatenation (#23591)`
a58506b2a6	2022-04-26	`Address review comments`
6a3d6cc32b	2022-04-26	`Update to the released version of DBSQL connector`
7b3bf4e435	2022-04-26	`DatabricksSqlOperator - switch to databricks-sql-connector 2.x`
f02b0b6b40	2022-04-25	`Further improvement of Databricks Jobs operators (#23199)`
8b6b0848a3	2022-04-23	`Use new Breese for building, pulling and verifying the images. (#23104)`

2.6.0¶

Latest change: 2022-04-13

Commit	Committed	Subject
40831144be	2022-04-13	`Prepare for RC2 release of March Databricks provider (#22979)`
7be57eb256	2022-04-10	`Databricks SQL operators are now Python 3.10 compatible (#22886)`
aa8c08db38	2022-04-10	`Databricks: Correctly handle HTTP exception (#22885)`
6933022e94	2022-04-10	`Fix new MyPy errors in main (#22884)`
56ab82ed7a	2022-04-07	`Prepare mid-April provider documentation. (#22819)`
1b12c93ed3	2022-03-31	`Refactor 'DatabricksJobRunLink' to not create ad hoc TaskInstances (#22571)`
95169d1d07	2022-03-27	`Add a link to Databricks Job Run (#22541)`
352d7f72dd	2022-03-27	`More operators for Databricks Repos (#22422)`
c063fc688c	2022-03-25	`Update black precommit (#22521)`

2.5.0¶

Latest change: 2022-03-22

Commit	Committed	Subject
d7dbfb7e26	2022-03-22	`Add documentation for bugfix release of Providers (#22383)`
cc920963a6	2022-03-15	`Operator for updating Databricks Repos (#22278)`

2.4.0¶

Latest change: 2022-03-14

Commit	Committed	Subject
16adc035b1	2022-03-14	`Add documentation for Classifier release for March 2022 (#22226)`
12e9e2c695	2022-03-14	`Databricks hook - retry on HTTP Status 429 as well (#21852)`
af9d85ccd8	2022-03-13	`Skip some tests for Databricks from running on Python 3.10 (#22221)`
4014194320	2022-03-08	`Add new options to DatabricksCopyIntoOperator (#22076)`

2.3.0¶

Latest change: 2022-03-07

Commit	Committed	Subject
f5b96315fe	2022-03-07	`Add documentation for Feb Providers release (#22056)`
62bf1276f6	2022-03-01	`Add-showing-runtime-error-feature-to-DatabricksSubmitRunOperator (#21709)`
27d19e7626	2022-02-27	`Databricks SQL operators (#21363)`
a1845c68f9	2022-02-26	`Databricks: add support for triggering jobs by name (#21663)`
7cca82495b	2022-02-15	`Updated Databricks docs for correct jobs 2.1 API and links (#21494)`
0a2d0d1ecb	2022-02-12	`Added template_ext = ('.json') to databricks operators #18925 (#21530)`
d94fa37830	2022-02-08	`Fixed changelog for January 2022 (delayed) provider's release (#21439)`
6c3a67d4fc	2022-02-05	`Add documentation for January 2021 providers release (#21257)`
602abe8394	2022-01-20	`Remove ':type' lines now sphinx-autoapi supports typehints (#20951)`

2.2.0¶

Latest change: 2021-12-31

Commit	Committed	Subject
f77417eb0d	2021-12-31	`Fix K8S changelog to be PyPI-compatible (#20614)`
97496ba2b4	2021-12-31	`Update documentation for provider December 2021 release (#20523)`
0bf424f37f	2021-12-30	`Fix mypy databricks operator (#20598)`
d56e7b56bb	2021-12-30	`Fix template_fields type to have MyPy friendly Sequence type (#20571)`
a0821235fb	2021-12-30	`Use typed Context EVERYWHERE (#20565)`
c5c18c54fa	2021-12-29	`Databricks: fix verification of Managed Identity (#20550)`
d3b3161f0d	2021-12-28	`Remove 'host' as an instance attr in 'DatabricksHook' (#20540)`
58afc19377	2021-12-28	`Add 'wait_for_termination' argument for Databricks Operators (#20536)`
e7659d08b0	2021-12-27	`Update connection object to ''cached_property'' in ''DatabricksHook'' (#20526)`
cad39274d9	2021-12-14	`Fix MyPy Errors for Databricks provider. (#20265)`

2.1.0¶

Latest change: 2021-12-10

Commit	Committed	Subject
820bfed515	2021-12-10	`Prepare docs for provider's RC2 release (#20205)`
66f94f95c2	2021-12-10	`Remove db call from 'DatabricksHook.__init__()' (#20180)`
545ca59ba9	2021-12-08	`Unhide changelog entry for databricks (#20128)`
637db1a0ba	2021-12-07	`Update documentation for RC2 release of November Databricks Provider (#20086)`
728e94a47e	2021-12-05	`Refactor DatabricksHook (#19835)`
4925b37b66	2021-12-04	`Databricks hook: fix expiration time check (#20036)`
853576d901	2021-11-30	`Update documentation for November 2021 provider's release (#19882)`
11998848a4	2021-11-24	`Databricks: add more methods to represent run state information (#19723)`
56bdfe7a84	2021-11-22	`Databricks - allow Azure SP authentication on other Azure clouds (#19722)`
244627e3da	2021-11-14	`Databricks: allow to specify PAT in Password field (#19585)`
0a4a8bdb94	2021-11-14	`Databricks jobs 2.1 (#19544)`
8ae878953b	2021-11-09	`Update Databricks API from 2.0 to 2.1 (#19412)`
28b51fb7bd	2021-11-08	`Authentication with AAD tokens in Databricks provider (#19335)`
3a0c455855	2021-11-07	`Update Databricks operators to match latest version of API 2.0 (#19443)`
d9567eb106	2021-10-29	`Prepare documentation for October Provider's release (#19321)`
f5ad26dcdd	2021-10-21	`Fixup string concatenations (#19099)`

2.0.2¶

Latest change: 2021-09-30

Commit	Committed	Subject
840ea3efb9	2021-09-30	`Update documentation for September providers release (#18613)`
ef037e7021	2021-09-29	`Static start_date and default arg cleanup for misc. provider example DAGs (#18597)`
0b7b13372f	2021-09-18	`Move DB call out of ''DatabricksHook.__init__'' (#18339)`

2.0.1¶

Latest change: 2021-08-30

Commit	Committed	Subject
0a68588479	2021-08-30	`Add August 2021 Provider's documentation (#17890)`
be75dcd39c	2021-08-23	`Update description about the new ''connection-types'' provider meta-data`
76ed2a49c6	2021-08-19	`Import Hooks lazily individually in providers manager (#17682)`
87f408b1e7	2021-07-26	`Prepares docs for Rc2 release of July providers (#17116)`
b916b75079	2021-07-15	`Prepare documentation for July release of providers. (#17015)`
866a601b76	2021-06-28	`Removes pylint from our toolchain (#16682)`

2.0.0¶

Latest change: 2021-06-18

Commit	Committed	Subject
bbc627a3da	2021-06-18	`Prepares documentation for rc2 release of Providers (#16501)`
cbf8001d76	2021-06-16	`Synchronizes updated changelog after buggfix release (#16464)`
1fba5402bb	2021-06-15	`More documentation update for June providers release (#16405)`
9c94b72d44	2021-06-07	`Updated documentation for June 2021 provider release (#16294)`
37681bca00	2021-05-07	`Auto-apply apply_default decorator (#15667)`
807ad32ce5	2021-05-01	`Prepares provider release after PIP 21 compatibility (#15576)`
df143aee8d	2021-04-29	`An initial rework of the "Concepts" docs (#15444)`
49cae1f052	2021-04-17	`Add documentation for Databricks connection (#15410)`
68e4c4dcb0	2021-03-20	`Remove Backport Providers (#14886)`

1.0.1¶

Latest change: 2021-02-04

Commit	Committed	Subject
88bdcfa0df	2021-02-04	`Prepare to release a new wave of providers. (#14013)`
ac2f72c98d	2021-02-01	`Implement provider versioning tools (#13767)`
a9ac2b040b	2021-01-23	`Switch to f-strings using flynt. (#13732)`
3fd5ef3555	2021-01-21	`Add missing logos for integrations (#13717)`
295d66f914	2020-12-30	`Fix Grammar in PIP warning (#13380)`
6cf76d7ac0	2020-12-18	`Fix typo in pip upgrade command :( (#13148)`

1.0.0¶

Latest change: 2020-12-09

Commit	Committed	Subject
32971a1a2d	2020-12-09	`Updates providers versions to 1.0.0 (#12955)`
b40dffa085	2020-12-08	`Rename remaing modules to match AIP-21 (#12917)`
9b39f24780	2020-12-08	`Add support for dynamic connection form fields per provider (#12558)`
bd90136aaf	2020-11-30	`Move operator guides to provider documentation packages (#12681)`
c34ef853c8	2020-11-20	`Separate out documentation building per provider (#12444)`
0080354502	2020-11-18	`Update provider READMEs for 1.0.0b2 batch release (#12449)`
7ca0b6f121	2020-11-18	`Enable Markdownlint rule MD003/heading-style/header-style (#12427) (#12438)`
ae7cb4a1e2	2020-11-17	`Update wrong commit hash in backport provider changes (#12390)`
6889a333cf	2020-11-15	`Improvements for operators and hooks ref docs (#12366)`
7825e8f590	2020-11-13	`Docs installation improvements (#12304)`
b027223132	2020-11-13	`Add install/uninstall api to databricks hook (#12316)`
85a18e13d9	2020-11-09	`Point at pypi project pages for cross-dependency of provider packages (#12212)`
59eb5de78c	2020-11-09	`Update provider READMEs for up-coming 1.0.0beta1 releases (#12206)`
b2a28d1590	2020-11-09	`Moves provider packages scripts to dev (#12082)`
7e0d08e1f0	2020-11-09	`Add how-to Guide for Databricks operators (#12175)`
4e8f9cc8d0	2020-11-03	`Enable Black - Python Auto Formmatter (#9550)`
8c42cf1b00	2020-11-03	`Use PyUpgrade to use Python 3.6 features (#11447)`
5a439e84eb	2020-10-26	`Prepare providers release 0.0.2a1 (#11855)`
872b1566a1	2020-10-25	`Generated backport providers readmes/setup for 2020.10.29 (#11826)`
349b0811c3	2020-10-20	`Add D200 pydocstyle check (#11688)`
16e7129719	2020-10-13	`Added support for provider packages for Airflow 2.0 (#11487)`
0a0e1af800	2020-10-03	`Fix Broken Markdown links in Providers README TOC (#11249)`
ca4238eb4d	2020-10-02	`Fixed month in backport packages to October (#11242)`
5220e4c384	2020-10-02	`Prepare Backport release 2020.09.07 (#11238)`
54353f8745	2020-09-27	`Increase type coverage for five different providers (#11170)`
966a06d96b	2020-09-18	`Fetching databricks host from connection if not supplied in extras. (#10762)`
9549274d11	2020-09-09	`Upgrade black to 20.8b1 (#10818)`
fdd9b6f65b	2020-08-25	`Enable Black on Providers Packages (#10543)`
bfefcce0c9	2020-08-25	`Updated REST API call so GET requests pass payload in query string instead of request body (#10462)`
3696c34c28	2020-08-24	`Fix typo in the word "release" (#10528)`
2f2d8dbfaf	2020-08-25	`Remove all "noinspection" comments native to IntelliJ (#10525)`
ee7ca128a1	2020-08-22	`Fix broken Markdown refernces in Providers README (#10483)`
cdec301254	2020-08-07	`Add correct signature to all operators and sensors (#10205)`
7d24b088cd	2020-07-25	`Stop using start_date in default_args in example_dags (2) (#9985)`
e13a14c873	2020-06-21	`Enable & Fix Whitespace related PyDocStyle Checks (#9458)`
d0e7db4024	2020-06-19	`Fixed release number for fresh release (#9408)`
12af6a0800	2020-06-19	`Final cleanup for 2020.6.23rc1 release preparation (#9404)`
c7e5bce57f	2020-06-19	`Prepare backport release candidate for 2020.6.23rc1 (#9370)`
f6bd817a3a	2020-06-16	`Introduce 'transfers' packages (#9320)`
0b0e4f7a4c	2020-05-26	`Preparing for RC3 relase of backports (#9026)`
00642a46d0	2020-05-26	`Fixed name of 20 remaining wrongly named operators. (#8994)`
f1073381ed	2020-05-22	`Add support for spark python and submit tasks in Databricks operator(#8846)`
375d1ca229	2020-05-19	`Release candidate 2 for backport packages 2020.05.20 (#8898)`
12c5e5d8ae	2020-05-17	`Prepare release candidate for backport packages (#8891)`
f3521fb0e3	2020-05-16	`Regenerate readme files for backport package release (#8886)`
92585ca4cb	2020-05-15	`Added automated release notes generation for backport operators (#8807)`
649935e8ce	2020-04-27	`[AIRFLOW-8472]: 'PATCH' for Databricks hook '_do_api_call' (#8473)`
16903ba3a6	2020-04-24	`[AIRFLOW-8474]: Adding possibility to get job_id from Databricks run (#8475)`
5648dfbc30	2020-03-23	`Add missing call to Super class in 'amazon', 'cloudant & 'databricks' providers (#7827)`
3320e432a1	2020-02-24	`[AIRFLOW-6817] Lazy-load 'airflow.DAG' to keep user-facing API untouched (#7517)`
4d03e33c11	2020-02-22	`[AIRFLOW-6817] remove imports from 'airflow/__init__.py', replaced implicit imports with explicit imports, added entry to 'UPDATING.MD' - squashed/rebased (#7456)`
97a429f9d0	2020-02-02	`[AIRFLOW-6714] Remove magic comments about UTF-8 (#7338)`
83c037873f	2020-01-30	`[AIRFLOW-6674] Move example_dags in accordance with AIP-21 (#7287)`
c42a375e79	2020-01-27	`[AIRFLOW-6644][AIP-21] Move service classes to providers package (#7265)`