Package apache-airflow-providers-apache-spark¶

Apache Spark

This is detailed commit list of changes for versions provider package: apache.spark. For high-level changelog, see package information including changelog.

2.1.2¶

Latest change: 2022-03-14

Commit	Committed	Subject
16adc035b1	2022-03-14	`Add documentation for Classifier release for March 2022 (#22226)`

2.1.1¶

Latest change: 2022-03-07

Commit	Committed	Subject
f5b96315fe	2022-03-07	`Add documentation for Feb Providers release (#22056)`
6322dad2ca	2022-02-25	`fix param rendering in docs of SparkSubmitHook (#21788)`

2.1.0¶

Latest change: 2022-02-08

Commit	Committed	Subject
d94fa37830	2022-02-08	`Fixed changelog for January 2022 (delayed) provider's release (#21439)`
6c3a67d4fc	2022-02-05	`Add documentation for January 2021 providers release (#21257)`
39e395f981	2022-02-04	`Add more SQL template fields renderers (#21237)`
cb73053211	2022-01-27	`Add optional features in providers. (#21074)`
602abe8394	2022-01-20	`Remove ':type' lines now sphinx-autoapi supports typehints (#20951)`

2.0.3¶

Latest change: 2021-12-31

Commit	Committed	Subject
f77417eb0d	2021-12-31	`Fix K8S changelog to be PyPI-compatible (#20614)`
97496ba2b4	2021-12-31	`Update documentation for provider December 2021 release (#20523)`
83f8e178ba	2021-12-31	`Even more typing in operators (template_fields/ext) (#20608)`
d56e7b56bb	2021-12-30	`Fix template_fields type to have MyPy friendly Sequence type (#20571)`
485ff6cc64	2021-12-29	`Fix MyPy errors in Apache Providers (#20422)`
dad2f8103b	2021-12-15	`Fix mypy providers (#20190)`
1a2a2498d6	2021-12-14	`Fix mypy spark hooks (#20290)`
a50d2ac872	2021-12-02	`Ensure Spark driver response is valid before setting UNKNOWN status (#19978)`

2.0.2¶

Latest change: 2021-11-30

Commit	Committed	Subject
853576d901	2021-11-30	`Update documentation for November 2021 provider's release (#19882)`
79b30ff59c	2021-11-24	`fix bug of SparkSql Operator log going to infinite loop. (#19449)`
ae044884d1	2021-11-03	`Cleanup of start_date and default arg use for Apache example DAGs (#18657)`
d9567eb106	2021-10-29	`Prepare documentation for October Provider's release (#19321)`
86a2a19ad2	2021-10-17	`More f-strings (#18855)`
42dc0767b8	2021-10-08	`Remove unnecessary string concatenations in AirflowException messages (#18817)`

2.0.1¶

Latest change: 2021-08-30

Commit	Committed	Subject
0a68588479	2021-08-30	`Add August 2021 Provider's documentation (#17890)`
be75dcd39c	2021-08-23	`Update description about the new ''connection-types'' provider meta-data`
76ed2a49c6	2021-08-19	`Import Hooks lazily individually in providers manager (#17682)`
87f408b1e7	2021-07-26	`Prepares docs for Rc2 release of July providers (#17116)`
91f4d80ff0	2021-07-23	`Updating Apache example DAGs to use XComArgs (#16869)`
b916b75079	2021-07-15	`Prepare documentation for July release of providers. (#17015)`
866a601b76	2021-06-28	`Removes pylint from our toolchain (#16682)`

2.0.0¶

Latest change: 2021-06-18

Commit	Committed	Subject
bbc627a3da	2021-06-18	`Prepares documentation for rc2 release of Providers (#16501)`
cbf8001d76	2021-06-16	`Synchronizes updated changelog after buggfix release (#16464)`
1fba5402bb	2021-06-15	`More documentation update for June providers release (#16405)`
5c86e3d509	2021-06-14	`Make SparkSqlHook use Connection (#15794)`
9c94b72d44	2021-06-07	`Updated documentation for June 2021 provider release (#16294)`
37681bca00	2021-05-07	`Auto-apply apply_default decorator (#15667)`

1.0.3¶

Latest change: 2021-05-01

Commit	Committed	Subject
807ad32ce5	2021-05-01	`Prepares provider release after PIP 21 compatibility (#15576)`
5b2fe0e740	2021-04-27	`Add Connection Documentation for Popular Providers (#15393)`
4b031d39e1	2021-04-27	`Make Airflow code Pylint 2.8 compatible (#15534)`
657384615f	2021-04-27	`Fix 'logging.exception' redundancy (#14823)`
9015beb316	2021-04-10	`Pass environment variables to process with yarn kill command (#15304)`
68e4c4dcb0	2021-03-20	`Remove Backport Providers (#14886)`

1.0.2¶

Latest change: 2021-02-27

Commit	Committed	Subject
589d6dec92	2021-02-27	`Prepare to release the next wave of providers: (#14487)`
f9c9e9c38f	2021-02-11	`Use apache.spark provider without kubernetes (#14187)`

1.0.1¶

Latest change: 2021-02-04

Commit	Committed	Subject
88bdcfa0df	2021-02-04	`Prepare to release a new wave of providers. (#14013)`
ac2f72c98d	2021-02-01	`Implement provider versioning tools (#13767)`
a9ac2b040b	2021-01-23	`Switch to f-strings using flynt. (#13732)`
295d66f914	2020-12-30	`Fix Grammar in PIP warning (#13380)`
6cf76d7ac0	2020-12-18	`Fix typo in pip upgrade command :( (#13148)`
5090fb0c89	2020-12-15	`Add script to generate integrations.json (#13073)`

1.0.0¶

Latest change: 2020-12-09

Commit	Committed	Subject
32971a1a2d	2020-12-09	`Updates providers versions to 1.0.0 (#12955)`
b40dffa085	2020-12-08	`Rename remaing modules to match AIP-21 (#12917)`
9b39f24780	2020-12-08	`Add support for dynamic connection form fields per provider (#12558)`
bd90136aaf	2020-11-30	`Move operator guides to provider documentation packages (#12681)`
c34ef853c8	2020-11-20	`Separate out documentation building per provider (#12444)`
0080354502	2020-11-18	`Update provider READMEs for 1.0.0b2 batch release (#12449)`
ae7cb4a1e2	2020-11-17	`Update wrong commit hash in backport provider changes (#12390)`
6889a333cf	2020-11-15	`Improvements for operators and hooks ref docs (#12366)`
7825e8f590	2020-11-13	`Docs installation improvements (#12304)`
85a18e13d9	2020-11-09	`Point at pypi project pages for cross-dependency of provider packages (#12212)`
59eb5de78c	2020-11-09	`Update provider READMEs for up-coming 1.0.0beta1 releases (#12206)`
b2a28d1590	2020-11-09	`Moves provider packages scripts to dev (#12082)`
4e8f9cc8d0	2020-11-03	`Enable Black - Python Auto Formmatter (#9550)`
8c42cf1b00	2020-11-03	`Use PyUpgrade to use Python 3.6 features (#11447)`
5a439e84eb	2020-10-26	`Prepare providers release 0.0.2a1 (#11855)`
872b1566a1	2020-10-25	`Generated backport providers readmes/setup for 2020.10.29 (#11826)`
349b0811c3	2020-10-20	`Add D200 pydocstyle check (#11688)`
16e7129719	2020-10-13	`Added support for provider packages for Airflow 2.0 (#11487)`
d305876bee	2020-10-12	`Remove redundant None provided as default to dict.get() (#11448)`
0a0e1af800	2020-10-03	`Fix Broken Markdown links in Providers README TOC (#11249)`
ca4238eb4d	2020-10-02	`Fixed month in backport packages to October (#11242)`
5220e4c384	2020-10-02	`Prepare Backport release 2020.09.07 (#11238)`
f3e87c5030	2020-09-22	`Add D202 pydocstyle check (#11032)`
fdd9b6f65b	2020-08-25	`Enable Black on Providers Packages (#10543)`
d760265452	2020-08-25	`PyDocStyle: No whitespaces allowed surrounding docstring text (#10533)`
d1bce91bb2	2020-08-25	`PyDocStyle: Enable D403: Capitalized first word of docstring (#10530)`
3696c34c28	2020-08-24	`Fix typo in the word "release" (#10528)`
ee7ca128a1	2020-08-22	`Fix broken Markdown refernces in Providers README (#10483)`
7c206a82a6	2020-08-22	`Replace assigment with Augmented assignment (#10468)`
3b3287d7ac	2020-08-05	`Enforce keyword only arguments on apache operators (#10170)`
7d24b088cd	2020-07-25	`Stop using start_date in default_args in example_dags (2) (#9985)`
33f0cd2657	2020-07-22	`apply_default keeps the function signature for mypy (#9784)`
1427e4acb4	2020-07-22	`Update Spark submit operator for Spark 3 support (#8730)`
4d74ac2111	2020-07-19	`Increase typing for Apache and http provider package (#9729)`
0873070e08	2020-07-11	`Mask other forms of password arguments in SparkSubmitOperator (#9615)`
13a827d80f	2020-07-09	`Ensure Kerberos token is valid in SparkSubmitOperator before running 'yarn kill' (#9044)`
067806d598	2020-06-29	`Add tests for spark_jdbc_script (#9491)`
d0e7db4024	2020-06-19	`Fixed release number for fresh release (#9408)`
12af6a0800	2020-06-19	`Final cleanup for 2020.6.23rc1 release preparation (#9404)`
c7e5bce57f	2020-06-19	`Prepare backport release candidate for 2020.6.23rc1 (#9370)`
40bf8f28f9	2020-06-18	`Detect automatically the lack of reference to the guide in the operator descriptions (#9290)`
f6bd817a3a	2020-06-16	`Introduce 'transfers' packages (#9320)`
0b0e4f7a4c	2020-05-26	`Preparing for RC3 relase of backports (#9026)`
00642a46d0	2020-05-26	`Fixed name of 20 remaining wrongly named operators. (#8994)`
375d1ca229	2020-05-19	`Release candidate 2 for backport packages 2020.05.20 (#8898)`
12c5e5d8ae	2020-05-17	`Prepare release candidate for backport packages (#8891)`
f3521fb0e3	2020-05-16	`Regenerate readme files for backport package release (#8886)`
92585ca4cb	2020-05-15	`Added automated release notes generation for backport operators (#8807)`
7506c73f17	2020-05-10	`Add default 'conf' parameter to Spark JDBC Hook (#8787)`
487b5cc50c	2020-05-06	`Add guide for Apache Spark operators (#8305)`
87969a350d	2020-04-09	`[AIRFLOW-6515] Change Log Levels from Info/Warn to Error (#8170)`
be1451b0e1	2020-04-02	`[AIRFLOW-7026] Improve SparkSqlHook's error message (#7749)`
4bde99f132	2020-03-23	`Make airflow/providers pylint compatible (#7802)`
7e6372a681	2020-03-23	`Add call to Super call in apache providers (#7820)`
2327aa5a26	2020-03-12	`[AIRFLOW-7025] Fix SparkSqlHook.run_query to handle its parameter properly (#7677)`
024b4bf962	2020-03-10	`[AIRFLOW-7024] Add the verbose parameter support to SparkSqlOperator (#7676)`
b59042b5ab	2020-02-28	`[AIRFLOW-6949] Respect explicit 'spark.kubernetes.namespace' conf to SparkSubmitOperator (#7575)`
97a429f9d0	2020-02-02	`[AIRFLOW-6714] Remove magic comments about UTF-8 (#7338)`
0481b9a957	2020-01-12	`[AIRFLOW-6539][AIP-21] Move Apache classes to providers.apache package (#7142)`