Package apache-airflow-providers-apache-spark

Apache Spark

This is detailed commit list of changes for versions provider package: apache.spark. For high-level changelog, see package information including changelog.

4.7.1

Latest change: 2024-01-26

Commit

Committed

Subject

0b680c9492

2024-01-26

Revert "Provide the logger_name param in providers hooks in order to override the logger name (#36675)" (#37015)

2b4da0101f

2024-01-22

Prepare docs 2nd wave of Providers January 2024 (#36945)

c439ab87c4

2024-01-10

Standardize airflow build process and switch to Hatchling build backend (#36537)

6bd450da1e

2024-01-10

Provide the logger_name param in providers hooks in order to override the logger name (#36675)

ded01a5aba

2024-01-08

Bump min version for grpcio-status in spark provider (#36662)

4.7.0

Latest change: 2024-01-07

Commit

Committed

Subject

19ebcac239

2024-01-07

Prepare docs 1st wave of Providers January 2024 (#36640)

6937ae7647

2023-12-30

Speed up autocompletion of Breeze by simplifying provider state (#36499)

ed9080ae6a

2023-12-29

change spark connection form and add spark connections docs (#36419)

4.6.0

Latest change: 2023-12-23

Commit

Committed

Subject

b15d5578da

2023-12-23

Re-apply updated version numbers to 2nd wave of providers in December (#36380)

f5883d6e7b

2023-12-23

Prepare 2nd wave of providers in December (#36373)

1b4a7edc54

2023-12-13

SparkSubmit Connection Extras can be overridden (#36151)

195abf8f71

2023-12-11

SparkSubmit: Adding propertyfiles option (#36164)

cd476acd8f

2023-12-11

Follow BaseHook connection fields method signature in child classes (#36086)

4.5.0

Latest change: 2023-12-08

Commit

Committed

Subject

999b70178a

2023-12-08

Prepare docs 1st wave of Providers December 2023 (#36112)

d0918d77ee

2023-12-07

Bump minimum Airflow version in providers to Airflow 2.6.0 (#36017)

0b23d5601c

2023-11-24

Prepare docs 2nd wave of Providers November 2023 (#35836)

99534e47f3

2023-11-19

Use reproducible builds for provider packages (#35693)

99df205f42

2023-11-16

Fix and reapply templates for provider documentation (#35686)

9103ea1702

2023-11-16

Add support for Spark Connect to pyspark decorator (#35665)

4.4.0

Latest change: 2023-11-08

Commit

Committed

Subject

1b059c57d6

2023-11-08

Prepare docs 1st wave of Providers November 2023 (#35537)

3b8db612ff

2023-11-07

Add information about Qubole removal and make it possible to release it (#35492)

0a4ed7d557

2023-11-01

Add pyspark decorator (#35247)

880a85bbb7

2023-11-01

Add use_krb5ccache option to SparkSubmitOperator (#35331)

4.3.0

Latest change: 2023-10-28

Commit

Committed

Subject

d1c58d86de

2023-10-28

Prepare docs 3rd wave of Providers October 2023 - FIX (#35233)

3592ff4046

2023-10-28

Prepare docs 3rd wave of Providers October 2023 (#35187)

645d52f129

2023-10-21

Add 'use_krb5ccache' option to 'SparkSubmitHook' (#34386)

dd7ba3cae1

2023-10-19

Pre-upgrade 'ruff==0.0.292' changes in providers (#35053)

b75f9e8806

2023-10-18

Upgrade pre-commits (#35033)

4.2.0

Latest change: 2023-10-13

Commit

Committed

Subject

e9987d5059

2023-10-13

Prepare docs 1st wave of Providers in October 2023 (#34916)

0c8e30e43b

2023-10-05

Bump min airflow version of providers (#34728)

7ebf4220c9

2023-09-28

Refactor usage of str() in providers (#34320)

4.1.5

Latest change: 2023-09-08

Commit

Committed

Subject

21990ed894

2023-09-08

Prepare docs for 09 2023 - 1st wave of Providers (#34201)

a7310f9c91

2023-09-05

Refactor regex in providers (#33898)

4.1.4

Latest change: 2023-08-26

Commit

Committed

Subject

c077d19060

2023-08-26

Prepare docs for Aug 2023 3rd wave of Providers (#33730)

32feab4100

2023-08-22

Refactor: Simplify code in Apache/Alibaba providers (#33227)

c645d8e40c

2023-08-12

D401 Support - Providers: Airbyte to Atlassian (Inclusive) (#33354)

4.1.3

Latest change: 2023-08-05

Commit

Committed

Subject

60677b0ba3

2023-08-05

Prepare docs for Aug 2023 1st wave of Providers (#33128)

4f83e831d2

2023-07-31

Validate conn_prefix in extra field for Spark JDBC hook (#32946)

4.1.2

Latest change: 2023-07-29

Commit

Committed

Subject

d06b7af69a

2023-07-29

Prepare docs for July 2023 3rd wave of Providers (#32875)

e93460383f

2023-07-26

Move all k8S classes to cncf.kubernetes provider (#32767)

225e3041d2

2023-07-06

Prepare docs for July 2023 wave of Providers (RC2) (#32381)

3878fe6fab

2023-07-05

Remove spurious headers for provider changelogs (#32373)

cb4927a018

2023-07-05

Prepare docs for July 2023 wave of Providers (#32298)

8c37b74a20

2023-06-28

D205 Support - Providers: Apache to Common (inclusive) (#32226)

09d4718d3a

2023-06-27

Improve provider documentation and README structure (#32125)

4.1.1

Latest change: 2023-06-20

Commit

Committed

Subject

79bcc2e668

2023-06-20

Prepare RC1 docs for June 2023 wave of Providers (#32001)

8b146152d6

2023-06-20

Add note about dropping Python 3.7 for providers (#32015)

6becb70316

2023-06-16

SparkSubmitOperator: rename spark_conn_id to conn_id (#31952)

13890788ae

2023-06-07

Apache provider docstring improvements (#31730)

9276310a43

2023-06-05

Improve docstrings in providers (#31681)

a473facf6c

2023-06-01

Add D400 pydocstyle check - Apache providers only (#31424)

4.1.0

Latest change: 2023-05-19

Commit

Committed

Subject

45548b9451

2023-05-19

Prepare RC2 docs for May 2023 wave of Providers (#31416)

abea189022

2023-05-18

Use '__version__' in providers not 'version' (#31393)

f5aed58d9f

2023-05-18

Fixing circular import error in providers caused by airflow version check (#31379)

d9ff55cf6d

2023-05-16

Prepare docs for May 2023 wave of Providers (#31252)

eef5bc7f16

2023-05-03

Add full automation for min Airflow version for providers (#30994)

a7eb32a5b2

2023-04-30

Bump minimum Airflow version in providers (#30917)

d23a3bbed8

2023-04-04

Add mechanism to suspend providers (#30422)

4.0.1

Latest change: 2023-04-02

Commit

Committed

Subject

55dbf1ff1f

2023-04-02

Prepare docs for April 2023 wave of Providers (#30378)

5d1f201bb0

2023-03-22

Only restrict spark binary passed via extra (#30213)

d9dea5ce17

2023-03-22

Validate host and schema for Spark JDBC Hook (#30223)

b3259877fa

2023-03-15

Add spark3-submit to list of allowed spark-binary values (#30068)

4.0.0

Latest change: 2022-11-15

Commit

Committed

Subject

12c3c39d1a

2022-11-15

pRepare docs for November 2022 wave of Providers (#27613)

9358928815

2022-11-13

Remove custom spark home and custom binarires for spark (#27646)

78b8ea2f22

2022-10-24

Move min airflow version to 2.3.0 for all providers (#27196)

2a34dc9e84

2022-10-23

Enable string normalization in python formatting - providers (#27205)

f8db64c35c

2022-09-28

Update docs for September Provider's release (#26731)

06acf40a43

2022-09-13

Apply PEP-563 (Postponed Evaluation of Annotations) to non-core airflow (#26289)

e5ac6c7cfb

2022-08-10

Prepare docs for new providers release (August 2022) (#25618)

d2459a241b

2022-07-13

Add documentation for July 2022 Provider's release (#25030)

0de31bd73a

2022-06-29

Move provider dependencies to inside provider folders (#24672)

510a6bab45

2022-06-28

Remove 'hook-class-names' from provider.yaml (#24702)

3.0.0

Latest change: 2022-06-09

Commit

Committed

Subject

dcdcf3a2b8

2022-06-09

Update release notes for RC2 release of Providers for May 2022 (#24307)

717a7588bc

2022-06-07

Update package description to remove double min-airflow specification (#24292)

aeabe994b3

2022-06-07

Prepare docs for May 2022 provider's release (#24231)

b4a5783a2a

2022-06-06

chore: Refactoring and Cleaning Apache Providers (#24219)

9dc2851671

2022-06-05

Fix backwards-compatibility introduced by fixing mypy problems (#24230)

027b707d21

2022-06-05

Add explanatory note for contributors about updating Changelog (#24229)

a2bfc0e62d

2022-06-05

AIP-47 - Migrate spark DAGs to new design #22439 (#24210)

71e4deb1b0

2022-05-16

Add typing for airflow/configuration.py (#23716)

2.1.3

Latest change: 2022-03-22

Commit

Committed

Subject

d7dbfb7e26

2022-03-22

Add documentation for bugfix release of Providers (#22383)

2.1.2

Latest change: 2022-03-14

Commit

Committed

Subject

16adc035b1

2022-03-14

Add documentation for Classifier release for March 2022 (#22226)

2.1.1

Latest change: 2022-03-07

Commit

Committed

Subject

f5b96315fe

2022-03-07

Add documentation for Feb Providers release (#22056)

6322dad2ca

2022-02-25

fix param rendering in docs of SparkSubmitHook (#21788)

2.1.0

Latest change: 2022-02-08

Commit

Committed

Subject

d94fa37830

2022-02-08

Fixed changelog for January 2022 (delayed) provider's release (#21439)

6c3a67d4fc

2022-02-05

Add documentation for January 2021 providers release (#21257)

39e395f981

2022-02-04

Add more SQL template fields renderers (#21237)

cb73053211

2022-01-27

Add optional features in providers. (#21074)

602abe8394

2022-01-20

Remove ':type' lines now sphinx-autoapi supports typehints (#20951)

2.0.3

Latest change: 2021-12-31

Commit

Committed

Subject

f77417eb0d

2021-12-31

Fix K8S changelog to be PyPI-compatible (#20614)

97496ba2b4

2021-12-31

Update documentation for provider December 2021 release (#20523)

83f8e178ba

2021-12-31

Even more typing in operators (template_fields/ext) (#20608)

d56e7b56bb

2021-12-30

Fix template_fields type to have MyPy friendly Sequence type (#20571)

485ff6cc64

2021-12-29

Fix MyPy errors in Apache Providers (#20422)

dad2f8103b

2021-12-15

Fix mypy providers (#20190)

1a2a2498d6

2021-12-14

Fix mypy spark hooks (#20290)

a50d2ac872

2021-12-02

Ensure Spark driver response is valid before setting UNKNOWN status (#19978)

2.0.2

Latest change: 2021-11-30

Commit

Committed

Subject

853576d901

2021-11-30

Update documentation for November 2021 provider's release (#19882)

79b30ff59c

2021-11-24

fix bug of SparkSql Operator log  going to infinite loop. (#19449)

ae044884d1

2021-11-03

Cleanup of start_date and default arg use for Apache example DAGs (#18657)

d9567eb106

2021-10-29

Prepare documentation for October Provider's release (#19321)

86a2a19ad2

2021-10-17

More f-strings (#18855)

42dc0767b8

2021-10-08

Remove unnecessary string concatenations in AirflowException messages (#18817)

2.0.1

Latest change: 2021-08-30

Commit

Committed

Subject

0a68588479

2021-08-30

Add August 2021 Provider's documentation (#17890)

be75dcd39c

2021-08-23

Update description about the new ''connection-types'' provider meta-data

76ed2a49c6

2021-08-19

Import Hooks lazily individually in providers manager (#17682)

87f408b1e7

2021-07-26

Prepares docs for Rc2 release of July providers (#17116)

91f4d80ff0

2021-07-23

Updating Apache example DAGs to use XComArgs (#16869)

b916b75079

2021-07-15

Prepare documentation for July release of providers. (#17015)

866a601b76

2021-06-28

Removes pylint from our toolchain (#16682)

2.0.0

Latest change: 2021-06-18

Commit

Committed

Subject

bbc627a3da

2021-06-18

Prepares documentation for rc2 release of Providers (#16501)

cbf8001d76

2021-06-16

Synchronizes updated changelog after buggfix release (#16464)

1fba5402bb

2021-06-15

More documentation update for June providers release (#16405)

5c86e3d509

2021-06-14

Make SparkSqlHook use Connection (#15794)

9c94b72d44

2021-06-07

Updated documentation for June 2021 provider release (#16294)

37681bca00

2021-05-07

Auto-apply apply_default decorator (#15667)

1.0.3

Latest change: 2021-05-01

Commit

Committed

Subject

807ad32ce5

2021-05-01

Prepares provider release after PIP 21 compatibility (#15576)

5b2fe0e740

2021-04-27

Add Connection Documentation for Popular Providers (#15393)

4b031d39e1

2021-04-27

Make Airflow code Pylint 2.8 compatible (#15534)

657384615f

2021-04-27

Fix 'logging.exception' redundancy (#14823)

9015beb316

2021-04-10

Pass environment variables to process with yarn kill command (#15304)

68e4c4dcb0

2021-03-20

Remove Backport Providers (#14886)

1.0.2

Latest change: 2021-02-27

Commit

Committed

Subject

589d6dec92

2021-02-27

Prepare to release the next wave of providers: (#14487)

f9c9e9c38f

2021-02-11

Use apache.spark provider without kubernetes (#14187)

1.0.1

Latest change: 2021-02-04

Commit

Committed

Subject

88bdcfa0df

2021-02-04

Prepare to release a new wave of providers. (#14013)

ac2f72c98d

2021-02-01

Implement provider versioning tools (#13767)

a9ac2b040b

2021-01-23

Switch to f-strings using flynt. (#13732)

295d66f914

2020-12-30

Fix Grammar in PIP warning (#13380)

6cf76d7ac0

2020-12-18

Fix typo in pip upgrade command :( (#13148)

5090fb0c89

2020-12-15

Add script to generate integrations.json (#13073)

1.0.0

Latest change: 2020-12-09

Commit

Committed

Subject

32971a1a2d

2020-12-09

Updates providers versions to 1.0.0 (#12955)

b40dffa085

2020-12-08

Rename remaing modules to match AIP-21 (#12917)

9b39f24780

2020-12-08

Add support for dynamic connection form fields per provider (#12558)

bd90136aaf

2020-11-30

Move operator guides to provider documentation packages (#12681)

c34ef853c8

2020-11-20

Separate out documentation building per provider  (#12444)

0080354502

2020-11-18

Update provider READMEs for 1.0.0b2 batch release (#12449)

ae7cb4a1e2

2020-11-17

Update wrong commit hash in backport provider changes (#12390)

6889a333cf

2020-11-15

Improvements for operators and hooks ref docs (#12366)

7825e8f590

2020-11-13

Docs installation improvements (#12304)

85a18e13d9

2020-11-09

Point at pypi project pages for cross-dependency of provider packages (#12212)

59eb5de78c

2020-11-09

Update provider READMEs for up-coming 1.0.0beta1 releases (#12206)

b2a28d1590

2020-11-09

Moves provider packages scripts to dev (#12082)

4e8f9cc8d0

2020-11-03

Enable Black - Python Auto Formmatter (#9550)

8c42cf1b00

2020-11-03

Use PyUpgrade to use Python 3.6 features (#11447)

5a439e84eb

2020-10-26

Prepare providers release 0.0.2a1 (#11855)

872b1566a1

2020-10-25

Generated backport providers readmes/setup for 2020.10.29 (#11826)

349b0811c3

2020-10-20

Add D200 pydocstyle check (#11688)

16e7129719

2020-10-13

Added support for provider packages for Airflow 2.0 (#11487)

d305876bee

2020-10-12

Remove redundant None provided as default to dict.get() (#11448)

0a0e1af800

2020-10-03

Fix Broken Markdown links in Providers README TOC (#11249)

ca4238eb4d

2020-10-02

Fixed month in backport packages to October (#11242)

5220e4c384

2020-10-02

Prepare Backport release 2020.09.07 (#11238)

f3e87c5030

2020-09-22

Add D202 pydocstyle check (#11032)

fdd9b6f65b

2020-08-25

Enable Black on Providers Packages (#10543)

d760265452

2020-08-25

PyDocStyle: No whitespaces allowed surrounding docstring text (#10533)

d1bce91bb2

2020-08-25

PyDocStyle: Enable D403: Capitalized first word of docstring (#10530)

3696c34c28

2020-08-24

Fix typo in the word "release" (#10528)

ee7ca128a1

2020-08-22

Fix broken Markdown refernces in Providers README (#10483)

7c206a82a6

2020-08-22

Replace assigment with Augmented assignment (#10468)

3b3287d7ac

2020-08-05

Enforce keyword only arguments on apache operators (#10170)

7d24b088cd

2020-07-25

Stop using start_date in default_args in example_dags (2) (#9985)

33f0cd2657

2020-07-22

apply_default keeps the function signature for mypy (#9784)

1427e4acb4

2020-07-22

Update Spark submit operator for Spark 3 support (#8730)

4d74ac2111

2020-07-19

Increase typing for Apache and http provider package (#9729)

0873070e08

2020-07-11

Mask other forms of password arguments in SparkSubmitOperator (#9615)

13a827d80f

2020-07-09

Ensure Kerberos token is valid in SparkSubmitOperator before running 'yarn kill' (#9044)

067806d598

2020-06-29

Add tests for spark_jdbc_script (#9491)

d0e7db4024

2020-06-19

Fixed release number for fresh release (#9408)

12af6a0800

2020-06-19

Final cleanup for 2020.6.23rc1 release preparation (#9404)

c7e5bce57f

2020-06-19

Prepare backport release candidate for 2020.6.23rc1 (#9370)

40bf8f28f9

2020-06-18

Detect automatically the lack of reference to the guide in the operator descriptions (#9290)

f6bd817a3a

2020-06-16

Introduce 'transfers' packages (#9320)

0b0e4f7a4c

2020-05-26

Preparing for RC3 relase of backports (#9026)

00642a46d0

2020-05-26

Fixed name of 20 remaining wrongly named operators. (#8994)

375d1ca229

2020-05-19

Release candidate 2 for backport packages 2020.05.20 (#8898)

12c5e5d8ae

2020-05-17

Prepare release candidate for backport packages (#8891)

f3521fb0e3

2020-05-16

Regenerate readme files for backport package release (#8886)

92585ca4cb

2020-05-15

Added automated release notes generation for backport operators (#8807)

7506c73f17

2020-05-10

Add default 'conf' parameter to Spark JDBC Hook (#8787)

487b5cc50c

2020-05-06

Add guide for Apache Spark operators (#8305)

87969a350d

2020-04-09

[AIRFLOW-6515] Change Log Levels from Info/Warn to Error (#8170)

be1451b0e1

2020-04-02

[AIRFLOW-7026] Improve SparkSqlHook's error message (#7749)

4bde99f132

2020-03-23

Make airflow/providers pylint compatible (#7802)

7e6372a681

2020-03-23

Add call to Super call in apache providers (#7820)

2327aa5a26

2020-03-12

[AIRFLOW-7025] Fix SparkSqlHook.run_query to handle its parameter properly (#7677)

024b4bf962

2020-03-10

[AIRFLOW-7024] Add the verbose parameter support to SparkSqlOperator (#7676)

b59042b5ab

2020-02-28

[AIRFLOW-6949] Respect explicit 'spark.kubernetes.namespace' conf to SparkSubmitOperator (#7575)

97a429f9d0

2020-02-02

[AIRFLOW-6714] Remove magic comments about UTF-8 (#7338)

0481b9a957

2020-01-12

[AIRFLOW-6539][AIP-21] Move Apache classes to providers.apache package (#7142)

Was this entry helpful?