Reference for Database Migrations

Here’s the list of all the Database Migrations that are executed via when you run airflow db migrate.

Warning

Those migration details are mostly used here to make the users aware when and what kind of migrations will be executed during migrations between specific Airflow versions. The intention here is that the “DB conscious” users might perform an analysis on the migrations and draw conclusions about the impact of the migrations on their Airflow database. Those users might also want to take a look at the ERD Schema of the Database document to understand how the internal DB of Airflow structure looks like. However, you should be aware that the structure is internal and you should not access the DB directly to retrieve or modify any data - you should use the REST API to do that instead.

Revision ID

Revises ID

Airflow Version

Description

22ed7efa9da2 (head)

8684e37832e6

2.10.0

Add dag_schedule_dataset_alias_reference table.

8684e37832e6

41b3bc7c0272

2.10.0

Add dataset_alias_dataset association table.

41b3bc7c0272

ec3471c1e067

2.10.0

Add try_number to audit log.

ec3471c1e067

05e19f3176be

2.10.0

Add dataset_alias_dataset_event.

05e19f3176be

d482b7261ff9

2.10.0

Add dataset_alias.

d482b7261ff9

c4602ba06b4b

2.10.0

Add task_instance_history.

c4602ba06b4b

677fdbb7fc54

2.10.0

Added DagPriorityParsingRequest table.

677fdbb7fc54

0fd0c178cbe8

2.10.0

add new executor field to db.

0fd0c178cbe8

686269002441

2.10.0

Add indexes on dag_id column in referencing tables.

686269002441

bff083ad727d

2.9.2

Fix inconsistency between ORM and migration files.

bff083ad727d

1949afb29106

2.9.2

Remove idx_last_scheduling_decision index on last_scheduling_decision in dag_run table.

1949afb29106

ee1467d4aa35

2.9.0

update trigger kwargs type and encrypt.

ee1467d4aa35

b4078ac230a1

2.9.0

add display name for dag and task instance.

b4078ac230a1

8e1c784a4fc7

2.9.0

Change value column type to longblob in xcom table for mysql.

8e1c784a4fc7

ab34f260b71c

2.9.0

Adding max_consecutive_failed_dag_runs column to dag_model table.

ab34f260b71c

d75389605139

2.9.0

add dataset_expression in DagModel.

d75389605139

1fd565369930

2.9.0

Add run_id to (Audit) log table and increase event name length.

1fd565369930

88344c1d9134

2.9.0

Add rendered_map_index to TaskInstance.

88344c1d9134

10b52ebd31f7

2.8.1

Drop unused TI index.

10b52ebd31f7

bd5dfbe21f88

2.8.0

Add processor_subdir to ImportError.

bd5dfbe21f88

f7bf2a57d0a6

2.8.0

Make connection login/password TEXT.

f7bf2a57d0a6

375a816bbbf4

2.8.0

Add owner_display_name to (Audit) Log table.

375a816bbbf4

405de8318b3a

2.8.0

add new field ‘clear_number’ to dagrun.

405de8318b3a

788397e78828

2.7.0

add include_deferred column to pool.

788397e78828

937cbd173ca1

2.7.0

Add custom_operator_name column.

937cbd173ca1

c804e5c76e3e

2.7.0

Add index to task_instance table.

c804e5c76e3e

98ae134e6fff

2.6.2

Add onupdate cascade to task_map table.

98ae134e6fff

6abdffdd4815

2.6.0

Increase length of user identifier columns in ab_user and ab_register_user tables.

6abdffdd4815

290244fb8b83

2.6.0

add dttm index on log table.

290244fb8b83

1986afd32c1b

2.5.0

Add is_orphaned to DatasetModel.

1986afd32c1b

ee8d93fcc81e

2.5.0

Add DagRunNote and TaskInstanceNote.

ee8d93fcc81e

e07f49787c9d

2.5.0

Add updated_at column to DagRun and TaskInstance.

e07f49787c9d

b0d31815b5a6

2.4.3

Add case-insensitive unique constraint for username.

b0d31815b5a6

ecb43d2a1842

2.4.2

Add missing auto-increment to columns on FAB tables.

ecb43d2a1842

1486deb605b4

2.4.0

Add processor_subdir column to DagModel, SerializedDagModel and CallbackRequest tables.

1486deb605b4

f4ff391becb5

2.4.0

add dag_owner_attributes table.

f4ff391becb5

0038cd0c28b4

2.4.0

Remove smart sensors.

0038cd0c28b4

44b7034f6bdc

2.4.0

Add Dataset model.

44b7034f6bdc

424117c37d18

2.4.0

compare types between ORM and DB.

424117c37d18

f5fcbda3e651

2.4.0

Add DagWarning model.

f5fcbda3e651

3c94c427fdf6

2.3.3

Add indexes for CASCADE deletes on task_instance.

3c94c427fdf6

1de7bc13c950

2.3.2

Add cascade to dag_tag foreign key.

1de7bc13c950

b1b348e02d07

2.3.1

Add index for event column in log table.

b1b348e02d07

75d5ed6c2b43

2.3.0

Update dag.default_view to grid.

75d5ed6c2b43

909884dea523

2.3.0

Add map_index to Log.

909884dea523

48925b2719cb

2.3.0

Update migration for FAB tables to add missing constraints.

48925b2719cb

4eaab2fe6582

2.3.0

Add map_index to TaskFail.

4eaab2fe6582

c97c2ab6aa23

2.3.0

Migrate RTIF to use run_id and map_index.

c97c2ab6aa23

c306b5b5ae4a

2.3.0

add callback request table.

c306b5b5ae4a

a3bcd0914482

2.3.0

Switch XCom table to use run_id and add map_index.

a3bcd0914482

e655c0453f75

2.3.0

add data_compressed to serialized_dag.

e655c0453f75

f9da662e7089

2.3.0

Add TaskMap and map_index on TaskInstance.

f9da662e7089

786e3737b18f

2.3.0

Add LogTemplate table to track changes to config values log_filename_template.

786e3737b18f

5e3ec427fdd3

2.3.0

Add timetable_description column to DagModel for UI.

5e3ec427fdd3

587bdf053233

2.3.0

Increase length of email and username in ab_user and ab_register_user table to 256 characters.

587bdf053233

c381b21cb7e4

2.2.4

Add index for dag_id column in job table.

c381b21cb7e4

be2bfac3da23

2.2.4

Create a session table to store web session data.

be2bfac3da23

7b2661a43ba3

2.2.3

Add has_import_errors column to DagModel.

7b2661a43ba3

142555e44c17

2.2.0

Change TaskInstance and TaskReschedule tables from execution_date to run_id.

142555e44c17

54bebd308c5f

2.2.0

Add data_interval_[start|end] to DagModel and DagRun.

54bebd308c5f

30867afad44a

2.2.0

Adds trigger table and deferrable operator columns to task instance.

30867afad44a

e9304a3141f0

2.2.0

Rename concurrency column in dag table to`` max_active_tasks``.

e9304a3141f0

83f031fd9f1c

2.2.0

Make XCom primary key columns non-nullable.

83f031fd9f1c

ccde3e26fe78

2.2.0

Improve MSSQL compatibility.

ccde3e26fe78

092435bf5d12

2.1.4

Add index on state, dag_id for queued dagrun.

092435bf5d12

97cdd93827b8

2.1.4

Add max_active_runs column to dag_model table.

97cdd93827b8

a13f7613ad25

2.1.3

Add queued_at column in dag_run table.

a13f7613ad25

e165e7455d70

2.1.0

Resource based permissions for default Flask-AppBuilder views.

e165e7455d70

90d1635d7b86

2.1.0

Add description field to Variable model.

90d1635d7b86

2e42bb497a22

2.1.0

Increase maximum length of pool name in task_instance table to 256 characters.

2e42bb497a22

8646922c8a04

2.0.2

Rename last_scheduler_run column in DAG table to last_parsed_time.

8646922c8a04

449b4072c2da

2.0.2

Change default pool_slots to 1.

449b4072c2da

82b7c48c147f

2.0.2

Increase size of connection.extra field to handle multiple RSA keys.

82b7c48c147f

e959f08ac86c

2.0.1

Remove can_read permission on config resource for User and Viewer role.

e959f08ac86c

64a7d6477aae

2.0.0

Change field in DagCode to MEDIUMTEXT for MySql.

64a7d6477aae

61ec73d9401f

2.0.0

Fix description field in connection to be text.

61ec73d9401f

2c6edca13270

2.0.0

Add description field to connection table.

2c6edca13270

849da589634d

2.0.0

Resource based permissions.

849da589634d

45ba3f1493b9

2.0.0

Prefix DAG permissions.

45ba3f1493b9

364159666cbd

2.0.0

add-k8s-yaml-to-rendered-templates.

364159666cbd

52d53670a240

2.0.0

Add creating_job_id to DagRun table.

52d53670a240

98271e7606e2

2.0.0

fix_mssql_exec_date_rendered_task_instance_fields_for_MSSQL.

98271e7606e2

bef4f3d11e8b

2.0.0

Add scheduling_decision to DagRun and DAG.

bef4f3d11e8b

e1a11ece99cc

2.0.0

Drop KubeResourceVersion and KubeWorkerId.

e1a11ece99cc

b247b1e3d1ed

2.0.0

Add external executor ID to TI.

b247b1e3d1ed

e38be357a868

2.0.0

Add queued by Job ID to TI.

e38be357a868

8d48763f6d53

2.0.0

Add sensor_instance table.

8d48763f6d53

8f966b9c467a

2.0.0

Add unique constraint to conn_id.

8f966b9c467a

3c20cacc0044

2.0.0

Set conn_type as non-nullable.

3c20cacc0044

b25a55525161

2.0.0

Add run_type column in dag_run table.

b25a55525161

bbf4a7ad0465

2.0.0

Increase length of pool name.

bbf4a7ad0465

cf5dc11e79ad

2.0.0

Remove id column from xcom.

cf5dc11e79ad

03afc6b6f902

2.0.0

Drop user and chart table.

03afc6b6f902

92c57b58940d

1.10.13

Increase length of Flask-AppBuilder ab_view_menu.name column.

92c57b58940d

da3f683c3a5a

1.10.13

Create FAB Tables.

da3f683c3a5a

a66efa278eea

1.10.12

Add dag_hash Column to serialized_dag table.

a66efa278eea

952da73b5eff

1.10.11

Add Precision to execution_date in RenderedTaskInstanceFields table.

952da73b5eff

852ae6c715af

1.10.10

Add dag_code table.

852ae6c715af

a4c2fd67d16b

1.10.10

Add RenderedTaskInstanceFields table.

a4c2fd67d16b

7939bcff74ba

1.10.10

Add pool_slots field to task_instance.

7939bcff74ba

fe461863935f

1.10.8

Add DagTags table.

fe461863935f

08364691d074

1.10.7

Increase length for connection password.

08364691d074 (merge_point)

a56c9515abdc, 004c1210f153, 74effc47d867, b3b105409875

1.10.7

Straighten out the migrations.

a56c9515abdc

c8ffec048a3b

1.10.3

Remove dag_stat table.

004c1210f153

939bb1e647c8

1.10.4

Increase queue name size limit.

74effc47d867

6e96a59344a4

1.10.5

change datetime to datetime2(6) on MSSQL tables.

b3b105409875

d38e04c12aa2

1.10.7

Add root_dag_id to DAG.

d38e04c12aa2

6e96a59344a4

1.10.7

Add serialized_dag table.

6e96a59344a4 (branch_point)

939bb1e647c8

1.10.4

Make TaskInstance.pool not nullable.

939bb1e647c8 (branch_point)

dd4ecb8fbee3

1.10.3

task reschedule foreign key on cascade delete.

dd4ecb8fbee3

c8ffec048a3b

1.10.3

Add schedule interval to dag.

c8ffec048a3b (branch_point)

41f5f12752f8

1.10.3

Add description and default_view column to dag table.

41f5f12752f8

03bc53e68815

1.10.2

Add superuser field.

03bc53e68815 (merge_point)

0a2a5b66e19d, bf00311e1990

1.10.2

Merge migrations Heads.

0a2a5b66e19d

9635ae0956e7

1.10.2

Add task_reschedule table.

bf00311e1990

dd25f486b8ea

1.10.2

Add index to task_instance table.

dd25f486b8ea

9635ae0956e7

1.10.2

Add index on log table.

9635ae0956e7 (branch_point)

856955da8476

1.10.0

Create index on task_fail table.

856955da8476

f23433877c24

1.10.0

Fix Sqlite foreign key.

f23433877c24

05f30312d566

1.10.0

Fix MySQL not null constraint.

05f30312d566 (merge_point)

86770d1215c0, 0e2a74e0fc9f

1.10.0

Merge migrations Heads.

86770d1215c0

27c6a30d7c24

1.10.0

Add kubernetes scheduler uniqueness.

27c6a30d7c24

33ae817a1ff4

1.10.0

Add executor_config column to task_instance table.

33ae817a1ff4

d2ae31099d61

1.10.0

Add Kubernetes resource check-pointing.

0e2a74e0fc9f

d2ae31099d61

1.10.0

Add time zone awareness.

d2ae31099d61 (branch_point)

947454bf1dff

1.8.2

Increase text size for MySQL (not relevant for other DBs’ text types).

947454bf1dff

bdaa763e6c56

1.8.2

Create index on job_id column in task_instance table.

bdaa763e6c56

cc1e65623dc7

1.8.2

Make xcom value column a large binary.

cc1e65623dc7

127d2bf2dfa7

1.8.2

Add max_tries column to task_instance.

127d2bf2dfa7

5e7d17757c7a

1.7.1.3

Add dag_id/state index on dag_run table.

5e7d17757c7a

8504051e801b

1.7.1.3

Add pid field to TaskInstance.

8504051e801b

4addfa1236f1

1.7.1.3

Add indices on xcom table.

4addfa1236f1

f2ca10b85618

1.7.1.3

Add fractional seconds to MySQL tables.

f2ca10b85618

64de9cddf6c9

1.7.1.3

Add dag_stats table.

64de9cddf6c9

211e584da130

1.7.1.3

Add task_fail table.

211e584da130

2e82aab8ef20

1.7.1.3

Add TI state index.

2e82aab8ef20

1968acfc09e3

1.7.1

Rename user table.

1968acfc09e3

bba5a7cfc896

1.7.0

Add is_encrypted column to variable table.

bba5a7cfc896

bbc73705a13e

1.7.0

Add a column to track the encryption state of the ‘Extra’ field in connection.

bbc73705a13e

4446e08588

1.7.0

Add notification_sent column to sla_miss table.

4446e08588

561833c1c74b

1.6.2

Add start_date and end_date in dag_run table.

561833c1c74b

40e67319e3a9

1.6.2

Add password column to user table.

40e67319e3a9

2e541a1dcfed

1.6.0

Add conf column in dag_run table.

2e541a1dcfed

1b38cef5b76e

1.6.0

Change task_instance.task_duration type to FLOAT.

1b38cef5b76e

502898887f84

1.6.0

Add dag_run table.

502898887f84

52d714495f0

1.6.0

Adding extra column to Log table.

52d714495f0

338e90f54d61

1.5.2

Add indices in job table.

338e90f54d61

13eb55f81627

1.5.0

Add operator and queued_dttm to task_instance table.

13eb55f81627

1507a7289a2f

1.5.0

Maintain history for compatibility with earlier migrations.

1507a7289a2f

e3a246e0dc1

1.5.0

Add is_encrypted column in connection table.

e3a246e0dc1 (base)

None

1.5.0

current schema.

Was this entry helpful?