airflow.providers.apache.kylin.operators.kylin_cube¶
Module Contents¶
-
class
airflow.providers.apache.kylin.operators.kylin_cube.KylinCubeOperator(*, kylin_conn_id: str = 'kylin_default', project: Optional[str] = None, cube: Optional[str] = None, dsn: Optional[str] = None, command: Optional[str] = None, start_time: Optional[str] = None, end_time: Optional[str] = None, offset_start: Optional[str] = None, offset_end: Optional[str] = None, segment_name: Optional[str] = None, is_track_job: bool = False, interval: int = 60, timeout: int = 60 * 60 * 24, eager_error_status=('ERROR', 'DISCARDED', 'KILLED', 'SUICIDAL', 'STOPPED'), **kwargs)[source]¶ Bases:
airflow.models.BaseOperatorThis operator is used to submit request about kylin build/refresh/merge, and can track job status . so users can easier to build kylin job
For more detail information in Apache Kylin
- Parameters
kylin_conn_id (str) -- The connection id as configured in Airflow administration.
project (str) -- kylin project name, this param will overwrite the project in kylin_conn_id:
cube (str) -- kylin cube name
dsn (str) -- (dsn , dsn url of kylin connection ,which will overwrite kylin_conn_id. for example: kylin://ADMIN:KYLIN@sandbox/learn_kylin?timeout=60&is_debug=1)
command (str) -- (kylin command include 'build', 'merge', 'refresh', 'delete', 'build_streaming', 'merge_streaming', 'refresh_streaming', 'disable', 'enable', 'purge', 'clone', 'drop'. build - use /kylin/api/cubes/{cubeName}/build rest api,and buildType is ‘BUILD’, and you should give start_time and end_time refresh - use build rest api,and buildType is ‘REFRESH’ merge - use build rest api,and buildType is ‘MERGE’ build_streaming - use /kylin/api/cubes/{cubeName}/build2 rest api,and buildType is ‘BUILD’ and you should give offset_start and offset_end refresh_streaming - use build2 rest api,and buildType is ‘REFRESH’ merge_streaming - use build2 rest api,and buildType is ‘MERGE’ delete - delete segment, and you should give segment_name value disable - disable cube enable - enable cube purge - purge cube clone - clone cube,new cube name is {cube_name}_clone drop - drop cube)
start_time (Optional[str]) -- build segment start time
end_time (Optional[str]) -- build segment end time
offset_start (Optional[str]) -- streaming build segment start time
offset_end (Optional[str]) -- streaming build segment end time
segment_name (str) -- segment name
is_track_job (bool) -- (whether to track job status. if value is True,will track job until job status is in("FINISHED", "ERROR", "DISCARDED", "KILLED", "SUICIDAL", "STOPPED") or timeout)
interval (int) -- track job status,default value is 60s
timeout (int) -- timeout value,default value is 1 day,60 * 60 * 24 s
eager_error_status (tuple) -- (jobs error status,if job status in this list ,this task will be error. default value is tuple(["ERROR", "DISCARDED", "KILLED", "SUICIDAL", "STOPPED"]))