• Home
• airflow.providers.http
• airflow.providers.http.hooks
• airflow.providers.http.hooks.http

↑↓ Navigate ⏎ Select Esc Close

airflow.providers.http.hooks.http

Classes

HttpHook	Interact with HTTP servers.
SessionConfig	Configuration container for an asynchronous HTTP session.
AsyncHttpSession	Wrapper around an aiohttp.ClientSession providing a session bound HttpAsyncHook.
HttpAsyncHook	Interact with HTTP servers asynchronously.

Module Contents

class airflow.providers.http.hooks.http.HttpHook(method='POST', http_conn_id=default_conn_name, auth_type=None, tcp_keep_alive=True, tcp_keep_alive_idle=120, tcp_keep_alive_count=20, tcp_keep_alive_interval=30, adapter=None)

Bases: airflow.providers.common.compat.sdk.BaseHook

Interact with HTTP servers.

Parameters:

• method (str) – the API method to be called

• http_conn_id (str) – http connection that has the base API url i.e https://www.google.com/ and optional authentication credentials. Default headers can also be specified in the Extra field in json format.

• auth_type (Any) – The auth type for the service

• adapter (requests.adapters.HTTPAdapter | None) – An optional instance of requests.adapters.HTTPAdapter to mount for the session.

• tcp_keep_alive (bool) – Enable TCP Keep Alive for the connection.

• tcp_keep_alive_idle (int) – The TCP Keep Alive Idle parameter (corresponds to socket.TCP_KEEPIDLE).

• tcp_keep_alive_count (int) – The TCP Keep Alive count parameter (corresponds to socket.TCP_KEEPCNT)

• tcp_keep_alive_interval (int) – The TCP Keep Alive interval parameter (corresponds to socket.TCP_KEEPINTVL)

conn_name_attr = 'http_conn_id'

default_conn_name = 'http_default'

conn_type = 'http'

hook_name = 'HTTP'

default_host = ''

default_headers: dict[str, str]

http_conn_id = 'http_default'

method = ''

base_url: str = ''

adapter = None

merged_extra: dict

property auth_type

get_conn(headers=None, extra_options=None)

Create a Requests HTTP session.

Parameters:

• headers (dict[Any, Any] | None) – Additional headers to be passed through as a dictionary.

• extra_options (dict[str, Any] | None) – additional options to be used when executing the request

Returns:

A configured requests.Session object.

Return type:

requests.Session

run(endpoint=None, data=None, headers=None, extra_options=None, **request_kwargs)

Perform the request.

Parameters:

• endpoint (str | None) – the endpoint to be called i.e. resource/v1/query?

• data (dict[str, Any] | str | None) – payload to be uploaded or request parameters

• headers (dict[str, Any] | None) – additional headers to be passed through as a dictionary

• extra_options (dict[str, Any] | None) – additional options to be used when executing the request i.e. {‘check_response’: False} to avoid checking raising exceptions on non 2XX or 3XX status codes

• request_kwargs (Any) – Additional kwargs to pass when creating a request. For example, run(json=obj) is passed as requests.Request(json=obj)

check_response(response)

Check the status code and raise on failure.

Parameters:

response (requests.Response) – A requests response object.

Raises:

AirflowException – If the response contains a status code not in the 2xx and 3xx range.

run_and_check(session, prepped_request, extra_options)

Grab extra options, actually run the request, and check the result.

Parameters:

• session (requests.Session) – the session to be used to execute the request

• prepped_request (requests.PreparedRequest) – the prepared request generated in run()

• extra_options (dict[Any, Any]) – additional options to be used when executing the request i.e. {'check_response': False} to avoid checking raising exceptions on non 2XX or 3XX status codes

run_with_advanced_retry(_retry_args, *args, **kwargs)

Run the hook with retry.

This is useful for connectors which might be disturbed by intermittent issues and should not instantly fail.

Parameters:

_retry_args (dict[Any, Any]) – Arguments which define the retry behaviour. See Tenacity documentation at https://github.com/jd/tenacity

hook = HttpHook(http_conn_id="my_conn", method="GET")
retry_args = dict(
    wait=tenacity.wait_exponential(),
    stop=tenacity.stop_after_attempt(10),
    retry=tenacity.retry_if_exception_type(Exception),
)
hook.run_with_advanced_retry(endpoint="v1/test", _retry_args=retry_args)

url_from_endpoint(endpoint)

Combine base url with endpoint.

test_connection()

Test HTTP Connection.

class airflow.providers.http.hooks.http.SessionConfig(/, **data)

Bases: pydantic.BaseModel

Configuration container for an asynchronous HTTP session.

base_url: str

headers: dict[str, Any] | None = None

auth: aiohttp.BasicAuth | None = None

extra_options: dict[str, Any] | None = None

class airflow.providers.http.hooks.http.AsyncHttpSession(hook, request, config, method=None)

Bases: airflow.utils.log.logging_mixin.LoggingMixin

Wrapper around an aiohttp.ClientSession providing a session bound HttpAsyncHook.

This class binds an asynchronous HTTP client session to an HttpAsyncHook and applies connection configuration, authentication, headers, and retry logic consistently across requests. A single AsyncHttpSession instance is intended to be used for multiple HTTP calls within the same logical session.

Parameters:

• hook (HttpAsyncHook) – The HttpAsyncHook instance that owns this session and provides connection-level behavior such as retries and logging.

• request (collections.abc.Callable[Ellipsis, collections.abc.Awaitable[aiohttp.client_reqrep.ClientResponse]]) – A callable used to perform the underlying HTTP request. This is typically a bound aiohttp.ClientSession request method.

• config (SessionConfig) – Resolved session configuration containing base URL, headers, and authentication settings.

method

config

property http_conn_id: str

property base_url: str

property retry_limit: int

property retry_delay: float

property headers: dict[str, Any] | None

property extra_options: dict[str, Any] | None

property auth: aiohttp.BasicAuth | None

async run(endpoint=None, data=None, json=None, headers=None, extra_options=None)

Perform an asynchronous HTTP request call.

Parameters:

• endpoint (str | None) – Endpoint to be called, i.e. resource/v1/query?.

• data (dict[str, Any] | str | None) – Payload to be uploaded or request parameters.

• json (dict[str, Any] | str | None) – Payload to be uploaded as JSON.

• headers (dict[str, Any] | None) – Additional headers to be passed through as a dict.

• extra_options (dict[str, Any] | None) – Additional kwargs to pass when creating a request. For example, run(json=obj) is passed as aiohttp.ClientSession().get(json=obj).

class airflow.providers.http.hooks.http.HttpAsyncHook(method='POST', http_conn_id=default_conn_name, auth_type=aiohttp.BasicAuth, retry_limit=3, retry_delay=1.0)

Bases: airflow.providers.common.compat.sdk.BaseHook

Interact with HTTP servers asynchronously.

Parameters:

• method (str) – the API method to be called

• http_conn_id (str) – http connection id that has the base API url i.e https://www.google.com/ and optional authentication credentials. Default headers can also be specified in the Extra field in json format.

• auth_type (Any) – The auth type for the service

• retry_limit (int) – Maximum number of times to retry this job if it fails (default is 3)

• retry_delay (float) – Delay between retry attempts (default is 1.0)

conn_name_attr = 'http_conn_id'

default_conn_name = 'http_default'

conn_type = 'http'

hook_name = 'HTTP'

http_conn_id = 'http_default'

method = ''

base_url: str = ''

auth_type: Any

retry_limit = 3

retry_delay = 1.0

async config()

async session(method=None)

Create an AsyncHttpSession bound to a single aiohttp.ClientSession.

Airflow connection resolution happens exactly once here.

Parameters:

method (str | None) – Optional HTTP method to be used for requests made by the returned session. If provided, this value overrides the hook’s configured default method.

async run(session=None, endpoint=None, data=None, json=None, headers=None, extra_options=None)

Perform an asynchronous HTTP request call.

Parameters:

• session (aiohttp.ClientSession | None) – aiohttp.ClientSession

• endpoint (str | None) – Endpoint to be called, i.e. resource/v1/query?.

• data (dict[str, Any] | str | None) – Payload to be uploaded or request parameters.

• json (dict[str, Any] | str | None) – Payload to be uploaded as JSON.

• headers (dict[str, Any] | None) – Additional headers to be passed through as a dict.

• extra_options (dict[str, Any] | None) – Additional kwargs to pass when creating a request. For example, run(json=obj) is passed as aiohttp.ClientSession().get(json=obj).

Previous Next

Was this entry helpful?