airflow.providers.http.hooks.http

Module Contents

Classes

HttpHook

Interact with HTTP servers.

HttpAsyncHook

Interact with HTTP servers asynchronously.
class airflow.providers.http.hooks.http.HttpHook(method='POST', http_conn_id=default_conn_name, auth_type=None, tcp_keep_alive=True, tcp_keep_alive_idle=120, tcp_keep_alive_count=20, tcp_keep_alive_interval=30)[source]

Bases: airflow.hooks.base.BaseHook

Interact with HTTP servers.

Parameters

  • method (str) – the API method to be called

  • http_conn_id (str) – http connection that has the base API url i.e https://www.google.com/ and optional authentication credentials. Default headers can also be specified in the Extra field in json format.

  • auth_type (Any) – The auth type for the service

  • tcp_keep_alive (bool) – Enable TCP Keep Alive for the connection.

  • tcp_keep_alive_idle (int) – The TCP Keep Alive Idle parameter (corresponds to socket.TCP_KEEPIDLE).

  • tcp_keep_alive_count (int) – The TCP Keep Alive count parameter (corresponds to socket.TCP_KEEPCNT)

  • tcp_keep_alive_interval (int) – The TCP Keep Alive interval parameter (corresponds to socket.TCP_KEEPINTVL)

  • auth_args – extra arguments used to initialize the auth_type if different than default HTTPBasicAuth

property auth_type[source]
conn_name_attr = 'http_conn_id'[source]
default_conn_name = 'http_default'[source]
conn_type = 'http'[source]
hook_name = 'HTTP'[source]
get_conn(headers=None)[source]

Create a Requests HTTP session.

Parameters

headers (dict[Any, Any] | None) – additional headers to be passed through as a dictionary

run(endpoint=None, data=None, headers=None, extra_options=None, **request_kwargs)[source]

Perform the request.

Parameters

  • endpoint (str | None) – the endpoint to be called i.e. resource/v1/query?

  • data (dict[str, Any] | str | None) – payload to be uploaded or request parameters

  • headers (dict[str, Any] | None) – additional headers to be passed through as a dictionary

  • extra_options (dict[str, Any] | None) – additional options to be used when executing the request i.e. {‘check_response’: False} to avoid checking raising exceptions on non 2XX or 3XX status codes

  • request_kwargs (Any) – Additional kwargs to pass when creating a request. For example, run(json=obj) is passed as requests.Request(json=obj)

check_response(response)[source]

Check the status code and raise on failure.

Parameters

response (requests.Response) – A requests response object.

Raises

AirflowException – If the response contains a status code not in the 2xx and 3xx range.

run_and_check(session, prepped_request, extra_options)[source]

Grab extra options, actually run the request, and check the result.

Parameters

  • session (requests.Session) – the session to be used to execute the request

  • prepped_request (requests.PreparedRequest) – the prepared request generated in run()

  • extra_options (dict[Any, Any]) – additional options to be used when executing the request i.e. {'check_response': False} to avoid checking raising exceptions on non 2XX or 3XX status codes

run_with_advanced_retry(_retry_args, *args, **kwargs)[source]

Run the hook with retry.

This is useful for connectors which might be disturbed by intermittent issues and should not instantly fail.

Parameters

_retry_args (dict[Any, Any]) – Arguments which define the retry behaviour. See Tenacity documentation at https://github.com/jd/tenacity

hook = HttpHook(http_conn_id="my_conn", method="GET")
retry_args = dict(
    wait=tenacity.wait_exponential(),
    stop=tenacity.stop_after_attempt(10),
    retry=tenacity.retry_if_exception_type(Exception),
)
hook.run_with_advanced_retry(endpoint="v1/test", _retry_args=retry_args)
url_from_endpoint(endpoint)[source]

Combine base url with endpoint.

test_connection()[source]

Test HTTP Connection.

class airflow.providers.http.hooks.http.HttpAsyncHook(method='POST', http_conn_id=default_conn_name, auth_type=aiohttp.BasicAuth, retry_limit=3, retry_delay=1.0)[source]

Bases: airflow.hooks.base.BaseHook

Interact with HTTP servers asynchronously.

Parameters

  • method (str) – the API method to be called

  • http_conn_id (str) – http connection id that has the base API url i.e https://www.google.com/ and optional authentication credentials. Default headers can also be specified in the Extra field in json format.

  • auth_type (Any) – The auth type for the service

conn_name_attr = 'http_conn_id'[source]
default_conn_name = 'http_default'[source]
conn_type = 'http'[source]
hook_name = 'HTTP'[source]
async run(endpoint=None, data=None, json=None, headers=None, extra_options=None)[source]

Perform an asynchronous HTTP request call.

Parameters

  • endpoint (str | None) – Endpoint to be called, i.e. resource/v1/query?.

  • data (dict[str, Any] | str | None) – Payload to be uploaded or request parameters.

  • json (dict[str, Any] | str | None) – Payload to be uploaded as JSON.

  • headers (dict[str, Any] | None) – Additional headers to be passed through as a dict.

  • extra_options (dict[str, Any] | None) – Additional kwargs to pass when creating a request. For example, run(json=obj) is passed as aiohttp.ClientSession().get(json=obj).

Was this entry helpful?