airflow.providers.http.hooks.http

Module Contents

Classes

HttpHook

Interact with HTTP servers.
class airflow.providers.http.hooks.http.HttpHook(method='POST', http_conn_id=default_conn_name, auth_type=HTTPBasicAuth)[source]

Bases: airflow.hooks.base.BaseHook

Interact with HTTP servers.

Parameters

  • method (str) -- the API method to be called

  • http_conn_id (str) -- http connection that has the base API url i.e https://www.google.com/ and optional authentication credentials. Default headers can also be specified in the Extra field in json format.

  • auth_type (Any) -- The auth type for the service

conn_name_attr = http_conn_id[source]
default_conn_name = http_default[source]
conn_type = http[source]
hook_name = HTTP[source]
get_conn(self, headers=None)[source]

Returns http session for use with requests

Parameters

headers (Optional[Dict[Any, Any]]) -- additional headers to be passed through as a dictionary

run(self, endpoint=None, data=None, headers=None, extra_options=None, **request_kwargs)[source]

Performs the request

Parameters

  • endpoint (Optional[str]) -- the endpoint to be called i.e. resource/v1/query?

  • data (Optional[Union[Dict[str, Any], str]]) -- payload to be uploaded or request parameters

  • headers (Optional[Dict[str, Any]]) -- additional headers to be passed through as a dictionary

  • extra_options (Optional[Dict[str, Any]]) -- additional options to be used when executing the request i.e. {'check_response': False} to avoid checking raising exceptions on non 2XX or 3XX status codes

  • request_kwargs (Any) -- Additional kwargs to pass when creating a request. For example, run(json=obj) is passed as requests.Request(json=obj)

check_response(self, response)[source]

Checks the status code and raise an AirflowException exception on non 2XX or 3XX status codes

Parameters

response (requests.Response) -- A requests response object

run_and_check(self, session, prepped_request, extra_options)[source]

Grabs extra options like timeout and actually runs the request, checking for the result

Parameters

  • session (requests.Session) -- the session to be used to execute the request

  • prepped_request (requests.PreparedRequest) -- the prepared request generated in run()

  • extra_options (Dict[Any, Any]) -- additional options to be used when executing the request i.e. {'check_response': False} to avoid checking raising exceptions on non 2XX or 3XX status codes

run_with_advanced_retry(self, _retry_args, *args, **kwargs)[source]

Runs Hook.run() with a Tenacity decorator attached to it. This is useful for connectors which might be disturbed by intermittent issues and should not instantly fail.

Parameters

_retry_args (Dict[Any, Any]) -- Arguments which define the retry behaviour. See Tenacity documentation at https://github.com/jd/tenacity

hook = HttpHook(http_conn_id="my_conn", method="GET")
retry_args = dict(
    wait=tenacity.wait_exponential(),
    stop=tenacity.stop_after_attempt(10),
    retry=requests.exceptions.ConnectionError,
)
hook.run_with_advanced_retry(endpoint="v1/test", _retry_args=retry_args)
url_from_endpoint(self, endpoint)[source]

Combine base url with endpoint

test_connection(self)[source]

Test HTTP Connection

Was this entry helpful?