airflow.hooks.http_hook
¶
Module Contents¶
-
class
airflow.hooks.http_hook.
HttpHook
(method='POST', http_conn_id='http_default')[source]¶ Bases:
airflow.hooks.base_hook.BaseHook
Interact with HTTP servers.
- Parameters
http_conn_id (str) – connection that has the base API url i.e https://www.google.com/ and optional authentication credentials. Default headers can also be specified in the Extra field in json format.
method (str) – the API method to be called
-
get_conn
(self, headers=None)[source]¶ Returns http session for use with requests
- Parameters
headers (dict) – additional headers to be passed through as a dictionary
-
run
(self, endpoint, data=None, headers=None, extra_options=None, **request_kwargs)[source]¶ Performs the request
- Parameters
endpoint (str) – the endpoint to be called i.e. resource/v1/query?
data (dict) – payload to be uploaded or request parameters
headers (dict) – additional headers to be passed through as a dictionary
extra_options (dict) – additional options to be used when executing the request i.e. {‘check_response’: False} to avoid checking raising exceptions on non 2XX or 3XX status codes
**request_kwargs – Additional kwargs to pass when creating a request. For example,
run(json=obj)
is passed asrequests.Request(json=obj)
-
check_response
(self, response)[source]¶ Checks the status code and raise an AirflowException exception on non 2XX or 3XX status codes
- Parameters
response (requests.response) – A requests response object
-
run_and_check
(self, session, prepped_request, extra_options)[source]¶ Grabs extra options like timeout and actually runs the request, checking for the result
- Parameters
session (requests.Session) – the session to be used to execute the request
prepped_request (session.prepare_request) – the prepared request generated in run()
extra_options (dict) – additional options to be used when executing the request i.e. {‘check_response’: False} to avoid checking raising exceptions on non 2XX or 3XX status codes
-
run_with_advanced_retry
(self, _retry_args, *args, **kwargs)[source]¶ Runs Hook.run() with a Tenacity decorator attached to it. This is useful for connectors which might be disturbed by intermittent issues and should not instantly fail.
- Parameters
_retry_args (dict) – Arguments which define the retry behaviour. See Tenacity documentation at https://github.com/jd/tenacity
hook = HttpHook(http_conn_id='my_conn',method='GET') retry_args = dict( wait=tenacity.wait_exponential(), stop=tenacity.stop_after_attempt(10), retry=requests.exceptions.ConnectionError ) hook.run_with_advanced_retry( endpoint='v1/test', _retry_args=retry_args )