airflow.providers.http.hooks.http
¶
Module Contents¶
Classes¶
Interact with HTTP servers. |
- class airflow.providers.http.hooks.http.HttpHook(method='POST', http_conn_id=default_conn_name, auth_type=HTTPBasicAuth)[source]¶
Bases:
airflow.hooks.base.BaseHook
Interact with HTTP servers.
- Parameters
method (str) -- the API method to be called
http_conn_id (str) -- http connection that has the base API url i.e https://www.google.com/ and optional authentication credentials. Default headers can also be specified in the Extra field in json format.
auth_type (Any) -- The auth type for the service
- get_conn(self, headers=None)[source]¶
Returns http session for use with requests
- Parameters
headers (Optional[Dict[Any, Any]]) -- additional headers to be passed through as a dictionary
- run(self, endpoint=None, data=None, headers=None, extra_options=None, **request_kwargs)[source]¶
Performs the request
- Parameters
endpoint (Optional[str]) -- the endpoint to be called i.e. resource/v1/query?
data (Optional[Union[Dict[str, Any], str]]) -- payload to be uploaded or request parameters
headers (Optional[Dict[str, Any]]) -- additional headers to be passed through as a dictionary
extra_options (Optional[Dict[str, Any]]) -- additional options to be used when executing the request i.e. {'check_response': False} to avoid checking raising exceptions on non 2XX or 3XX status codes
request_kwargs (Any) -- Additional kwargs to pass when creating a request. For example,
run(json=obj)
is passed asrequests.Request(json=obj)
- check_response(self, response)[source]¶
Checks the status code and raise an AirflowException exception on non 2XX or 3XX status codes
- Parameters
response (requests.Response) -- A requests response object
- run_and_check(self, session, prepped_request, extra_options)[source]¶
Grabs extra options like timeout and actually runs the request, checking for the result
- Parameters
session (requests.Session) -- the session to be used to execute the request
prepped_request (requests.PreparedRequest) -- the prepared request generated in run()
extra_options (Dict[Any, Any]) -- additional options to be used when executing the request i.e.
{'check_response': False}
to avoid checking raising exceptions on non 2XX or 3XX status codes
- run_with_advanced_retry(self, _retry_args, *args, **kwargs)[source]¶
Runs Hook.run() with a Tenacity decorator attached to it. This is useful for connectors which might be disturbed by intermittent issues and should not instantly fail.
- Parameters
_retry_args (Dict[Any, Any]) -- Arguments which define the retry behaviour. See Tenacity documentation at https://github.com/jd/tenacity
hook = HttpHook(http_conn_id="my_conn", method="GET") retry_args = dict( wait=tenacity.wait_exponential(), stop=tenacity.stop_after_attempt(10), retry=requests.exceptions.ConnectionError, ) hook.run_with_advanced_retry(endpoint="v1/test", _retry_args=retry_args)