Salesforce To S3 Operator

Overview

Use the SalesforceToS3Operator to execute a Salesforce query to fetch data and upload to S3. The results of the query are initially written to a local, temporary directory and then uploaded to an S3 bucket.

Extract Customer Data from Salesforce

The following example demonstrates a use case of extracting customer data from a Salesforce instance and upload to a "landing" bucket in S3.

airflow/providers/amazon/aws/example_dags/example_salesforce_to_s3.pyView Source

    upload_salesforce_data_to_s3_landing = SalesforceToS3Operator(
        task_id="upload_salesforce_data_to_s3",
        salesforce_query="SELECT Id, Name, Company, Phone, Email, LastModifiedDate, IsActive FROM Customers",
        s3_bucket_name="landing-bucket",
        s3_key=f"{BASE_PATH}/{FILE_NAME}",
        salesforce_conn_id="salesforce",
        replace=True,
    )

Reference

This operator uses the SalesforceHook to interact with Salesforce. This hook is built with functionality from the Simple Salesforce package.

For further information, review the Simple Salesforce Documentation.

Was this entry helpful?