Configure SendGrid as a Source in your Pipeline

Perform the following steps to configure your SendGrid Source:

1. Click Pipelines in the Navigation Bar.

2. Click + Create Pipeline in the Pipelines List View.

3. On the Select Source Type page, select SendGrid.

4. On the Select Destination Type page, select the type of Destination you want to use.

5. In the Configure Source screen, specify the following:

   

   • Source Name: A unique name for your Source, not exceeding 255 characters. For example, SendGrid Source.

   • In the Connect to your SendGrid account section, specify the following:

     • API Key: The API key that you created in your SendGrid account.

6. Click Test & Continue to test the connection to your SendGrid Source. Hevo checks whether SendGrid accepts the API key and whether the key has at least one of the permissions required by Hevo. Once the test is successful, you can proceed to set up your Destination.

---

Data Replication

Hevo replicates data for all the objects selected on the Configure Objects page during Pipeline creation. By default, all supported objects and their available fields are selected. However, you can modify this selection while creating or editing the Pipeline.

Hevo ingests the following types of data from your Source objects:

• Historical Data: The first run of the Pipeline ingests all the data of the selected objects, except for the following:

  • Mailbox Provider Stat: Hevo ingests the statistics of the last three years, as SendGrid provides these statistics only for this period. Hevo first ingests the statistics of the last seven days, and then the older statistics.

  • Browser Stat, Client Stat, Country Location Stat, Device Stat, and Global Location Stat: Hevo ingests the statistics of the last seven days.

  • Event and Category: Hevo ingests only the events that SendGrid sends after the Pipeline is created. Read Event Webhook.

• Incremental Data: Once the historical load is complete, Hevo ingests data as per the sync frequency, as follows:

  • Statistics objects, such as Mailbox Provider Stat: Hevo ingests the statistics from the day of the previous run until the current day.

  • Event and Category: Hevo ingests the events that SendGrid has sent since the previous run.

  • All other objects: Hevo ingests the entire data in each run.

You can resync any object. When you resync the Contact, Single Send, or Transactional Template object, Hevo also resyncs its child objects. When you resync the Event object, Hevo ingests only the events that the webhook received in the last 24 hours, and does not resync the Category object. Read Event Webhook.

SendGrid limits the number of API requests that each endpoint can receive in a period. Read Rate limits. If a limit is reached, Hevo waits for the time that SendGrid specifies and retries the request. If the retry fails, the affected objects fail, and Hevo ingests their data in the next run.

Note: You can create a Pipeline with this Source only using the Merge load mode. The Append mode is not supported for this Source.

Event Webhook

Hevo ingests the Event and Category objects through the SendGrid Event Webhook. When the Pipeline runs, Hevo adds an Event Webhook named Hevo Data Pipeline to your SendGrid account, enables its signature verification, and selects all the engagement and deliverability events, such as Delivered, Bounced, Opened, and Clicked. If a webhook with the same URL already exists, Hevo uses it. The webhook URL is displayed in the Pipeline Setup tab of your Pipeline, and you can view the webhook in your SendGrid account under Settings > Mail Settings > Event Webhook.

To add the webhook, your API key must have the Mail Settings > Event Notification: Full Access permission. SendGrid limits the number of Event Webhooks in an account as per your plan. If this limit is reached, Hevo cannot add the webhook, and marks the Event and Category objects as Inaccessible. Hevo continues to ingest data for the other objects.

Hevo retains the events that the webhook receives for 24 hours. If the Pipeline is paused or fails for more than 24 hours, the events received during that period are lost, and you cannot recover them by resyncing the Pipeline or the objects. Therefore, resolve any Pipeline failures promptly.

Note: Each Pipeline that you create with this Source adds its own webhook, and Hevo does not remove the webhook when you delete the Pipeline. Therefore, when you delete a Pipeline, delete its webhook in your SendGrid account. Otherwise, the webhook continues to use one of the webhook slots of your account, and SendGrid continues to send events to it. Read Remove the webhook of a deleted Pipeline.

---

Schema and Primary Keys

Hevo uses the following schema to upload the records in the Destination. For a detailed view of the objects, fields, and relationships, click the ERD.

---

Data Model

The following is the list of tables (objects) that are created at the Destination when you run the Pipeline:

---