Creating your First Pipeline

Hevo provides a quick 4-step process to set up a data Pipeline.

A simple Pipeline can be set up in less than 3 minutes whereas a complex one could take you around 30 minutes. Post Pipeline setup, Hevo moves the data to the Destination in two steps:

1. Ingests the historical data from the Source in batches and loads them to the Destination.

2. Fetches the new, incremental data from the Source in micro-batches at frequent intervals (depending on Source configuration) and applies them to the Destination.

To get started with creating Pipelines in Hevo, you essentially need:

• An active Hevo account. You can start with a 14-day full feature, free trial account.

• You are assigned the Team Administrator, Team Collaborator, or Pipeline Administrator role in Hevo to create the Pipeline.

• Access to the Source system (Integration) where your data resides.

• A Destination (a database or data warehouse) to which the data must be replicated. If you do not have a Destination, you can use the Hevo-managed Google BigQuery Destination.

---

Types of Pipelines in Hevo

In Hevo, you can primarily create the following categories of Pipelines depending on the type of Source you select:

• Database Pipeline

• SaaS Pipeline

• Webhook Pipeline

Starting Release 2.21, Hevo provides an option to create streaming Pipelines with the log-based ingestion mode or convert existing log-based Pipelines to streaming ones for your database Sources. A streaming Pipeline ingests and loads data continuously.

To enable streaming for your Pipeline:

1. Click CONTACT SALES in the Pipeline Ingestion Schedule section while creating your Pipeline.

   

2. Click CONTINUE. Hevo creates your Pipeline with a 30-minute ingestion frequency till the Hevo Sales team enables streaming for the Pipeline.

   

Once the Hevo Sales team has enabled streaming for your Pipeline, Hevo will ingest and load data continuously. For such Pipelines, you can switch between frequency-based ingestion and streaming. If you change the ingestion mode, any data read by the Pipeline in the frequency mode is loaded to the Destination before switching to the streaming mode, and vice versa.