Pipelines

A Pipeline moves data from a Source system to a Destination system. Your Source can be a database, such as PostgreSQL, and the Destination system can be a data warehouse, such as Snowflake.

You can connect only one Source and Destination in a Pipeline. However, multiple Pipelines may load to the same Destination.

Creating a Pipeline in Hevo involves configuring the following:

• Source: You can configure a new Source by providing the required connection parameters or use an existing one. Read Sources for the various Sources that Hevo integrates with.

• Destination: You can use an existing Destination or specify the necessary connection parameters to configure a new one. At this time, you can replicate data from a Source database only to a Snowflake Destination. Read Destinations.

• Pipeline: In the Pipeline configuration, you can define the way in which you want Hevo to load data to your Destination. Here, you can specify how your Destination schema should evolve and also how often the data from the Source should be synchronized with the Destination.

• Objects: Hevo allows you to select the objects from which you want to replicate data. You can also define a primary key for your object here if you have opted to load data to the Destination with the Merge data loading policy.

---

• Articles in this section
  • Familiarizing with the Pipelines UI
  • Creating an Edge Pipeline
  • Working with Edge Pipelines
    • Editing Edge Pipelines
    • On-Demand Syncing Data in Edge Pipelines
    • Activating Edge Pipelines
    • Deleting Edge Pipelines
  • Handling Deletes in Log-based Pipelines
  • Schema Mapper
    • Limits on the Number of Destination Columns
  • Object and Schema Management
    • Familiarizing with the Object Configuration UI
    • Managing Objects in Pipelines
    • Pipeline Schema Management
  • Handling Failed Events in a Pipeline
  • Pipeline Alerts
  • Pipeline Job History (Edge)
    • Job Types (Edge)
    • Job and Object Statuses (Edge)
    • Viewing Pipeline Job Details (Edge)
    • Offset and Latency of Jobs and Objects (Edge)
    • Canceling Jobs (Edge)
    • Downloading Session Logs (Edge)