Why is there a delay in my Pipeline?

Hevo is built on real-time data ingestion architecture. This means that Hevo ingests data in real-time and writes to the Destination as soon as possible. Hevo’s architecture allows it to scale horizontally whenever it detects a higher volume of Events being ingested through the Pipelines. Still, there are situations where you might see a delay in your Pipelines, depending on your Pipeline configuration such as:

• Ingestion and Loading Frequencies of your Pipeline, which affect how often the data is replicated from your Source to the Destination. You must configure a high ingestion and loading frequency if you need your Source data to be updated quickly in the Destination. Read Ingestion and Loading Frequency.
• Applying complex Transformations on your Source data, which can cause delays in loading the data to the Destination.

• Replay of a large number of failed Events in one or more of your Pipelines. Replayed Events are fed back to the Pipelines and share the same resources that are used by the Pipeline itself. In this scenario, you can stop replaying the Events in the Pipeline from the Pipeline Overview page.

  Note: Pipelines across different accounts in Hevo do not affect each other.

• Slow Destination. In cases of Destinations where data upload is not done through files, for example, MySQL and Postgres, the Pipeline may experience delays. This happens because Hevo is not able to write data into the Destination as fast as it is ingesting it from the Source. If you think the slowness in the Destination is a temporary situation, you may wait till it gets resolved, else you should upgrade the hardware configuration of the Destination to allows it to accept a higher rate of writes.