Why does my Destination have records that are deleted in the Source?

When a record is deleted in the Source, Hevo updates the metadata column __hevo__marked_deleted, for it to TRUE in the Destination. This means that Hevo does not permanently delete the record in the Destination. Instead, it performs a soft delete. You can identify the records deleted in the Source using the __hevo__marked_deleted field. Refer to the sections below for Hevo’s behavior in Pipelines created with the various ingestion modes:

Log-based Pipelines

Hevo captures deletes for Pipelines created using log-based ingestion mode by changing the value of the __hevo__marked_deleted column to TRUE for the corresponding deleted record in the Source. The way Hevo replicates deletes to the Destination table depends on whether the Source table has a primary key:

• Source table with a primary key

  

• Source table without a primary key

  

Table and Custom SQL mode Pipelines

For Pipelines created using Table or Custom SQL ingestion modes, data is fetched using SQL queries. As a result, deletes are not captured and the __hevo__marked_deleted field is not created in the Destination table. After you delete a record in the Source table, the way Hevo loads data in the next Pipeline run depends on whether the Source table has a primary key:

• Source table with a primary key

  

• Source table without a primary key

  

  Note: In the example above, as the query mode for the object is Full Load, Hevo appends rows at the end of the Destination table after the next ingestion run. This behavior differs based on the query mode selected for your Source object.