Querying Audit Tables

You can query the Audit Tables to view details of the different data replication jobs performed by your Pipelines. For Pipelines, you can view details such as the Source and Destination names, the type, duration, and status of each job, and the type of data being loaded. Similarly, for the objects, you can view details such as the Source and Destination table names, job duration, and completion status, along with the number of Events ingested versus loaded or failed. To do this:

1. In the Navigation Bar, click Destinations.

2. In the Destination List View, search for and click the Destination where your Audit Tables are created.

   

3. In the left navigation pane, click Workbench.

   

4. In the Schema section, click the Search ( ) icon and search for the table you want to query. For example, job_details.

   

5. Hover over the name of the table and click the copy icon corresponding to it.

   Repeat this step to retrieve all table names that you want to use in your SQL query.

   

6. In the SQL Query section, enter the query that you want to execute on the Audit Tables and click RUN QUERY. Refer to section, Suggested Queries to know about some queries that can help you analyze the logs present in the tables.

   Note: Although Hevo does not charge you for querying the Audit Tables from the SQL workbench, the Destination may charge you for the queries. Refer to your respective Destination for such costs.

   

Suggested Queries

Depending on your use case, you can execute some of the following queries on the Audit Tables:

Note: Replace the placeholder values in the commands below with your own. For example, <source_type> with STRIPE.

Scenario 1

When you want to know the number of tasks that Hevo has executed for a Pipeline:

select * from job_details where pipeline_id = <pipeline_seq_number>

Note: The <pipeline_seq_number> is the sequential number assigned to the Pipeline at the time of creation. Read Pipeline List View to know where you can find it.

Scenario 2

When you want to know the number of historical Events that Hevo has loaded to your Destination across all Pipelines:

select sum(output_rows) from object_details where stage = 'LOAD' and mode = 'HISTORICAL'

Scenario 3

When you want to know the number of Events ingested from a specific Source across all Pipelines:

select sum(output_rows) from object_details where stage = 'INGESTION' and job_id in (select job_id from job_details where source = <source_type>)