Working with Datasets (Edge)
Activate is currently available for Early Access. Please contact your Hevo account executive or the Support team to enable it for your team. Alternatively, request for early access to try out one or more such features.
A dataset defines the data that Activate reads from your Source. It is a saved definition of the rows and columns you want to send to your Target applications. Read Familiarizing with the Datasets UI for the pages you use to view and manage your datasets.
Dataset Types
Activate supports two dataset types: SQL Query and Table.
An SQL Query dataset lets you define the data to send by writing an SQL query. You can filter rows, select and rename columns, and combine data from multiple tables.
A Table dataset sends data directly from a selected table. You select the database, schema, and table, and all columns and rows from that table are available for the Activation.
The following table compares the two dataset types:
| Criteria | SQL Query | Table |
|---|---|---|
| How you define it | You write an SQL query. | You select a database, schema, and table. |
| Filtering rows | Supported. Use a WHERE clause to send only the rows you want. |
Not supported. All rows in the table are sent. |
| Selecting columns | Supported. Name only the columns you want in the SELECT list. |
Not supported. You cannot restrict the columns in the dataset, so all of them reach the mapping step. |
| Renaming columns | Supported. Use column aliases to match the field names in your application. | Not supported. Columns keep their table names. |
| Combining tables | Supported. Use joins to bring data together from several tables. | Not supported. A Table dataset reads from one table. |
Create an SQL Query dataset when you need to shape the data before it is sent. For example, you may want to send only the customers who signed up in the last 30 days, or combine a customer table with a subscription table to send both sets of information to your Target application.
Create a Table dataset when the table already holds exactly what you want to send. For example, the tables built by Transformations have already been shaped for a specific purpose.
Dataset Reuse
A dataset is stored separately from an Activation, so the same dataset can be used in multiple Activations. Every Activation that uses it reads the same rows, while each one chooses its own Target object, sync behavior, and schedule. This means you write and validate a query once and reuse it wherever those rows are needed.
Note: Any change to the dataset applies to every Activation that uses it.
Example
Suppose a dataset named Active_Customers returns the customers who currently hold a subscription. Two Activations can read it at the same time:
| Activation | Target Object | Sync Behavior | Result |
|---|---|---|---|
| Sync Customers | Contacts | Insert or Update | Creates a contact for each new customer and updates the contacts that already exist. |
| Refresh Companies | Companies | Update Only | Updates the companies that already exist in HubSpot and skips the rows that have no match. |
Both Activations use the same set of customers, so you maintain one dataset instead of creating a separate query for each Target application.
Dataset Considerations
A dataset can be used by multiple Activations, so changes to the dataset can affect all of them. You must consider the following when creating or updating a dataset:
-
Column names: The columns a dataset returns are the values you map to fields in your Target application. If you rename a column in an SQL Query, any existing mapping that references the original column name is no longer valid. You cannot change a field mapping after an Activation is created, so you must delete each affected Activation and create it again with the updated mapping.
-
Removing a column: An Activation’s field mappings must continue to reference the columns returned by the dataset. If you remove a column that an Activation maps to, that Activation cannot sync the field. You must delete the affected Activation and create it again to map the remaining columns.
-
Column changes: Activate uses bookkeeping to compare each run with the previous result and identify the rows that changed. If you add, remove, rename, or reorder a column, or change its type, Activate cannot use the previous result for comparison. The next run of every Activation using the dataset therefore sends all rows again. For an Activation using the Mirror sync behavior, Activate cannot identify records that the dataset no longer returns. As a result, those records are not deleted in that run.
-
Source selection: Every table referenced by a dataset must be accessible through the Source selected for that dataset. The Source cannot be changed after the dataset is created. If you need to read the same data through a different Source, create a new dataset. A dataset also cannot be deleted while an Activation uses it. Read Deleting a Dataset for the steps.