Schema Mapper
Schema Mapper defines how data fetched from your Source is structured and stored in the Destination. It controls the mapping between Source Event Types and Destination tables, and manages how fields, data types, and table structures are created or updated in the Destination.
Schema Mapper is accessible from the Pipeline view and can be configured at any point after the Pipeline is created. You can use Schema Mapper to do the following:
-
Automatically map Source Event Types to Destination tables: Use the Auto Mapping feature to automate mappings between Source Event Types and Destination tables. When enabled, Auto Mapping:
-
Creates and manages all mappings automatically.
-
Creates tables with compatible and optimal data types in the Destination if they do not already exist.
-
Marks incompatible incoming Events as Failed for your review.
You can enable Auto Mapping for the entire Pipeline or for specific objects.
-
-
Manually map an incoming Event Type to a Destination table: Map each field of a Source Event Type to a matching column in a Destination table of your choice. You can modify the mappings or select a different Destination table at any time.
-
Omit unwanted fields from being stored in the Destination table: Skip fields you do not want to load by leaving them unmapped. Unmapped fields are not loaded to the Destination. This is useful for excluding sensitive information such as personal details and financial records, or large text fields such as descriptions and comments.
-
Flatten nested objects into a de-normalized Destination table: When incoming data contains nested objects, Schema Mapper suggests a flat, de-normalized structure for the Destination table. This improves query performance and reduces schema complexity.
-
Compress long names of Events and Event Types: Schema Mapper automatically creates shorter, meaningful names for Events and Event Types that exceed the character limit for table and column names in the Destination.
-
Delete existing mappings and create new Destination tables: You can delete an existing mapping and create a new Destination table directly from Schema Mapper, then map a Source Event Type to it.
In this Section
The following interactive diagram illustrates the features and actions offered by the Schema Mapper and provides a summary view of the section’s content. Click on any item in the diagram to view the related information.